Journal of Artificial Intelligence for Medical Sciences

Volume 2, Issue 1-2, June 2021, Pages 21 - 32

TMRGM: A Template-Based Multi-Attention Model for X-Ray Imaging Report Generation

Authors
Xuwen WangORCID, Yu Zhang, Zhen GuoORCID, Jiao Li*, ORCID
Institute of Medical Information and Library, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China

Xuwen Wang and Yu Zhang are co-first authors.

*Corresponding author. Email: li.jiao@imicams.ac.cn
Corresponding Author
Jiao Li
Received 15 November 2020, Accepted 14 April 2021, Available Online 5 May 2021.
DOI
https://doi.org/10.2991/jaims.d.210428.002How to use a DOI?
Keywords
Chest X-ray; Deep learning; Thoracic abnormality recognition; Medical imaging report generation; Attention mechanism; Medical imaging report template
Abstract

The rapid growth of medical imaging data brings heavy pressure to radiologists for imaging diagnosis and report writing. This paper aims to extract valuable information automatically from medical images to assist doctors in chest X-ray image interpretation. Considering the different linguistic and visual characteristics in reports of different crowds, we proposed a template-based multi-attention report generation model (TMRGM) for the healthy individuals and abnormal ones respectively. In this study, we developed an experimental dataset based on the IU X-ray collection to validate the effectiveness of TMRGM model. Specifically, our method achieves the BLEU-1 of 0.419, the METEOR of 0.183, the ROUGE score of 0.280, and the CIDEr of 0.359, which are comparable with the SOTA models. The experimental results indicate that the proposed TMRGM model is able to simulate the reporting process, and there is still much room for improvement in clinical application.

Copyright
© 2021 The Authors. Published by Atlantis Press B.V.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

Journal
Journal of Artificial Intelligence for Medical Sciences
Volume-Issue
2 - 1-2
Pages
21 - 32
Publication Date
2021/05/05
ISSN (Online)
2666-1470
DOI
https://doi.org/10.2991/jaims.d.210428.002How to use a DOI?
Copyright
© 2021 The Authors. Published by Atlantis Press B.V.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - JOUR
AU  - Xuwen Wang
AU  - Yu Zhang
AU  - Zhen Guo
AU  - Jiao Li
PY  - 2021
DA  - 2021/05/05
TI  - TMRGM: A Template-Based Multi-Attention Model for X-Ray Imaging Report Generation
JO  - Journal of Artificial Intelligence for Medical Sciences
SP  - 21
EP  - 32
VL  - 2
IS  - 1-2
SN  - 2666-1470
UR  - https://doi.org/10.2991/jaims.d.210428.002
DO  - https://doi.org/10.2991/jaims.d.210428.002
ID  - Wang2021
ER  -