Overview

Schema Version

https://atlas.rsna.org/schemas/2025-11/model.json

Name

ReportGuidedNet

Link

https://github.com/LisongDai/ReportGuidedNet

Indexing

Keywords: Deep learning, Vision-language model, Contrastive learning, Radiology report, Weak annotation, Lesion detection, Attention maps, Multiclass diagnosis, Brain MRI, Generalizability

Content: MR, NR

Author(s)

Lisong Dai

Jiayu Lei

Fenglong Ma

Zheng Sun

Haiyan Du

Houwang Zhang

Jingxuan Jiang

Jianyong Wei

Dan Wang

Guang Tan

Xinyu Song

Jinyu Zhu

Qianqian Zhao

Songtao Ai

Ai Shang

Zhaohui Li

Ya Zhang

Yuehua Li

Organization(s)

Institute of Diagnostic and Interventional Radiology, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Clinical Research Center, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Shanghai AI Laboratory

School of Computer Science and Technology, University of Science and Technology of China

The Pennsylvania State University College of Information Sciences and Technology

Department of Electrical Engineering, City University of Hong Kong

Department of Radiology, Affiliated Hospital of Nantong University

Department of Radiology, Shanghai Ninth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine

Department of Radiology, Shanghai Public Health Clinical Center

Department of Radiology, Wuhan Hankou Hospital

Cooperative Medianet Innovation Center, Shanghai Jiao Tong University

Version

1.0

License

Text: CC BY 4.0

URL: https://creativecommons.org/licenses/by/4.0/

Funding

Supported by the National Natural Science Foundation of China (8225024), the National Key Research and Development Program of China (2022ZD0160702), and Shanghai Science and Technology Innovation Action Plan (20S31907300).

Ethical review

Multicenter retrospective study approved by local ethics committee (approval no. 2023-KY-082[K]); waiver of informed consent.

Date

Updated: 2024-11-01

Published: 2024-10-09

Created: 2023-11-17

References

[1] Dai L, Lei J, Ma F, Sun Z, Du H, Zhang H, Jiang J, Wei J, Wang D, Tan G, Song X, Zhu J, Zhao Q, Ai S, Shang A, Li Z, Zhang Y, Li Y. "Boosting Deep Learning for Interpretable Brain MRI Lesion Detection through the Integration of Radiology Report Information". Radiology: Artificial Intelligence. 2024;6(6):e230520.. 2024-11-01. doi:10.1148/ryai.230520. PMID: 39377669. PMCID: PMC11605145.

Model

Architecture

Vision–language framework with shared multisequence MRI image encoder, contrastive learning to integrate structured radiology report text features from a knowledge-enhanced pretrained text encoder, and a transformer decoder to produce multiclass diagnoses; class activation mapping and attention module for 3D attention maps.

Availability

Open-source code: https://github.com/LisongDai/ReportGuidedNet

Clinical benefit

Assists detection and classification of brain lesions on noncontrast MRI, improves diagnostic accuracy and generalizability, and provides interpretable attention maps to support radiologist decision-making.

Clinical workflow phase

Clinical decision support systems; radiologist reading/interpretation assistance.

Degree of automation

Assistive decision support; generates automated multiclass diagnoses and attention maps for radiologist review.

Indications for use

Adult patients (≥18 years) undergoing noncontrast brain MRI in hospital settings; identification of 14 brain diseases plus normal brain status for triage/diagnostic support.

Input

Axial noncontrast brain MRI sequences (T1WI, T2WI, T2 FLAIR, DWI, ADC) and structured textual features mined from corresponding radiology reports (impression excluded).

Instructions

Use paired noncontrast brain MRI sequences and de-identified radiology reports; reports are structured via NLP to extract lesion characteristics; impression section excluded to avoid label leakage. Multicenter data suggest applicability across varying scanners and sites (see paper and supplement for preprocessing).

Limitations

Evaluated on a limited set of diseases; only axial images used; lacks pixel-level lesion segmentation; disease-level diagnosis may not reflect lesion-level localization in multi-lesion diseases; simple NLP techniques may not address all reporting variations; data from tertiary centers with fewer normal cases—generalizability to broader settings requires further validation.

Output

CDEs: RDE164.1, RDE166, RDE2013, RDE1775, RDE1762, RDE172, RDE1788, RDE1787

Description: Multiclass diagnosis among 15 categories (14 brain diseases plus normal brain) and 3D attention maps indicating regions supporting the diagnosis.

Recommendation

Use as an assistive tool to enhance radiologist performance in brain MRI lesion detection and interpretation; not a standalone diagnostic device.

Reproducibility

Public code repository provided; evaluation described with bootstrap resampling (10,000 replicates) and external multicenter testing.

Use

Intended: Detection and diagnosis

Out-of-scope: Diagnosis, Detection and diagnosis

Excluded: Image segmentation

User

Intended: Physician, Radiologist, Subspecialist diagnostic radiologist

Out-of-scope: Layperson

Excluded: Other