https://atlas.rsna.org/schemas/2025-11/model.json

RadBERT

https://dx.doi.org/10.1148/ryai.210258

1.0

ude.dscu@nannuhc

Supported by award no. W81XWH–20–1–0693 (log no. DM190543), Accelerating Innovation in Military Medicine Program, Office of the Assistant Secretary of Defense for Health Affairs, Department of Defense; and NSF Career Award no. 1750063.

Institutional review board approval for exemption from informed consent to use radiology reports; de-identified and de-duplicated VA radiology reports were used.

Transformer-based language models (BERT-base and RoBERTa) adapted to radiology via masked language model pretraining on radiology reports.

RadBERT models can be released upon request with a data usage agreement.

Improves performance on radiology NLP tasks (abnormal sentence classification, report coding, report summarization), enabling better follow-up identification, standardized documentation, and reduced radiologist workload.

Workflow optimization and clinical decision support through automated analysis of radiology reports; facilitates follow-up tracking and documentation.

Analysis of de-identified radiology reports to support tasks including sentence-level abnormality detection, assignment of diagnostic/reporting codes (e.g., BI-RADS, Lung-RADS, AAA, etc.), and extractive summarization of findings into impressions; intended for use in radiology departments and healthcare systems with similar report corpora.

De-identified radiology report text (sentences or full reports) from the VA nationwide corpus (2.16–4.42 million reports used for pretraining).

Pretrain RadBERT with masked language modeling on large radiology report corpora (WordPiece tokenization; no next sentence prediction), then fine-tune for downstream tasks. For extractive summarization, sentence embeddings from the pretrained model can be used directly (per cited method) without fine-tuning.

Only BERT-base sized models (≈110M parameters) were trained; BERT-large was not evaluated. Pretraining sizes of ~2M vs ~4M reports showed similar performance; the optimal corpus size within radiology is undetermined. Pretraining focused on VA reports; further specialization by modality/body part was not explored.

Use a radiology-specialized pretrained transformer (RadBERT) for radiology NLP tasks, especially when annotated data for fine-tuning are scarce.

For classification tasks, mean and SD over five runs with different random seeds were reported; significance assessed via bootstrap resampling (10,000 trials).