https://atlas.rsna.org/schemas/2025-11/model.json

Prediction–Saliency Correlation (PSC) for evaluating saliency methods in radiology AI

https://dx.doi.org/10.1148/ryai.220221

1.0

Supported by National Science Foundation (2046708) and National Institutes of Health (R01EB032716).

Retrospective study using fully de-identified public datasets; IRB approval exempt and HIPAA compliant.

Evaluation framework using standard CNN classifiers (DenseNet-121, ResNet-152, ResNet-50) and seven saliency methods (vanilla backpropagation, vanilla BP×image, Grad-CAM, guided-Grad-CAM, integrated gradients, SmoothGrad, XRAI). Includes assessment of a commercial black-box prototype.

Data used: CheXpert (https://stanfordmlgroup.github.io/competitions/chexpert) and a brain tumor MRI dataset (https://www.kaggle.com/datasets/masoudnickparvar/brain-tumor-mri-dataset).

Provides a quantitative method (PSC) to assess trustworthiness of saliency explanations in medical AI, potentially informing safer interpretation and deployment of AI outputs.

Research and validation; methodology to support evaluation of AI explainability prior to clinical deployment.

Analytical/evaluation tool; does not automate clinical decisions; quantifies agreement between model prediction changes and saliency map changes.

Quantitative evaluation of the sensitivity and robustness of saliency-based explanations for AI models using medical images (e.g., chest radiographs and brain MRI) in research settings.

Medical images (frontal chest radiographs from CheXpert; brain MRI images from a public dataset); model predictions and saliency maps from various saliency methods or a commercial prototype.

Apply defined perturbation strategies to create prediction-changing (for sensitivity) or saliency-changing (for robustness) adversarial images; compute PSC as Pearson correlation between prediction and saliency changes (measured via Jensen–Shannon divergence); assess AUC and SSIM as reported.

Study focused on attribution-based saliency methods; other explainability techniques (e.g., counterfactuals) not evaluated; generalization beyond evaluated datasets and models not established; commercial prototype evaluated without access to internal architecture; details such as image file formats/resolutions not specified.

Use PSC to validate saliency methods before relying on them for interpreting medical AI outputs; exercise caution as commonly used saliency maps showed low sensitivity and robustness under subtle perturbations.

Model-agnostic and generalizable across tested CNN architectures and two datasets; human reader study showed perturbed images are difficult to detect by experts.