Fairness-aware training strategies can reduce sex-based performance disparities in pneumonia detection AI models while maintaining overall accuracy, though improving one fairness metric may widen another, requiring careful consideration of clinical priorities and intended use cases.
- Baseline pneumonia AI models showed higher accuracy in male patients (AUROC 0.882) versus female patients (AUROC 0.837), with greater false-positive rates in women.
- Fairness-aware debiasing combined sex-balanced sampling with adversarial debiasing to reduce performance gaps while preserving overall AUROC of 0.864.
- The debiasing approach improved sensitivity more in men, widening the sensitivity gap—demonstrating that no single fairness metric captures complete equity.
- Clinical context matters: Fairness targets must align with intended use, as excess false positives can lead to unnecessary imaging, antibiotics, and patient anxiety.
- EU AI Act requirements for high-risk medical devices will mandate subgroup monitoring and documentation, encouraging hospitals to audit AI system performance across patient populations.
A fairness-aware training strategy reduced sex disparities in an artificial intelligence model for pneumonia detection without significantly compromising overall performance, German researchers have reported. However, improving one measure of fairness widened another gap, highlighting the difficulty of determining whether an algorithm is clinically equitable.
Dr. Matteo Haupt of the Department of Diagnostic and Interventional Radiology at Carl von Ossietzky University Oldenburg in Oldenburg, Germany, and colleagues published their findings in Scientific Reports.
The researchers analyzed 26,684 frontal adult chest x-rays from the RSNA Pneumonia Detection Challenge dataset, a curated subset of the U.S. National Institutes of Health ChestX-Ray8 repository. The training data were 57% male and 43% female.
ImageNet-pretrained InceptionV3 models were trained across five prespecified random seeds and combined into an ensemble. Evaluation used a fixed patient-level test set of 1,000 radiographs, comprising 250 pneumonia-positive and 250 pneumonia-negative cases for each recorded sex. A single validation-derived threshold was applied to both groups.
Sex-based performance differences
The baseline ensemble achieved an overall area under the receiver operating characteristic curve (AUROC) of 0.859. Performance was higher in male than female patients, with AUROCs of 0.882 and 0.837, respectively. Specificity was also higher in men, at 0.780 compared with 0.707, resulting in a greater false-positive burden among female cases.
To investigate possible differences in model behavior, the researchers compared Grad-CAM activation maps with segmented lung regions. On average, 64.7% of activation was located within the lungs in male images, compared with 60.8% in female images.
The result generated a hypothesis that subgroup-correlated nonpulmonary features, including soft-tissue attenuation, body habitus, or image-border characteristics, may have influenced the model. However, the study did not test this mechanism experimentally, and the authors stressed that Grad-CAM is a coarse associative tool rather than proof of how the algorithm reached a decision.
Debiasing reduces performance gaps
Combining sex-balanced sampling with adversarial debiasing preserved overall performance, producing an AUROC of 0.864, while reducing the AUROC, specificity, and false-positive disparities. Global performance differences between the baseline and mitigated models were small, and their confidence intervals included zero.
Importantly, sensitivity improved more in men, widening the sensitivity gap from 0.020 to 0.056. The mitigated system therefore cannot be described as uniformly fairer: It shifted the distribution of errors.
That trade-off matters clinically. In triage, missed pneumonia may be more consequential than false alarms. In decision support, excess false positives can contribute to additional imaging, antibiotic treatment, admissions, costs, anxiety, and workload. Fairness targets must therefore reflect the system’s intended clinical use.
European rules increase focus on subgroup monitoring
For Europe, AI-enabled medical devices classified as high risk under the EU AI Act will face requirements concerning data governance, accuracy, risk management, human oversight, documentation, and postmarket monitoring. Although the legislation does not prescribe hospital-level sex audits of every system, subgroup monitoring could help institutions assess whether performance remains appropriate for their populations.
The findings require independent multicenter validation. The study used one U.S.-derived public dataset, binary recorded sex, and an artificial 50% pneumonia prevalence. It did not evaluate a commercial product or prospective clinical deployment.
The authors declared no competing interests. Open-access publication was supported through Projekt DEAL.
