Algorithmic Bias and Fairness
Systematic differences in model performance across demographic or clinical subgroups.
Definition
Algorithmic bias occurs when a model performs differently across subgroups (e.g., by sex, race, age, comorbidity) due to training data imbalance, label noise, or proxy variables. Fairness analyses quantify and mitigate these gaps.What this means in practice
FDA's GMLP principles call for representative datasets and subgroup performance analysis. EU AI Act high-risk requirements explicitly address bias monitoring across the lifecycle.Examples
- A diagnostic AI trained predominantly on data from one ethnic group performs poorly when used on patients from other ethnic groups, leading to misdiagnoses.
- A risk stratification algorithm for disease progression overestimates risk for one sex due to proxy variables correlated with socioeconomic status in the training data.
- A wearable device's algorithm, designed to detect a medical condition, shows lower accuracy for individuals with darker skin tones because the training images were not diverse enough.
- •Failing to define fairness metrics relevant to the specific clinical context and patient population.
- •Overlooking the potential for bias to be introduced or amplified at various stages of the AI/ML lifecycle, from data acquisition to deployment.
- •Assuming that a globally accurate model is also fair across all demographic subgroups.
Frequently asked questions
Cross-references
Part of
A larger framework or document this term belongs to.
Related terms
Grouped by themeEditor's picks
· Hand-selected related conceptsMedical device that uses artificial intelligence or machine learning to perform its intended use.
EU regulation establishing risk-based requirements for AI systems, including most medical AI.
Guiding principles for the development of AI/ML-enabled medical devices.
More in Software & AI
· Same categoryAAMI Technical Information Report providing guidance on applying Agile software development practices within an IEC 62304-compliant medical device software lifecycle.
Resilience of an ML model to inputs deliberately crafted to cause misclassification.
The detailed procedural section of a PCCP that specifies how planned modifications to an AI/ML model will be developed, validated, and implemented.
Software providing healthcare professionals with knowledge and patient-specific information.
Primary references
3 sources- 1
Good Machine Learning PracticeVerifiedFDA/Health Canada/MHRAfda.gov
- 2
IMDRF - Software as a Medical DeviceVerifiedIMDRFimdrf.org
- 3
MDCG Software GuidanceVerifiedMDCGhealth.ec.europa.eu
Inline markers like [1] jump to the matching reference above.