Software & AI
Data Governance (AI/ML)
In one line
Policies, controls, and documentation for the data used to train, tune, and validate medical AI models.
Definition
Covers data provenance, consent, de-identification, demographic representativeness, labeling quality, version control, and audit trails. EU AI Act Article 10 codifies many of these as legal requirements for high-risk systems.
Good practice includes documenting where each dataset came from, the inclusion and exclusion criteria, how labels were created and checked, and how datasets were split so test data stays independent. Article 10 of the EU AI Act requires training, validation and test data to be relevant, sufficiently representative and, to the best extent possible, free of errors. Privacy rules such as HIPAA and the GDPR also apply to how health data is used.
Why it matters
Weak training-data governance is the most common reason medical AI submissions stall; reviewers want to see lineage as clearly as a DHF traces design.
Common pitfalls
- •Failing to document the complete lineage of training data, from raw acquisition to final processing, can lead to submission delays or rejections.
- •Using biased or unrepresentative datasets without mitigation strategies can result in AI models that perform poorly or inequitably across different patient populations, raising ethical and regulatory concerns.
- •Inadequate data labeling processes can introduce errors that propagate through the AI model, compromising its accuracy and reliability.
- •Neglecting to implement robust version control for datasets makes it difficult to reproduce model training or trace changes, hindering regulatory compliance.
- •Overlooking cybersecurity measures for data used in AI/ML development and deployment can expose sensitive patient information to breaches, violating privacy regulations like GDPR and HIPAA.
Sources
- EU AI Act Article 10 · European Parliament
- FDA - AI/ML-Enabled Medical Devices · FDA
- IMDRF - Software as a Medical Device · IMDRF