Software & AI
Synthetic Data
In one line
Artificially generated data used to augment training, validation, or stress-testing of medical AI.
Definition
Synthetic data is generated by simulators, generative models, or rule-based systems. Used to balance underrepresented subgroups, model rare events, or substitute for sensitive data - but introduces realism, leakage, and bias-amplification risks.
Regulators generally treat synthetic data as supplementary: it can help with training or stress testing, but performance claims should rest on real-world clinical data. FDA's Office of Science and Engineering Laboratories has published research on synthetic data and in silico testing. If synthetic data is used, manufacturers should document how it was generated, how realistic it is and how it could affect model behavior.
Why it matters
FDA has explicitly noted both the promise and the limits of synthetic data for training and validation; transparent reporting is expected.
Common pitfalls
- •A common pitfall is generating synthetic data that fails to capture the true underlying statistical properties or rare events accurately, leading to models that perform poorly in real-world scenarios.
- •Another mistake is using synthetic data without sufficient transparency regarding its generation methods, assumptions, and validation, which can hinder regulatory review.
- •Failing to address potential biases amplified during synthetic data generation is a significant pitfall, as it can perpetuate or exacerbate inequities in AI/ML-enabled medical devices.
- •Over-reliance on synthetic data without subsequent validation against real-world data can lead to a false sense of security regarding model performance and safety.
- •A pitfall is not adequately assessing the impact of synthetic data on the overall risk management strategy of a medical device, especially regarding safety and effectiveness.
Sources