Model Drift Monitoring
Ongoing surveillance of input data and model outputs to detect performance degradation post-deployment.
Definition
Drift monitoring tracks data drift (input distributions changing) and concept drift (the relationship between inputs and outputs changing). For locked algorithms this triggers retraining; for adaptive algorithms it gates change-control under a PCCP.What this means in practice
FDA TPLC and PCCP guidance both expect documented drift monitoring with thresholds, alerts, and a corrective-action playbook.Examples
- A continuous glucose monitor (CGM) with an AI component detects a statistical shift in blood glucose readings from a new patient population, indicating data drift that may require model re-calibration.
- An algorithm assisting in dermatological lesion classification starts performing poorly on images captured by newer smartphone models, revealing concept drift as the visual characteristics of the input data have changed relative to the algorithm's training.
- A diagnostic imaging system using AI identifies a gradual increase in false negatives for a specific pathology, which, through drift monitoring, is traced back to a change in data acquisition protocols at several clinical sites.
- •Failing to define clear, measurable thresholds for drift detection can lead to ineffective monitoring and delayed corrective actions.
- •Assuming that a "locked" algorithm negates the need for drift monitoring overlooks potential shifts in real-world data that could degrade performance.
- •Neglecting to establish a predetermined change control plan (PCCP) for adaptive algorithms can result in regulatory non-compliance when model updates are necessary.
- •Underestimating the resources, including personnel and infrastructure, required to effectively implement and maintain a drift monitoring program is a common pitfall.
- •Over-reliance on automated drift detection without human oversight or expert review can lead to false positives or missed critical drift events.
Frequently asked questions
Related terms
Shared paths + categoryDistinction between models whose behavior is fixed at release vs. those that continue learning.
FDA framework integrating premarket and post-market oversight across a device's life.
Methods (SHAP, saliency maps, prototype explanations) that help users understand why a model made a given prediction.
Structured documentation of a machine learning model's intended use, performance, and limitations.
Medical device that uses artificial intelligence or machine learning to perform its intended use.
Large pretrained model adaptable to many downstream clinical tasks via fine-tuning or prompting.
Latest in MedTech
Primary references
3 sources- 1FDA AI/ML Action PlanVerifiedFDAfda.gov
- 2FDA - Software as a Medical Device (SaMD)VerifiedFDAfda.gov
- 3FDA - AI/ML-Enabled Medical DevicesVerifiedFDAfda.gov
Inline markers like [1] jump to the matching reference above.