Quality & Risk
Summative vs. Formative Evaluation
In one line
Iterative usability studies (formative) vs. final validation testing (summative) of a device's user interface.
Definition
Formative evaluations are exploratory tests during design used to identify usability issues. Summative (validation) evaluations occur on the final design with representative users in realistic conditions to demonstrate critical tasks can be performed safely and effectively.
Formative work often uses small groups, prototypes and think-aloud methods, and results drive design changes. Summative validation per FDA guidance and IEC 62366-1 typically uses at least 15 participants per distinct user group, tests the final user interface including labeling and training, and focuses on critical tasks. Use errors seen in summative testing must be analyzed for root cause and residual risk.
Why it matters
Per FDA HF guidance and IEC 62366-1, summative studies typically include at least 15 users per distinct user group. Findings from summative studies that surface new use errors must be addressed before market.
Common pitfalls
- •Confusing formative testing with design verification; formative evaluations are for improvement, not formal verification.
- •Failing to adequately document formative evaluation findings and subsequent design changes, hindering traceability for regulatory submissions.
- •Conducting summative evaluations with non-representative users or in unrealistic environments, invalidating the study’s findings.
- •Underestimating the resources and time required for a robust summative evaluation, leading to rushed or poorly executed studies.
Sources
- FDA Human Factors Guidance · FDA
- MDIC Case for Quality · MDIC
- FDA - Quality Systems · FDA