Datasheets for Datasets
Structured documentation describing a dataset's motivation, composition, collection process, preprocessing, uses, and maintenance, the dataset equivalent of a model card.
Definition
Datasheets for Datasets is a documentation framework, introduced by Gebru et al. (2018), that prescribes a structured set of questions every dataset should answer: motivation for creation, composition (instances, labels, splits), collection process (who, how, when, consent), preprocessing/cleaning/labeling, uses (recommended, discouraged), distribution, and maintenance. The intent is to surface biases, gaps, and limitations of a dataset before it is used to train or evaluate a model, and to make those properties auditable downstream by model consumers and regulators.What this means in practice
For AI/ML medical devices, dataset documentation is rapidly becoming a regulatory expectation alongside model cards. FDA's Good Machine Learning Practice (GMLP) principles call for transparency about the data used to train and test devices. EU AI Act Article 10 explicitly requires training, validation, and testing dataset documentation for high-risk AI systems, including demographic representativeness and bias analyses. Datasheets are the most widely adopted structure for meeting those requirements.- •Documenting only the training set, validation and test set datasheets are equally important for evaluating generalization claims.
- •Glossing over consent and source legitimacy, these are increasingly material under GDPR, HIPAA, and AI Act scrutiny.
- •Treating datasheets as static, the document must be updated when datasets are augmented or relabeled.
Related terms
Grouped by themeEditor's picks
· Hand-selected related conceptsMedical device that uses artificial intelligence or machine learning to perform its intended use.
Systematic differences in model performance across demographic or clinical subgroups.
Policies, controls, and documentation for the data used to train, tune, and validate medical AI models.
EU regulation establishing risk-based requirements for AI systems, including most medical AI.
More in Software & AI
· Same categoryGuiding principles for the development of AI/ML-enabled medical devices.
Structured documentation of a machine learning model's intended use, performance, and limitations.
FDA mechanism to pre-authorize specific modifications to AI/ML-enabled devices.
Artificially generated data used to augment training, validation, or stress-testing of medical AI.
Primary references
3 sources- 1
Datasheets for Datasets (Gebru et al.)VerifiedarXivarxiv.org
- 2
GMLP Guiding PrinciplesVerifiedFDAfda.gov
- 3
IMDRF - Software as a Medical DeviceVerifiedIMDRFimdrf.org
Inline markers like [1] jump to the matching reference above.