Sensitive Data Classification for AI/ML Training Datasets

Overview

AI and machine learning models trained on personal data raise distinct classification challenges. Training data may contain direct personal data, inferred special categories, proxy variables for protected characteristics, and data whose consent scope does not extend to model training. The EU AI Act (Regulation (EU) 2024/1689) imposes additional requirements for high-risk AI systems, including data governance obligations under Art. 10 that intersect with GDPR classification requirements. This skill provides a framework for classifying training data, detecting bias-relevant features, documenting data provenance, and verifying consent coverage.

GDPR and AI Act Intersection

GDPR Requirements for Training Data

EU AI Act Art. 10 — Data Governance for High-Risk AI

The AI Act Art. 10 requires that training, validation, and testing datasets for high-risk AI systems:

1. Are subject to appropriate data governance and management practices (Art. 10(2))
2. Are relevant, sufficiently representative, and to the extent possible free of errors and complete (Art. 10(3))
3. Take into account the specific geographical, contextual, behavioural, or functional setting within which the AI system is intended to be used (Art. 10(4))
4. Where special category data processing is strictly necessary for bias detection and correction, this is permitted under Art. 10(5), subject to appropriate safeguards including pseudonymisation and GDPR compliance

Training Data Classification Framework

Classification Tier 1: Personal Data Content

Classification Tier 2: Bias and Proxy Detection

Even when a dataset does not directly contain Art. 9 special category data, it may contain proxy variables that correlate with protected characteristics:

Classification Tier 3: Consent and Purpose Scope

Data Card Documentation

A data card is a structured document accompanying each training dataset, providing transparency about its contents, provenance, and limitations. Modelled on the "Datasheets for Datasets" framework (Gebru et al., 2021) and adapted for GDPR compliance.

Required Data Card Fields for Vanguard Financial Services

Bias Detection Methodology

Step 1: Demographic Representation Analysis

For each training dataset, calculate representation statistics:

• What percentage of records come from each demographic group (to the extent known)?
• Does the representation match the target population?
• Are any groups under-represented by more than 20% relative to population?

Step 2: Proxy Variable Identification

Scan all features for proxy correlation with Art. 9 protected characteristics:

• Calculate correlation coefficient between each feature and known protected characteristics (where available)
• Flag features with |correlation| > 0.3 as potential proxies
• Document all proxy variables in the data card

Step 3: Disparate Impact Analysis

For classification or scoring models:

• Calculate model performance metrics by demographic group
• Apply the 80% (four-fifths) rule: if the selection rate for any protected group is less than 80% of the rate for the most favoured group, disparate impact may exist
• Document disparate impact analysis results in the data card

Step 4: AI Act Art. 10(5) Assessment

If bias detection requires processing special category data:

• Document why processing is "strictly necessary" for bias detection and correction
• Implement pseudonymisation of the special category data used for bias testing
• Ensure GDPR Art. 9(2) condition is established (typically Art. 9(2)(g) substantial public interest or Art. 9(2)(j) research)
• Process in a controlled environment with access restricted to the bias assessment team
• Delete special category data after bias assessment is complete

Enforcement and Regulatory Precedents

• Italian Garante — Clearview AI (2022): EUR 20 million fine for processing biometric data scraped from public sources for AI facial recognition training without lawful basis, consent, or transparency. Established that public availability does not provide lawful basis for AI training.
• Italian Garante — ChatGPT/OpenAI (2023): Temporary ban and subsequent enforcement requiring OpenAI to establish lawful basis for training data collection, implement age verification, and provide opt-out mechanisms. Highlighted that AI training on personal data requires GDPR compliance throughout the data lifecycle.
• CNIL — Enforcement Notice on AI Training Data (2024): CNIL published guidance sheets on AI training data requiring purpose limitation assessment, proportionality analysis, and specific measures when training data contains special category data.
• ICO — Generative AI and Data Protection Consultation (2024): ICO's position that legitimate interests is the most likely lawful basis for AI training but requires documented LIA considering data subject reasonable expectations.

Integration Points

• personal-data-test: Training data must first be classified as personal or non-personal
• special-category-data: Art. 9 data in training sets requires heightened protections
• pseudo-vs-anon-data: De-identification of training data must be validated
• classification-policy: Training data classified under enterprise classification tiers
• data-lineage-tracking: Full provenance from original collection to model deployment must be tracked