Data Protection Impact Assessment for AI/ML Systems

Overview

AI and ML systems present unique privacy challenges that traditional DPIA methodologies fail to adequately address. The EDPB Guidelines 04/2025 on processing personal data through AI systems establish a specialized framework that supplements the general DPIA requirements of GDPR Article 35 and WP248rev.01. AI-specific DPIAs must evaluate the entire ML pipeline — from training data collection through model deployment and inference — assessing risks that emerge from statistical learning, emergent model behaviours, and the opacity of algorithmic decision-making. This skill implements the EDPB's AI-specific DPIA methodology integrated with the EU AI Act risk classification framework.

AI-Specific DPIA Triggers

Mandatory DPIA Triggers for AI Systems

All AI processing that meets any of the following criteria requires a DPIA before deployment:

EDPB WP248 Criteria Applied to AI

AI systems frequently trigger multiple WP248 criteria simultaneously:

• Evaluation/scoring: Inherent to classification and regression models
• Automated decision-making: Core function of deployed AI systems
• Innovative technology: Novel model architectures, training techniques
• Large-scale processing: Training datasets containing millions of records
• Matching/combining datasets: Multi-source training data aggregation
• Vulnerable data subjects: When AI is applied to children, employees, patients

When an AI system meets two or more criteria, a DPIA is presumptively required.

AI DPIA Methodology — EDPB Framework

Phase 1: AI System Description (Art. 35(7)(a) Extended)

The systematic description must cover the complete AI lifecycle:

1.1 Training Phase Documentation

• Training data sources: Origin, collection method, consent status, lawful basis for each dataset
• Data categories: All personal data categories present in training data, including data that may be inadvertently included (background individuals in images, metadata in text corpora)
• Data volume: Number of records, data subjects affected, geographic scope
• Data preprocessing: Cleaning, augmentation, labelling processes and any human review
• Feature engineering: Which personal data attributes are used as features, how derived features are computed
• Training infrastructure: Where training occurs (cloud provider, jurisdiction), data residency during training

1.2 Model Architecture Documentation

• Model type: Neural network architecture (transformer, CNN, RNN), ensemble methods, decision trees
• Model parameters: Number of parameters, model size, complexity indicators
• Explainability characteristics: Inherent interpretability level (white-box, grey-box, black-box)
• Memorization risk: Assessed propensity for the model to memorize training data (higher for large models with small datasets)

1.3 Deployment Phase Documentation

• Inference inputs: What personal data is processed at inference time
• Output types: Classifications, scores, recommendations, generated content
• Decision pipeline: How model outputs feed into decisions affecting data subjects
• Human oversight: Level and effectiveness of human review in the decision chain
• Monitoring: Drift detection, performance monitoring, feedback loops

Phase 2: AI-Specific Necessity and Proportionality

2.1 Purpose Limitation for AI

• Is the AI system necessary for the stated purpose, or could simpler processing achieve it?
• Has the controller evaluated non-AI alternatives and documented why AI is required?
• Are the training data processing purposes compatible with the original collection purposes (Art. 6(4) assessment)?
• For repurposed data: has a compatibility assessment been conducted per EDPB Guidelines 04/2025?

2.2 Data Minimisation for AI

• Has the minimum dataset required for acceptable model performance been determined through ablation studies?
• Can synthetic data, federated learning, or differential privacy reduce the personal data requirement?
• Are there personal data elements in the training data that do not contribute to model performance?
• Has the controller assessed whether anonymised or pseudonymised data could achieve adequate performance?

2.3 Training Data Lawfulness Assessment

For each training dataset, document:

Phase 3: AI-Specific Risk Assessment

3.1 Privacy Risk Categories for AI

3.2 AI Risk Scoring Matrix

Combine likelihood and severity using the EDPB-recommended matrix:

                    Negligible    Limited    Significant    Maximum
Almost Certain      Medium        High       Very High      Very High
Likely              Medium        High       High           Very High
Possible            Low           Medium     High           High
Remote              Low           Low        Medium         High

3.3 AI Act Risk Level Integration

Cross-reference GDPR risk assessment with AI Act classification:

• Unacceptable risk (Art. 5 AI Act): Processing must not proceed — social scoring, real-time biometric identification in public spaces (with limited exceptions)
• High risk (Art. 6 + Annex III): Enhanced DPIA obligations, conformity assessment required
• Limited risk (Art. 50): Transparency obligations — inform users they are interacting with AI
• Minimal risk: Standard DPIA process applies

Phase 4: AI-Specific Mitigation Measures

Technical Measures

Organisational Measures

• Establish an AI Ethics Review Board with privacy representation
• Implement model cards documenting privacy properties for each deployed model
• Conduct regular model audits (minimum annually) testing for privacy leakage
• Maintain training data provenance documentation and deletion capability
• Define retraining triggers and ensure DPIA review accompanies each retraining cycle
• Implement incident response procedures specific to AI privacy incidents

Phase 5: Human Oversight Assessment

Per AI Act Art. 14 and GDPR Art. 22, assess the human oversight mechanism:

Prior Consultation Triggers for AI

Art. 36 prior consultation with the supervisory authority is required when:

1. The AI system produces high residual risk after all mitigation measures
2. The AI system processes special category data at scale with novel techniques
3. The supervisory authority's Art. 35(4) list specifically includes the AI use case
4. The AI system is deployed for real-time biometric identification under AI Act Art. 5 exceptions

Enforcement Precedents

• Clearview AI (Multiple DPAs, 2021-2024): Fines totalling over EUR 90 million across Italy (EUR 20M), France (EUR 20M), UK (GBP 7.5M), Greece (EUR 20M) for facial recognition AI deployed without DPIA, lawful basis, or transparency
• CNIL v. Clearview AI (SAN-2022-019): Specific finding that no DPIA was conducted for biometric AI processing
• Italian DPA v. Replika (2023): Ordered cessation of AI chatbot processing due to inadequate age verification and failure to conduct DPIA for AI processing affecting minors
• Spanish DPA v. CaixaBank (PS/00421/2020): EUR 6 million fine for automated credit scoring without adequate DPIA addressing algorithmic decision-making risks
• Dutch DPA v. Tax Authority (2020): Finding that algorithmic fraud detection system (SyRI) lacked proportionality and adequate DPIA for AI-driven profiling

Integration Points

• ai-training-lawfulness: Detailed lawful basis analysis for training data feeds into Phase 2
• ai-automated-decisions: Art. 22 assessment integrates with Phase 5 human oversight
• ai-model-privacy-audit: Technical privacy testing results feed into Phase 3 risk assessment
• ai-act-high-risk-docs: AI Act conformity assessment aligns with Phase 3.3 risk classification
• ai-bias-special-category: Bias assessment results feed into risk scoring for discrimination harms