AI Act Technical Documentation Annex IV: A guide to preparing your compliance file.

Why technical documentation is the critical point of compliance control

Annex IV of Regulation (EU) 2024/1689 is not merely an administrative formality. It constitutes the cornerstone of compliance control for high-risk AI systems.

Supervisory authorities, such as the AI Office or the CNIL, will rely on this file to assess the system's compliance. An incomplete or poorly structured file may result in fines of up to €35 million or 7% of global turnover, in accordance with Article 99 of the AI Act.

The technical documentation must demonstrate that the system complies with the requirements of Articles 8 to 15 of the AI Act. It must be updated throughout the system's lifecycle and be available for inspection for ten years after placing on the market.

The 9 pillars of Annex IV explained

Annex IV lists nine categories of information to be documented. Here is a detailed analysis of each pillar, along with the expected technical artefacts.

1. General description of the AI system

This section must provide an overview of the system, including:

• Name and version of the system
• Description of the main functionalities
• High-risk category (Annex III)
• Overall technical architecture
• Data flow diagram

Expected artefact: System identity sheet (maximum 1 page).

2. Detailed description of the system components and development process

This part must cover:

• Description of software and hardware components
• Explanation of the algorithms used
• Development methodology (e.g., Agile, Waterfall)
• Testing and validation phases

Expected artefact: Model card and datasheet (based on standards such as Model Cards for Model Reporting and Datasheets for Datasets).

3. Detailed information on training, validation, and test data

The requirements include:

• Origin and quality of the data
• Data collection and cleaning process
• Description of potential biases and mitigation measures
• Data performance metrics (e.g., representativeness, balance)

Expected artefact: Comprehensive datasheet with a register of data sources.

4. Risk assessment and mitigation measures

This section must document:

• Risk assessment methodology (e.g., FRIA)
• Identified risks and their severity
• Mitigation measures implemented
• Evidence of the effectiveness of the measures

Expected artefact: Risk matrix and mitigation plan.

5. Description of transparency and user information measures

The elements to document include:

• User instructions and warnings
• Explanations of the system's limitations
• User feedback mechanisms

Expected artefact: User manual and transparency sheet.

6. Description of human oversight measures

This part must include:

• Roles and responsibilities of human operators
• Supervision and intervention procedures
• Tools provided for oversight

Expected artefact: Human oversight procedure and intervention log.

7. Description of robustness, accuracy, and cybersecurity measures

The requirements cover:

• Performance metrics (e.g., accuracy, recall, F1-score)
• Robustness tests (e.g., adversarial testing)
• Cybersecurity measures (e.g., encryption, access control)

Expected artefact: Technical test report and cybersecurity audit.

8. Description of the post-market monitoring system

This section must detail:

• Continuous monitoring process
• Feedback collection mechanisms
• Update and maintenance procedures
• Incident management plan

Expected artefact: Post-market monitoring plan and incident log.

9. Register of significant modifications

The information to be recorded includes:

• Description of modifications
• Justification for changes
• Impact on compliance
• Evidence of risk reassessment

Expected artefact: Modification register with versioning.

3 common pitfalls to avoid in technical documentation

Compliant technical documentation is not just about ticking boxes. Here are the frequent mistakes that can compromise your compliance.

1. Relying on an average metric score

The AI Act requires a detailed analysis of system performance, including:

• Disaggregation of metrics by subgroups (e.g., gender, age, origin)
• Analysis of errors and their impacts
• Performance evaluation under real-world conditions

An overall accuracy or recall score is insufficient. Performance must be documented for each relevant group, and discrepancies must be explained.

2. Overlooking data traceability

Data traceability is a cornerstone of compliance. Authorities will require:

• A register of data sources
• Proof of consent or legal basis for each dataset
• A history of data transformations

Without this traceability, it is impossible to demonstrate compliance with the GDPR or Annex IV.

3. Neglecting human oversight

The AI Act mandates human oversight measures for high-risk systems. This includes:

• Clear definition of roles and responsibilities
• Training of human operators
• Documentation of intervention procedures
• A log of human actions

An AI system cannot operate in full autonomy. The documentation must prove that human oversight mechanisms are in place and operational.

How to turn legal text into an operational action plan

Moving from regulatory requirements to technical deliverables requires a structured methodology. Here is how to proceed.

1. Assign roles and responsibilities

Technical documentation involves several stakeholders:

2. Create a register of evidence

Each requirement of Annex IV must be linked to tangible evidence. Here is an example structure:

3. Prioritise actions with a risk matrix

Not all Annex IV requirements have the same level of criticality. Use a risk matrix to prioritise:

4. Automate evidence collection

Technical documentation must be maintained throughout the system's lifecycle. To achieve this:

• Integrate versioning tools (e.g., Git, DVC) to track modifications
• Use MLOps platforms (e.g., MLflow, Weights & Biases) to document experiments
• Automate report generation (e.g., Python scripts for datasheets)

Resources to start your technical documentation

To help you prepare your compliance file, AiActo provides a comprehensive and free Annex IV Workbook.

To download the Annex IV Workbook, visit the AiActo Document Generation page or use the direct link: PDF Download EN.