Assessments

Conformity assessment Fundamental Rights Impact Assessment Transparency obligations

Conformity assessment:

The IA Act requires suppliers of high-risk systems to submit their IA to a conformity assessment (Art.16). This consists of demonstrating that the system in question complies with the requirements imposed on it by the IA Act.

The regulation provides for different assessment modalities depending on the intended purpose of the high-risk system. 

Internal assessment procedure referred to in Annex VI of the IA Act.

Which systems should be subject to this assessment?

  • Categorisation or biometric identification and emotion recognition systems.
  • Systems used in critical infrastructure.
  • Systems used in the field of vocational education and training.
  • Systems used in the field of employment, management of workers and access to self-employment. 
  • Systems used in the area of access to and use of essential services and benefits (public or private).
  • Systems used in the field of law enforcement.
  • Systems used in the field of migration, asylum and border control.
  • Systems used in the field of the administration of justice and democratic processes.

How does this procedure work?

This is an internal assessment, i.e. carried out by the system provider, in which the following aspects should be verified:

  • That a quality management system has been established which complies with the requirements of Art. 17 of the IA Act.
  • That the information contained in the technical documentation has been examined for compliance with the requirements applicable to high risk systems.
  • That the design and development process of the system, as well as its post-marketing monitoring, is consistent with the information contained in the technical documentation.

External assessment procedure, based on the assessment of the quality management system and of the technical documentation, with the involvement of a notified body, as referred to in Annex VII of the IA Act.

Which systems should be subject to this assessment?

This mode of assessment is mandatory for categorisation or biometric identification and emotion recognition systems in the following circumstances:

  • Where the European Union has not established harmonised standards or applicable common specifications. 
  • When these harmonised standards or common specifications exist, but the supplier has not implemented them.
  • Where any of the harmonised standards referred to in point (a) have been published with a restriction (i.e. they do not cover all the elements to be assessed), the issues covered by the restriction should be subject to this assessment option.

Even if the above circumstances do not apply, providers of categorisation or biometric identification and emotion recognition systems may voluntarily submit to this assessment procedure as an alternative to the internal procedure provided for in Annex VI if they wish so.

How does this procedure work?

The assessment will be carried out by an external body named notified body. These are supplier-independent organisations, appointed by the competent authority of each member state, to carry out assessment tasks.

The conformity assessment shall consist of the following parts:

  • Evaluation of the quality management system:
    • The supplier must submit an application for assessment of the quality management system of his system to the notified body of his choice.
    • The application must include the information detailed in point 3 of Annex VII of the IA Act.
    • The notified body shall notify the supplier of the conclusions of the assessment and its reasoned decision.
    • Any intended change to the quality management system must be reported to the notified body, which must decide whether the system continues to comply with the legal requirements or whether it needs to be reassessed.
  • Control of technical documentation:
    • The supplier must send an application for assessment of the technical documentation of the system to the notified body of his choice. 
    • The application must include the information detailed in point 4 of Annex VII of the IA Act.
    • The notified body may require full access to the training, validation and test data used and/or request additional evidence. If these means are insufficient, it may also request access to the training models of the system as a last resort.
    • The notified body shall forward the conclusions of the assessment and its reasoned decision.
    • If the assessment is passed positively, the notified body shall issue a certificate.
    • Any changes that may affect compliance with legal requirements or the intended purpose of the system must be reported to the notified body, which shall decide whether the system needs to be reassessed or whether it is possible to address the changes in isolation. In the latter case, the body shall issue a supplement to the certificate.
  • Monitoring of the approved quality management system:
    • After approval of the quality management system, the notified body shall monitor its proper implementation.
    • The supplier shall allow the notified body access to the facilities where the system is designed, developed and tested and shall share all necessary information.
    • The notified body shall carry out periodic audits and shall provide the supplier with the resulting reports. As part of these audits the notified body may carry out additional tests.

Specific assessment procedure for some systems:

Which systems should be subject to this assessment?

Systems subject to the regulations listed in Annex I, Section A of the IA Act. These include:

  • Toys;
  • Pleasure boats and jet skis;
  • Lifts;
  • Equipment and protective systems used in potentially explosive atmospheres.
  • Radio equipment;
  • Pressure equipment;
  • Cable car facilities;
  • Personal protective equipment, equipment for the protection of gaseous fuels;
  • Medical devices and in vitro diagnostic products;
  • Machines meeting the definition of Directive 2006/42/EC. 

How does this procedure work?

  • AI systems shall be subject to the conformity assessments provided for in the regulations listed in Annex I, Section A, of the AI Act (those mentioned above) to which they are subject.
  • In carrying out the above-mentioned assessments, it will be necessary to incorporate the examination of compliance with the obligations imposed by the IA Act on high-risk IA systems. In some cases, bodies having the status of notified bodies under the above-mentioned legal acts will be able to carry out this part of the assessment, where certain requirements under Art. 31 of the IA Act have been taken into account in order to acquire the status of notified bodies. 
  • In addition, as part of the conformity assessment, the check of the technical documentation which forms part of the external assessment procedure described in Annex VII of the IA Act (and which has been explained in the previous section) must also be carried out as part of the conformity assessment.
  • In cases where these other rules allow the supplier not to undergo the external assessment, the supplier may also choose this option in relation to the standards of the IA Act, provided that the European Union has created harmonised standards relating to these issues and that the supplier has implemented them.

Irrespective of the applicable procedure, in case the system is substantially modified after having passed the assessment, it shall be subject to a new conformity assessment. 

For these purposes, changes to systems that continue to learn after their development shall not constitute a substantial modification, provided that they were predetermined prior to the initial conformity assessment and are covered by the technical documentation.

Fundamental Rights Impact Assessment:

Article 27 of the IA Act requires that, prior to their development, certain systems must be subject to an assessment of the impact that their use may have on fundamental rights. 

This assessment shall be compulsory in the following cases:

  • Where a public body or a private body providing public services deploys an IA system for one of the following areas:
    • Biometric identification or categorisation and emotion recognition
    • Education and vocational training 
    • Employment, employee management and access to self-employment. 
    • Access to and enjoyment of essential services and benefits (public or private).
    • Law enforcement.
    • Migration, asylum and border control.
    • Administration of justice and democratic processes.
  • In any case, where an AI system is developed for any of the following purposes:
    • Assess the creditworthiness of natural persons or establish their credit score, with the exception of AI systems used for the purpose of detecting financial fraud.
    • Assess risks and pricing in relation to individuals in the case of life and health insurance.

The fundamental rights impact assessment should include the following aspects:

  • A description of the processes in which the developer will use the AI system in accordance with the intended purpose of the AI system.
  • A description of the time period and frequency in which the system is intended to be used.
  • The categories of individuals and groups likely to be affected by the use of the system in the specific context in which it will be used.
  • The specific risks of harm likely to affect the identified categories of persons or groups of persons.
  • A description of the human supervision measures to be applied in accordance with the instructions for use of the system.
  • The measures envisaged to be taken in case of materialisation of these risks, including internal governance and whistleblowing mechanisms.
  • Where some of the above elements are also subject to the data protection impact assessment provided for in Article 35 of Regulation (EU) 2016/679 or Article 27 of Directive (EU) 2016/680, the two assessments shall be carried out jointly.

Once the assessment has been carried out, the form to be developed by the EU IA Office (not yet available) will have to be filled in. This should be submitted to the competent market surveillance authority. 

The assessment should only be carried out before using the system for the first time. On subsequent occasions when the system is to be used in similar circumstances, it is possible to rely on assessments that have been carried out previously, or even those that may have been carried out by the supplier. 

In the event of any changes affecting the issues assessed, the information will need to be updated. 

Transparency obligations:

Among the many rules established by the IA Act, there are transparency obligations that are scattered throughout the articles of the regulation and its annexes. 

To make it easier to understand these requirements, here is an outline that summarises the transparency rules established by law. 

Transparency obligations for high-risk systems:

According to Article 13, high-risk systems must be designed and developed in such a way that their operation is sufficiently transparent to allow developers to interpret their results and use them correctly. 

For this purpose, and in order to meet the requirements of the IA Act for high-risk systems, the systems must be accompanied by the technical documentation provided for in Article 11 and Annex IV of the Act. This technical documentation includes the instructions for use of the system. Briefly, the minimum content required by these documents is as follows: 

  • An overview of the system. This includes:
    • The identity and contact details of the supplier and his authorised representative, if any.
    • A description of the version of the system reflecting its relationship to previous versions;
    • Information that allows developers to interpret the system’s responses and use it appropriately. 
    • The computational and hardware resources required, the expected lifetime of the high-risk AI system and any maintenance and care measures necessary to ensure the proper functioning of the system. 
    • A description of the mechanisms included in the IA system to enable users to collect, store and correctly interpret the records of its operation. 
    • How the system can interact with hardware or software that is not part of the AI system itself.
    • The relevant software or microprogramme versions and any requirements related to version upgrades.
    • The description of all the ways in which the AI system is brought to market or put into service.
    • Where the AI system is a component of products, photographs or illustrations showing external features, marking and internal layout of these products.
  • A detailed description of the elements of the AI system and the process for its development. This includes:
    • The methods and steps performed for the development of the AI system, including whether other previously trained systems have been used.
    • The description of the system design, this includes the general logic of the AI system and algorithms, and the key design choices.
    • The description of the system architecture.
    • Sheets describing the training methodologies and techniques and the training datasets used, including a general description of these datasets, information on their provenance, scope, processing and main characteristics.
    • Assessment of planned human supervision measures. 
    • Changes to the system that have been predetermined by the supplier at the time of the initial conformity assessment, if any.
    • The validation and testing procedures used, including information on the validation and testing data used, the metrics used to measure accuracy and robustness, as well as potentially discriminatory impacts.
    • Test records and all test reports dated and signed by the responsible persons.
    • Cybersecurity measures in place.
  • Information on the monitoring, operation and control of the system. This includes:
    • Their capabilities and limitations and the expected degree of accuracy, both overall and for specific individuals or groups of individuals.
    • Foreseeable unintended outcomes and sources of risk to health and safety, fundamental rights and discrimination in accordance with their intended purpose.
    • Input data specifications.
    • A detailed description of the risk management system.
    • A description of the relevant changes made by the supplier to the system throughout its lifecycle.
    • A list of the harmonised standards published in the Official Journal of the EU applied, the solutions adopted to meet the requirements set out in the IA Act, and a list of other relevant standards and technical specifications applied.
  • A copy of the EU Declaration of Conformity.
  • A detailed description of the post-market surveillance system adopted.

Transparency obligations for general-purpose models:

In addition to the obligations for high-risk systems, the IA Act provides for specific transparency obligations for general-purpose systems in Articles 50 to 53 and Annexes XI and XII. It is possible that a general-purpose system may also qualify as high risk. In this case, both the obligations for high-risk systems and those only for general-purpose systems would apply. 

The transparency obligations for general-purpose models are as follows:

  • Systems interacting with natural persons should be designed and developed in such a way that people are informed that they are interacting with an AI
  • Output from systems generating synthetic audio, image, video or text content must be marked in a machine-readable format and detectable as artificially generated or manipulated.
  • Developers of systems that generate or manipulate image, audio or video content that constitutes a deepfake must disclose that the content has been artificially generated or manipulated.
  • If the AI is an emotion recognition or biometric categorisation system, the natural persons exposed to it shall be informed of its operation. 
  • Developers of an AI system that generates or manipulates text that is published for the purpose of informing the public on matters of public interest must disclose that the text has been generated or manipulated by an AI. This obligation does not apply where the content has been subject to a process of human review or editorial control, provided that it has been determined who is responsible for the published content.
  • Develop and make publicly available a sufficiently detailed summary of the content used for the formation of the general-purpose IA model, according to a template provided by the IA Office (not yet available).
  • Develop and keep up to date the technical documentation of the model, which shall contain at least the following elements:
    • A general description of the model. This includes:
      • The tasks the model is intended to perform and the type and nature of AI systems into which it can be integrated;
      • Applicable acceptable use policies;
      • The release date and distribution methods;
      • The architecture and the number of parameters;
      • The mode and format of inputs and outputs (text, images, etc.).
      • The licence.
    • A detailed description of the elements of the model and relevant process information for its development. This includes:
      • The technical means necessary for the model to integrate with other AI systems.
      • The design specifications of the model and the training process.
      • Information on the data used for training, testing and validation of the model, as well as their provenance, the treatment they have undergone, the main characteristics and the methods applied to identify biases.
      • The computational resources used to train the model.
      • The known or estimated energy consumption of the model.
  • Where the model is to be classified as a general-purpose systemic risk model in accordance with Section 51 of the IA Act, the following additional information must be provided:
    • A detailed description of the assessment strategies used to identify and mitigate potential systemic risks arising from the use of the model. 
    • A description of the measures taken to carry out contradictory tests (internal or external) as well as model adaptation and adjustment.
  • Develop and make available to suppliers intending to integrate the general-purpose model into their IA system documentation and information which shall include, as a minimum, the following content:
    • An overview of the general-purpose AI model. This includes:
      • The tasks the model is intended to perform and the type and nature of the AI systems in which it can be integrated.
      • Applicable acceptable use policies.
      • The release date and distribution methods.
      • How the model interacts or can be used to interact with hardware or software that is not part of the model itself.
      • Relevant software versions related to the use of the general-purpose AI model.
      • The architecture and number of parameters.
      • The mode and format of inputs and outputs (text, images, etc.).
      • The model licence.
    • A description of the elements of the model and the process for its development. This includes:
      • The technical means necessary for the model to integrate with other AI systems.
      • The mode and format of inputs and outputs and their maximum size (text, images, etc.).
      • Information on the data used for training, testing and validation, including the type and provenance of the data and how it has been processed.
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.

Strictly Necessary Cookies

Strictly Necessary Cookie should be enabled at all times so that we can save your preferences for cookie settings.