Education contracts

The AI system shall be accompanied by accurate information on what it is used for, the identity and contact details of the provider or representatives.

Ensure that the AI system’s result generation process meets the following requirements:

• Well documented and reproducible to detect future problems.
• Use the simplest and most intelligible model in favour of transparency.
• Take into account the AI skills and literacy of all users.

The following information about the AI system will be documented and made public:

• The development process and the people who have participated in it.
• Concise, clear instructions on how it works, written in accessible language.
• The process by which a given result is achieved and the expected results.
• Any limitations.

The AI system identifies that the information is provided by an AI.

Measures shall be taken to ensure that the level of transparency and explainability of the AI system does not pose a risk to individuals’ privacy or to the security of the system itself.

With regard to the data used by the AI system, compliance with the following requirements shall be ensured:

• Documentation on the type of data used, its origin and the changes it has undergone at each stage of its life cycle.
• Measures shall be taken to verify the representativeness and adequacy of the data used.
• Safeguards to verify the age of users.
• Awareness of algorithmic biases.

The AI system shall have the following communication tools:

• Ensure informed consent.
• Notifications in the event of a serious incident or error.

Policy on the participation of different interest groups in the educational community.

Network with other educational centres that use a similar AI system.

The AI system will include measures for inclusion and equity, especially in relation to the following aspects:

• Assessment of results for different groups of students.
• Measures to increase accessibility.
• Measures to support its use by people in vulnerable situations.
• Measures to ensure that the system adapts to a wide range of individual preferences and abilities.

The AI system development team should include individuals who are not male, individuals from diverse backgrounds or cultures, and individuals from diverse disciplines of knowledge (beyond computer science and engineering).

The educational institution shall promote an organisational policy that prioritises diversity, equity, and inclusion.

The AI system will have measures and mechanisms in place to detect, correct and mitigate potential biases throughout its life cycle.

In particular, those present in:

• The design or implementation phase.
• Training data and subsequent data.
• Changing or inadequate categories.

The AI system shall include safety mechanisms to prevent the following incidents:

• Anomalies or actions outside predetermined limits.
• Deviated outputs and feedback loops.

Measures for the detection, correction and mitigation of risks shall comply with applicable data protection regulations. Specifically, Regulation (EU) 2016/679, Directive (EU) 2016/680 and Regulation (EU) 2018/1725.

The system shall be accompanied by the following information:

• The level of accuracy and security.
• The possible behaviours and scenarios that must never be violated in order to guarantee safety and preserve non-maleficence.
• The supervision and control mechanisms (both internal and external).

Students use the AI system as a learning tool that enhances their educational experience.

Before being marketed/used, the AI system shall undergo an assessment that includes the following aspects:

• The potential impacts on the health, safety and rights of the persons concerned.
• The risk that the data collected, generated or used will not be used to discriminate negatively against individuals.
• The potential risks and harm that would result from inaccurate predictions in the anticipated scenarios.
• The risks of using the system beyond its intended uses.

The AI system will have means to identify, assess and take action on the risks arising from its use and the data collected after it is put into operation.

The AI system shall have a general protocol on data traceability for risk management.

The AI system will have the following measures in place:

• Measures to mitigate bias or attempts at manipulation.
• Revocation of consent without compromising the functioning of the system.
• Measures to resist negative effects.
• Measures for continuous review of the accuracy of decisions made by the system.

Before being marketed/used, the AI system shall undergo:

• Assessment to ensure its security, integrity and resilience against different forms of attack.
• Assessment to verify that it has not been compromised or hacked.

Measures to ensure that persons involved in controlling decisions made by the AI system understand and are trained to perform that function.

The AI system shall be accompanied by information and documentation relating to the following aspects:

• The limits of the control mechanisms.
• Human oversight measures.

Before the AI system is marketed/used, the following shall be determined:

• The person legally responsible for the actions and/or decisions taken throughout its life cycle.
• The person responsible for human oversight measures.

The educational institution shall take the following measures:

• It will carry out an assessment to determine the influence of the system on decision-making.
• It shall establish mechanisms for periodic review of responsibility and accountability with the different members of the educational community who interact within it.
• It shall ensure the improvement of education and student results without compromising their participation or the quality of teaching.

The educational centre shall adopt the following measures:

• Facilitate demonstration of compliance with current legislation.
• Forms of redress or compensation in the event of damage caused by the AI system.
• Determine protection measures for users and persons affected by the AI system.

The educational centre undertakes not to make any statements that could lead to an underestimation of the capabilities of the AI system and that cannot be demonstrated by sharing data and code.

The educational centre will develop a code of ethics for the responsible use of AI with the following points:

• Promotion of responsible use.
• Determination of the system’s objectives and review of its operation from this perspective.
• Measures to guarantee the rights and well-being of individuals in the decision-making process.

Data protection mechanisms in place throughout the entire data lifecycle, ensuring:

• Personal data is processed in accordance with the legally required protection.
• The data provided by the data subject will not be sold, rented or transferred without their consent.
• Maintaining or improving privacy and protection throughout its life cycle.
• Anonymisation, encryption and aggregation techniques.
• The data subject may request its deletion.
• Language that is understandable and accessible to all potential users, including those with special needs.

A protocol for obtaining the consent of the persons concerned before initiating any data processing through the AI system.

The design and deployment of the system guarantees equal access and non-discrimination without compromising their private data and dignity.

The AI system will have a record of data access information that clearly identifies who accessed the data, when, and for what purpose.

Measures shall be taken to ensure that those responsible for data processing and management have the necessary skills to perform these functions and up-to-date knowledge of the subject matter.

The following measures shall be taken to ensure the adequacy and responsible use of the data used by the AI system:

• Procedure to ensure respect for the context of use, performance and outcome.
• Documentation of collection and preparation.
• Advanced security and privacy measures, such as pseudonymisation.
• Suitability for the needs of all users.
• Knowledge of ethical and legal measures.

The AI system is designed to ensure the principles of data minimisation and protection from the design phase onwards.

The AI system shall include an evaluation process to ensure that it does not reduce the number of possible options and does not compel or force certain decisions to be made.

The AI system shall have mechanisms in place to ensure:

• Adaptability to autonomy based on the preferences and requirements of each student.
• Not using manipulative or subliminal techniques or jeopardising student autonomy.
• Identifying that the content has been generated by an AI system.
• Awareness of interaction with an AI system.

The AI system shall have the following mechanisms:

• Human supervision measures to manage its degree of autonomy and adaptability.
• Mechanism to limit the degree of autonomy, preventing undesirable or high-risk behaviour.
• Measures to adapt and learn from its environment without compromising safety or ethical standards.

The AI system ensures that its operations are transparent, clear, honest and do not encourage students to overestimate their abilities.

Before being placed on the market or used, the AI system shall undergo an assessment of its impacts on people’s rights, safety, health and the environment. Particular attention shall be paid to the following areas:

• People’s quality of life.
• Sustainable development.
• Copyright and intellectual property.
• Social access and the digital divide.

The educational centre will guarantee protection, working conditions and fair remuneration for people working in the acquired AI system.

The system will have mechanisms in place to record its energy consumption and environmental impact throughout its life cycle.

The deployer or supplier shall implement mechanisms and strategies aimed at improving the energy efficiency of the system and reducing its environmental footprint.

Before being placed on the market or used, the AI system shall undergo an assessment covering at least the following aspects:

• Energy efficiency.
• Exploring the use of renewable energy sources and green computing practices.