Health Contracts

The system will be accompanied by precise information on how it works, its limitations, how it achieves a given result and how it contributes to a clinical decision, as well as the identity and contact details of the provider or representatives.

It shall be ensured that the process of generating system results complies with the following requirements:

• It is well documented in an understandable and accessible language for clinical users to detect future problems.
• It uses the simplest and most intelligent model in favor of transparency.
• It takes into consideration the artificial intelligence skills and literacy of all users.
• It performs automatic records of its operation.
• It is reproducible.

The AI system will be accompanied by additional materials as necessary to enable clinical users to understand how the AI system works and to explain it to patients.

The following information related to the system will be documented and made public:

• Its development process and the people who have been involved.
• The technical information necessary for its use.
• Concise, clear instructions, written in an accessible language, on how it works.
• The process by which the system arrives at a given result and the expected results.
• The possible limitations of the system.
• Information related to the exchange and cession of data as well as the ways and mechanisms that allow interested parties to exercise their rights.

The system will have a mechanism that allows the identification of the content generated as a result of its creation.

The system will have a mechanism to identify the most relevant clinical variables used when making a prediction.

The healthcare organization and clinical users will be informed about the metrics that have been used to optimize and validate the AI system as well as the validation methodology to develop the system.

Measures will be taken to ensure that the level of transparency and explainability of the system does not pose a risk to the privacy of individuals or the security of the system itself.

It will be guaranted that the system has been validated in the clinical setting, by professionals in the field in which the system is intended to be used, that the results it gives are well explained and understandable, are appropriate and help in the task to be performed.

The system will have the following communication tools:

• A function that allowed clinical users and patients to comment on its performance.
• A notification system in case of serious incident or error.

A procedure will be established to communicate on a regular basis information on which clinical decisions the participants are assisted by an AI system.

In relation to the data used by the system, the following requirements will be ensured:

• The system will be accompanied by documentation regarding the type of data used, its origin, and the changes it has undergone in each phase of its life cycle.
• Measures will be taken to verify the representativeness and adequacy of the data used in relation to the population to which it will be applied.
• The system will have safeguards to verify the age of individuals.

A policy of participation of different interest groups (stakeholders) will be established, including clinical users and patients.

The system will have measures and mechanisms to detect, correct and mitigate possible errors throughout its life cycle.

In particular, the measures will be aimed at the lower levels present in:

• The design or implementation phase.
• Training and post training data.
• Categories changed or inadequate.
• Cognitive biases (e.g., automaticity bias and confirmation bias).

Before its commercialization/use, the system will be subjected to an evaluation of the impact of its possible variations, especially those that could affect people’s rights, health or safety.

The system will comply with classification standards that address age, ethnicity, race, gender or disability-related biases in accordance with applicable regulations.

The IA system will include measures of inclusion and equity, especially in relation to the following aspects:

• Testing of the results of the system for different affected groups.
• Measures to increase accessibility.
• Measures to encourage its use by people in vulnerable situations.
• Measures to ensure that the system adapts to a wide range of individual preferences and abilities.

The AI system development group should include people of non-male gender, people from diverse backgrounds or cultures and from different disciplines of knowledge (beyond data science, engineering and health sciences).

The system will be accompanied by the following information:

• The level of accuracy and security of the system.
• The manner and degree of influence that the actions of users may have on the performance of the system.
• The possible behaviors and scenarios that should never be transgressed in order to guarantee safety and preserve the non-maleficence of the system.
• The mechanisms of supervision and control of the system (both internal and external).
• User guides and best practices for AI systems written in a language accessible to clinical users.

The system will have clear and defined mechanisms for clinical users to contact the developers of the AI system when necessary.

The system shall have mechanisms to ensure the periodic updating of the information available on the performance of the IA system.

Prior to its commercialization/use, the system will be subject to an evaluation that incorporates the following aspects:

• The possible impacts of the system on the health, safety and rights of the people affected.
• The risk that data collected, generated or used by the system is used to discriminate against people in a negative way.
• The possible risks and damages that would be caused by the system in case of inaccurate predictions in the predicted scenarios.
• The risks involved in using the system beyond its intended uses.
• The possible risks derived from the fact that people from a clinical specialty different from the main actor are involved during the deployment of the solution.

The system will include the following measures:

• Resistance measures against errors arising from interaction with other systems or people.
• Resistance measures and mitigation of errors arising from attempts of manipulation by unauthorized persons.
• Periodic evaluation or self-evaluation methodologies recognized to ensure the proper functioning of the system and to foresee possible malfunctions.

It will be guaranteed that prior to its commercialization/use the system will have been subjected to evaluations that corroborate its conformity with the applicable legal requirements.

The system will have means that allow the following functions:

• Identify, evaluate and adopt measures with respect to the risks arising from its use and the data collected after it becomes operational.
• Report residual system risks that have not been corrected or mitigated.

The system will be equipped with tools to validate and clinically contrast the results.

Prior to the use/commercialization of the system it will be determined:

• What will be the contribution of the system in clinical decision making.
• The degree of responsibility of the people involved in the control, operation and human supervision of the system.
• The people responsible for updating the system.

It will provide the people involved in the control of the decisions taken by the system, especially the clinical users, with adequate training so that they have an adequate understanding of how it works and have the necessary training to perform this function.

The system will have the tools that will allow the people in charge of its control:

• Intervene, modify, eliminate or reverse decisions made by the system.
• Enable modifications in the clinical protocol for data processing and use of the system.
• Limit the use of the system for specific patient groups or clinical contexts when performance is unacceptable.

The system will be accompanied by information and documentation related to the following aspects:

• The limits of the control mechanisms of the system.
• The human supervision measures available in the system.

It will be guaranteed that in the development of the system have participated clinical experts in the context of the solution, so that the developer has had knowledge of it.

Prior to the use/commercialization of the system it will be determined:

• The person legally responsible for the actions and/or decisions they take throughout their life cycle, in particular, what is the degree of responsibility attributed to the clinical user, the health organization to which he/she belongs, and the creator of the system.
• Mechanisms for periodic review of accountability and review of accounts with the different agents involved in the development and use of the IA system.

Prior to the commercialization/use of the system, the following measures will be taken:

• Facilitate the demonstration of compliance with current legislation.
• Determine forms of reparation or compensation in case of damages caused by the system, which shall include the responsibility of the persons who have created the system when appropriate.
• Determine protection figures for users and persons affected by the system.
• Determine whether the AI system can be considered as a medical device according to the Medical Device Regulations (MDR) or in vitro diagnostic (IVDR) and how it would be classified according to these regulations.

The system will have data protection mechanisms present throughout its life cycle aimed at guaranteeing the following aspects:

• That personal data will be treated in accordance with the level of protection legally required in order to avoid misuse, alternation, loss, or disposal by third parties.
• That the data provided by the person concerned are not sold, logged, or transferred without their consent.
• That the system has processes to maintain or improve data privacy and protection throughout its life cycle.
• That anonymity, encryption and aggregation techniques are used.
• That there is a protocol to obtain the consent of the affected persons before initiating any data processing through the system, and the information related to these actions is understandable and accessible to all possible users, including those with special needs.
• That there is a mechanism through which the person concerned may request that their information be removed.

The system will have a record of the data access information that will clearly identify who is accessing the data, when and for what purpose.

Measures will be taken to ensure that the persons responsible for data processing and management have the necessary skills to perform these functions and updated knowledge on the subject.

In order to guarantee the adequacy and reusability of the data used by the system, the following measures will be taken:

• A procedure shall be established to ensure that the sets of data used are adequate and relevant to the performance and expected results of the system and that they are appropriate for the context in which the system will be used.
• The process of collecting and preparing the data used will be documented.
• The system will use privacy measures (such as pseudonymization of data) and limit the reuse of sensitive personal data.

The system is/has been designed in such a way as to ensure that it adheres to the principles of data minimization and data protection from design and by default.

The system will be accompanied by an evaluation process to ensure that it does not reduce the number of possible choices and does not oblige or force them to make certain decisions in general or at any stage of their life cycle.

The system will be accompanied by a procedure that will allow patients to question:

• The results and the intended use of the system.
• Specific aspects about the use of your data.
• The existing gaps, the validation processes and the expected performance of the IA system.
• The division in the contribution to decision making between the AI system and the clinical user.

The system will be accompanied by a verification protocol to certify that it helps people make better and more informed decisions.

The system will include mechanisms that guarantee the following aspects:

• That clinical users are aware that they interact with an artificial intelligence and are sufficiently trained and informed to do so.
• That patients are informed about their exposure to the AI system and how it works.
• It does not jeopardize the autonomy levels of clinical users or patients and adapts to different preferences and needs.

The system will have the following mechanisms:

• Human supervision measures that make it possible to manage their degree of autonomy and adaptability.
• Mechanism for limiting the degree of autonomy of the system to avoid undesired or high-risk behavior.
• Adapt and learn from their environment without compromising safety and ethical standards.

It will ensure that in the design, implementation and use of the system, consideration has been given to the limitations and obstacles arising from the digital divide and unequal social access.

Prior to its commercialization and/or use, the system will be subject to an assessment of its impact on human rights, safety, health and the environment.

This evaluation takes the following areas into special consideration:

• Impact on the labor rights of the healthcare organization’s personnel.
• Impact and risks related to sustainable development.
• Measures of fair compensation and ethical treatment to the people involved in all phases of the life cycle of the system and those affected by its implementation and use.

The system will have mechanisms that will allow it to record its energy consumption and environmental impact throughout its life cycle.

Prior to its commercialization and/or use, the system will be subjected to an evaluation in which the following aspects, as a minimum, will be evaluated:

• The energy efficiency of the system.
• Explore the use of renewable energy sources and green computing practices.