“Compliance with this assessment list is not evidence of legal compliance, nor is it intended as guidance to ensure compliance with applicable law. Given the application-specificity of AI systems, the assessment list will need to be tailored to the specific use case and context in which the system operates. In addition, this chapter offers a general recommendation on how to implement the assessment list for Trustworthy AI though a governance structure embracing both operational and management level.” (High-Level Expert Group on AI, 2019, p. 24)“TRUSTWORTHY AI ASSESSMENT LIST (PILOT VERSION) “Diversity, non-discrimination and fairness Unfair bias avoidance: - Did you establish a strategy or a set of procedures to avoid creating or reinforcing unfair bias in the AI system, both regarding the use of input data as well as for the algorithm design? - Did you assess and acknowledge the possible limitations stemming from the composition of the used data sets? - Did you consider diversity and representativeness of users in the data? Did you test for specific populations or problematic use cases? - Did you research and use available technical tools to improve your understanding of the data, model and performance” (High-Level Expert Group on AI, 2019, p. 29)- “Did you put in place processes to test and monitor for potential biases during the development, deployment and use phase of the system? - Depending on the use case, did you ensure a mechanism that allows others to flag issues related to bias, discrimination or poor performance of the AI system? - Did you establish clear steps and ways of communicating on how and to whom such issues can be raised? - Did you consider others, potentially indirectly affected by the AI system, in addition to the (end)users? - Did you assess whether there is any possible decision variability that can occur under the same conditions? - If so, did you consider what the possible causes of this could be? - In case of variability, did you establish a measurement or assessment mechanism of the potential impact of such variability on fundamental rights? - Did you ensure an adequate working definition of “fairness” that you apply in designing AI systems? - Is your definition commonly used? Did you consider other definitions before choosing this one? - Did you ensure a quantitative analysis or metrics to measure and test the applied definition of fairness? - Did you establish mechanisms to ensure fairness in your AI systems? Did you consider other potential mechanisms?” (High-Level Expert Group on AI, 2019, p. 30)