Back Share
ChallengeInstances

There may be lack of accountability for centering stakeholder experience and agency in the design of AI driven by commercial interests

AI corporate-ethics education

AI systems may impact on relationships between decision makers such that judgements previously made at a professional level must involve technology providers. It may not be clear in advance how the user experience of AI tools has been considered in evaluation, nor what input those using the tool can have into ongoing development. This may occur in both smaller studies (e.g., individual site) or larger deployment.“When citizens are subject to decisions, predictions, or classifications produced by AI systems, situations may arise where such individuals are unable to hold directly accountable the parties responsible for these outcomes. AI systems automate cognitive functions that were previously attributable exclusively to accountable human agents. This can complicate the designation of responsibility in algorithmically generated outcomes, because the complex and distributed character of the design, production, and implementation processes of AI systems may make it difficult to pinpoint accountable parties. In cases of injury or negative consequence, such an accountability gap may harm the autonomy and violate the rights of the affected individuals.” (Leslie, 2019, p. 4)"The interface between A/IS and practitioners, as well as other stakeholders, is gaining broader attention in domains such as healthcare diagnostics, and there are many other contexts where there may be different levels of involvement with the technology. We should recognize that, for example, occupational therapists and their assistants may have on-theground expertise in working with a patient, who might be the “end user” of a robot or socialA/IS technology. In order to develop a product that is ethically aligned, stakeholders’ feedback is crucial to design a system that takes ethical and social issues into account. There are successful user experience (UX) design concepts, such as accessibility, that consider human physical disabilities, which should be incorporated into A/IS as they are more widely deployed. It is important to continuously consider the impact of A/IS through unanticipated use and on unforeseen interests."p.130-131

Overarching Principles Respect for persons
Reference
Title There may be lack of accountability for centering stakeholder experience and agency in the design of AI driven by commercial interests