In this chapter, law and technology scholar Jonathan Zittrain warns of the danger of relying on answers for which we have no explanations. There are benefits to utilising solutions discovered through trial and error rather than rigorous proof: though aspirin was discovered in the late 19th century, it was not until the late 20th century that scientists were able to explain how it worked. But doing so accrues ‘intellectual debt’. This intellectual debt is compounding quickly in the realm of AI, especially in the subfield of machine learning. Whereas we know that ML models can create efficient, effective answers, we don’t always know why the models come to the conclusions they do. This makes it difficult to detect when they are malfunctioning, being manipulated, or producing unreliable results. When several systems interact, the ledger moves further to the red. Society’s movement from basic science towards applied technology that bypasses rigorous investigative research inches us closer to a world in which we are reliant on an oracle AI, one in which we trust regardless of our ability to audit its trustworthiness. Zittrain concludes that we must create an intellectual debt ‘balance sheet’ by allowing academics to scrutinise the systems.

In this chapter, Fruzsina Molnár-Gábor and Johanne Giesecke consider specific aspects of how the application of AI-based systems in medical contexts may be guided under international standards. They sketch the relevant international frameworks for the governance of medical AI. Among the frameworks that exist, the World Medical Association’s activity appears particularly promising as a guide for standardisation processes. The organisation has already unified the application of medical expertise to a certain extent worldwide, and its guidance is anchored in the rules of various legal systems. It might provide the basis for a certain level of conformity of acceptance and implementation of new guidelines within national rules and regulations, such as those on new technology applications within the AI field. In order to develop a draft declaration, the authors then sketch out the potential applications of AI and its effects on the doctor–patient relationship in terms of information, consent, diagnosis, treatment, aftercare, and education. Finally, they spell out an assessment of how further activities of the WMA in this field might affect national rules, using the example of Germany.

In this chapter, the law scholar Christoph Krönke focuses on the legal challenges faced by healthcare AI Alter Egos, especially in the European Union. Firstly, the author outlines the functionalities of AI Alter Egos in the healthcare sector. Based on this, he explores the applicable legal framework as AI Alter Egos have two main functions: collecting a substantive database and proposing diagnoses. The author spells out that concerning the database, European data protection laws, especially the GDPR, are applicable. For healthcare AI in general, the author analyses the European Medical Devices Regulation (MDR). He argues that MDR regulates the market and ensures high standards with regard to the quality of medical devices. Altogether, the author concludes that AI Alter Egos are regulated by an appropriate legal framework in the EU, but it has to be open for developments in order to remain appropriate.

In this chapter, Mathias Paul explores the topic of AI systems in the financial industry. After outlining different areas of application of AI in the financial sector and different regulatory regimes relevant to robo-finance, the author analyses the risks emerging from AI applications in the financial industry. He argues that AI systems applied in this sector usually do not create new risks. Instead, existing risks can actually be mitigated through AI applications. The author then analyses personal responsibility frameworks that have been suggested by scholars in the field of robo-finance, and shows why they are not a sufficient approach for regulation. He concludes by discussing the Draft AI Act proposed by the European Commission as a suitable regulatory approach based on the risks linked to specific AI systems and AI based practices.

In this chapter, the ethics and international law scholar Silja Voeneky and the mathematician Thorsten Schmidt propose a new adaptive regulation scheme for AI-driven products and services. To this end, the authors examine different regulatory regimes, including the European Medical Devices Regulation (MDR), and the proposed AI Act by the European Commission and analyse the advantages and drawbacks. They conclude that regulatory approaches in general and with regard to AI driven high risk products and services have structural and specific deficits. Hence, a new regulatory approach is suggested by the authors, which avoids these shortcomings. At its core, the proposed adaptive regulation requires that private actors, as companies developing and selling high risk AI driven products and services, pay a proportionate amount of money as a financial guarantee into a fund before the product or service enters the market. The authors lay down what amount of regulatory capital can be seen as proportionate and the accompanying rules and norms to implement adaptive regulation.

This chapter by the philosopher Johanna Thoma focuses on the ‘moral proxy problem’, which arises when an autonomous artificial agent makes a decision as a proxy for a human agent, without it being clear for whom specifically it does so. Thoma recognises that, in general, there are broadly two categories of agents an artificial agent can be a proxy for: low-level agents (individual users or the kinds of human agents artificial agents are usually replacing) and high-level agents (designers, distributors, or regulators). She argues that we do not get the same recommendations under different agential frames: whilst the former suggests the agents be programmed without risk neutrality, which is common in choices made by humans, the latter suggests the contrary, since the choices are considered part of an aggregate of many similar choices. The author argues that the largely unquestioned implementation of risk neutrality in the design of artificial agents deserves critical scrutiny. Such scrutiny should reveal that the treatment of risk is intimately connected with our answer to the questions about agential perspective and responsibility.

In this chapter, Thomas Burri, an international lawyer, examines how general ethical norms on AI diffuse into domestic law directly, without engaging international law. The chapter discusses various ethical AI frameworks and shows how they influenced the European Union Commission’s proposal for an AI Act. It reveals the origins of the EU proposal and explains the substance of the future EU AI regulation. The chapter concludes that, overall, international law has played a marginal role in this process; it was largely sidelined.

In this chapter, the philosopher Mathias Risse reflects on the medium and long-term prospects and challenges democracy faces from AI. Comparing the political nature of AI systems with traffic infrastructure, the author points out AI’s potential to greatly strengthen democracy, but only with the right efforts. The chapter starts with a critical examination of the relation between democracy and technology with a historical perspective before outlining the techno skepticism prevalent in several grand narratives of AI. Finally, the author explores the possibilities and challenges that AI may lead to in the present digital age. He argues that technology critically bears on what forms of human life get realised or imagined, as it changes the materiality of democracy (by altering how collective decision making unfolds) and what its human participants are like. In conclusion, Mathias Risse argues that both technologists and citizens need to engage with ethics and political thoughts generally to have the spirit and dedication to build and maintain a democracy-enhancing AI infrastructure.

In this chapter, Philipp Kellmeyer discusses how to protect mental privacy and mental integrity in the interaction of AI-based neurotechnology from the perspective of philosophy, ethics, neuroscience, and psychology. The author argues that mental privacy and integrity are important anthropological goods that need to be protected from unjustified interferences. He then outlines the current scholarly discussion and policy initiatives about neurorights and takes the position that while existing human rights provide sufficient legal instruments, an approach is required that makes these rights actionable and justiciable to protect mental privacy and mental integrity, for example, by connecting fundamental rights to specific applied laws.

The author spells out the different key features of AI systems, introducing inter alia the notions of machine learning and deep learning as well as the use of AI systems as part of robotics.

In this chapter, the philosopher Christoph Durt elaborates a novel view on AI and its relation to humans. He contends that AI is neither merely a tool, nor an artificial subject, nor necessarily a simulation of human intelligence. These misconceptions of AI have led to grave misunderstandings of the opportunities and dangers of AI. A more comprehensive concept of AI is needed to better understand the possibilities of responsible AI. The chapter shows the roots of the misconceptions in the Turing Test. The author argues that the simplicity of the setup of the Turing Test is deceptive, and that Turing was aware that the text exchanges can develop in much more intricate ways than usually thought. The Turing Test only seemingly avoids difficult philosophical questions by passing on the burden to an evaluator, who is part of the setup, and hides in plain sight his or her decisive contribution. Durt shows that, different from all previous technology, AI processes meaningful aspects of the world as experienced and understood by humans. He delineates a more comprehensive picture according to which AI integrates into the human lifeworld through its interrelations with humans and data.

In this chapter, the law scholar Boris Paal identifies a conflict between two objectives pursued by the data protection law, the comprehensive protection of privacy and personal rights and the facilitation of an effective and competitive data economy. Focusing on the European Union’s General Data Protection Regulation (GDPR), the author recognises its failure to address the implications of AI, the development of which depends on the access to large amounts of data. The regulation is observed as not only immensely burdensome for controllers but also likely to significantly limit the output of AI-based applications. In general, the main principles of the GDPR seem to be in direct conflict with the functioning and underlying mechanisms of AI applications, which evidently, were not considered sufficiently whilst the regulation was being drafted. Hence, Paal argues that establishing a separate legal basis governing the permissibility of processing operations using AI-based applications should be considered; the enhanced legal framework should seek to reconcile data protection with the openness for new opportunities of AI developments.

In this chapter, political philosopher Alex Leveringhaus asks whether Lethal Autonomous Weapons (AWS) are morally repugnant and whether this entails that they should be prohibited by international law. To this end, Leveringhaus critically surveys three prominent ethical arguments against AWS: firstly, AWS create ‘responsibility gaps’; secondly, that their use is incompatible with human dignity; and ,thirdly, that AWS replace human agency with artificial agency. He argues that some of these arguments fail to show that AWS are morally different from more established weapons. However, the author concludes that AWS are currently problematic due to their lack of predictability.

In this chapter, the law scholar Ebrahim Afsah outlines different implications of AI for the area of national security. He argues that while AI overlaps with many challenges to the national security arising from cyberspace, it also creates new risks, including the emergence of a superintelligence in the future, the development of autonomous weapons, the enhancement of existing military capabilities, and threats to foreign relations and economic stability. Most of these risks, however, Afsah concludes, can be subsumed under existing normative frameworks.