The actors that are active in the financial world process vast amounts of information, starting from customer data and account movements over market trading data to credit underwriting or money-laundering checks. It is one thing to collect and store these data, yet another challenge to interpret and make sense of them. AI helps with both, for example, by checking databases or crawling the Internet in search of relevant information, by sorting it according to predefined categories or by finding its own sorting parameter. It is hence unsurprising that AI has started to fundamentally change many aspects of finance. This chapter takes AI scoring and creditworthiness assessments as an example for how AI is employed in financial services (Section 16.2), for the ethical challenges this raises (Section 16.3), and for the legal tools that attempt to adequately balance advantages and challenges of this technique (Section 16.4). It closes with a look at scoring beyond the credit situation (Section 16.5).

This chapter urges justice systems to recognize five key rights of the accused with respect to digital and machine evidence of guilt or innocence, in line with systemic goals such as accuracy, fairness, dignity, and public legitimacy: (1) front-end development and testing safeguards to minimize error and bias (based on a consensus view of algorithmic fairness); (2) meaningful and equitable pretrial access, including disclosure requirements, eliminating trade secret privileges, allowing defense testing, and defense access to potentially exculpatory technologies; (3) contestation, including a right to be heard in the development and testing process and access to experts both to review government evidence and develop defense evidence; and (4) a factfinding process that is epistemically competent but that also keeps a human in the loop, to protect equity and mercy, avoid automation complacency, and reject dehumanizing technologies. The chapter offers proposals to operationalize each right and discusses how each would apply to the various categories of digital and machine evidence discussed in Chapters 1–6.

Chapter 3 takes the conception of autonomy outlined in chapter 2 and explains how it grounds moral evaluation of algorithmic systems. It begins by offering a view of what it takes to respect autonomy and to respect persons in virtue of their autonomy, drawing on a number of different normative moral theories. The argument starts with a description of a K-12 teacher evaluation program from Washington, DC. It then considers several puzzles about the case. Next, the chapter provides an account of respecting autonomy and what that means for individuals’ moral claims. It explains how that account can help us understand the DC case, and we will offer a general account of the moral requirements of algorithmic systems. Specifically, we offer the Reasonable Endorsement Test, according to which an action is morally permissible only if it would be allowed by principles that each person subject to it could reasonably endorse. The chapter applies that test to the Loomis, Houston Schools, and Wagner cases. Finally, the chapter explains why the book does not focus directly on “fairness.”