Introduction
We can observe a growing number of algorithmic decision-making (ADM) systems and applications of artificial intelligence (AI) in various areas of society. Their more widespread use means that they increasingly support or replace human decision-making. The utility of such applications largely stems from an ability to process information and to deal with certain cognitive tasks in a way that humans are incapable of. Indeed, ADM systems have already found their way into widely different domains, informing financial transactions, targeted advertising based on behavioral predictions, and medical diagnoses, to name just a few examples (e.g., Crawford and Calo Reference Crawford and Calo2016; Eubanks Reference Eubanks2017).
In light of these developments, it indeed seems “unlikely that ICTs [Information and Communication Technologies] would have a profound effect on society, economy, culture, etc., but not on politics” (Wohlers and Bernier 2016, p. 30). What, then, do the capacities of ADM systems mean for politics and democratic decision-making processes? Can they produce better political decisions or even—at least partially—replace political actors in the decision-making process? Political scientists are likely to frown at this question and would perhaps want to quickly dismiss it as nonsense. In fact, our discipline has left such issues almost exclusively to scholars from neighboring fields, such as law, data science, philosophy or science and technology studies.
Yet, as recent claims show, we should better be prepared to engage in an academic discussion about such issues. Already, one can find journalistic and scholarly contributions to the public debate that seriously consider using algorithmic systems in political decision-making in democratic systems of the Western worlds (Carpio 2018; van der Wal and Yan 2017) or even openly endorse a setting in which humans are at least partially governed by machines (Lisi Reference Lisi and Brockman2015). Where intelligent machines are deemed unfit for this task of aiding in political decision-making, this is usually not due to concerns derived from fundamental concepts of political science, but rather because of a lack of maturity of these systems. Moreover, the idea of employing machines in democratic politics does not seem completely remote from the point of view of the citizenry. A survey by the Center for the Governance of Change at the Spanish IE University (2019) found that in some European countries, such as Germany or the Netherlands, some 30 percent of citizens could see their politicians replaced by smart machines. While it is not certain how to interpret this finding—does it reflect trust in machines or dissatisfaction with politicians?—it nonetheless suggests that citizens themselves might at some point give a chance to machines taking over parts of political decision-making. Some countries are already experimenting with this possibility. In Japan—an established democracy of the developed world—an AI-based robot ran for mayor in the Japanese city Tama near Tokyo, with the goal of replacing politicians entirely by 2050 (Cole 2018).
These signs of a readiness to endorse machine intelligence in politics are also well in line with a certain discontent with existing democratic politics among big tech companies. As Nemitz (Reference Nemitz2018) has noted, there is a tendency in the tech sector to see liberal democracy as a flawed arrangement that needs to be overcome; and to see technology as the means to address mankind’s various challenges. Broussard (2018) recently termed this mentality “technochauvinism,” and Morozov (2014) has similarly diagnosed a mindset of solutionism, referring to the idea that a technological fix can be found for any human-made problem, including social and political ones. When seen against the backdrop of a widespread dissatisfaction with the working of democracy, politicians, and political parties (Dalton Reference Dalton2008), the promise of solutionism may well have particular appeal. Indeed, in view of the messiness of democratic politics and frequent news about politicians’ errors and wrongdoings, machine agents may be appealing as they are not plagued by ideological stubbornness, psychological biases, or personal flaws. In fact, such arguments were used by the supporters of the robot-candidate running for mayor in Tama city (Cole 2018).
However, in what follows, we show that algorithms are not a quick fix to the messiness of democratic politics, because using machines in political decision-making risks underestimating or misrepresenting the inherent complexity and nature of democracy itself. From the point of view of political science, as we will detail below, a central problem lies in the fact that the idea of producing better decisions with the help of machines would require that there is some clear standard of what constitutes a good decision in the first place. This is, however, a complex, context-sensitive, and thorny question that scholars can hardly ever hope to reach agreement on. It is therefore important to explore if and under what conditions ADM systems can play a significant role in democratic politics.
To do so, we first spell out the fundamental properties of liberal-democratic politics that form hard and unalterable barriers to using machines for better political decision-making. Second, we also discuss where machines can actually play a role in political decision-making and on what this depends.
The capabilities of ADM systems and AI
The promise of ADM systems rests on technological advances in the collection and large amounts of data and in its processing with machine learning techniques. These have opened up new capabilities for dealing with specific cognitive tasks in ways that can surpass humans. Leaving behind older forms of AI, realized as deductively created expert rule systems, applications of machine learning can inductively generate or adapt decision rules based on the processing of data. This means that they learn to make decisions that optimize a pre-defined performance criterion, e.g., maximizing expected utility. A widespread form of machine learning systems analyzes empirical data (labeled training data) to “learn” a model that best predicts a feature of interest. Taking the example of credit default risk scoring, a learning algorithm is used to find associations between already known outcomes, also called ground truth (the labels, credit default versus no default), and individual characteristics (such as consumption patterns). The goal is to estimate the effects of these characteristics on the outcome variable credit default risk by finding a model that best fits the data (based on a pre-defined performance criterion, such as overall expected utility).
In any case, these applications generate a model about a part of reality that then forms the basis for decision-making. As they can embody “complex rules that challenge or confound human capacities for action and comprehension […] or whose decision-making logic is difficult to explain after the fact” (Mittelstadt et al. Reference Mittelstadt, Allo, Taddeo, Wachter and Floridi2016, p. 3), these applications acquire a certain degree of autonomy when performing a cognitive task—like predicting credit default, image recognition or winning chess games—in a way that is not explicitly pre-programmed.
While such ADM systems differ from expert systems with pre-programmed rules, as they instead acquire decision rules from data (Franklin 2014), they still only fall under weak or narrow AI since they serve to deal with specific tasks. Within the category of narrow AI, however, applications may vary considerably in the complexity of the tasks and their learned decision model, whether they are automated, and whether they are static (trained once) or dynamic (learn and adapt themselves based on new data inputs). Some ADM systems, like algorithmic models that have been trained with suitable data, are more like tools that people can use and query to provide, e.g., a prediction about an outcome of interest. Other applications more clearly fall under the ambit of modern AI as they implement the concept of a rational agent which selects actions to optimize a performance measure through both interaction with and adaptation to its environment (Russell and Norvig Reference Russell and Norvig2016, pp. 36–38). This holds true as much for an automated vacuum cleaner as for a software agent optimizing financial investment decisions. While this differs from human agency, which also comprises non-rational aspects, independent learning processes leading to decisions with consequences of moral importance lend ADM systems a socially relevant agency (Turner 2019, pp. 65–66).
In any case, the autonomy and complexity of ADM systems means that most people can hardly comprehend the operations of these applications. Various scholars have therefore pointed to problems of transparency and accountability that arise with the use of algorithmic systems (Janssen and Kuk Reference Janssen and Kuk2016; Lepri et al. Reference Lepri, Oliver, Letouzé, Pentland and Vinck2018; Mittelstadt et al. Reference Mittelstadt, Allo, Taddeo, Wachter and Floridi2016). This, however, is likely an inherent quality of having technologies which can perform tasks that humans cannot (similarly) solve. Indeed, the bottom-up, data-driven approach to building forms of machine intelligence solving cognitive tasks has been remarkably successful. Existing applications already assist humans even in rather sophisticated tasks. For instance, so-called legal AI applications can be used to retrieve and arrange information in ways that are usually done by paralegals; and the decision-making quality of AI agents is apparently already valued at the high-level of management in a business context (Mayer-Schoenberger and Ramge Reference Mayer-Schoenberger and Ramge2019).
Various scholars have also suggested that the analytical capabilities of machine learning can enhance political decision-making. However, existing contributions see the potential of improved decision making on rather different levels. They refer to the larger process of policy making (Höchtl et al. Reference Höchtl, Parycek and Schöllhammer2016), to the management of processes and resources in so-called smart cities (Meijer and Bolívar Reference Meijer and Bolívar2016) and to administrative decision-making (Chen and Hsieh Reference Chen and Hsieh2014). Overall, there is a tendency to stress the information-processing and knowledge-generating part of technological applications, and little attention is given to genuine features of democratic politics which can heavily reduce the relevance of information and knowledge (for an exception see van der Voort et al. Reference van der Voort, Klievink, Arnaboldi and Meijer2019). In order to assess the usefulness of ADM systems and AI in political decision-making, it is necessary to formulate a more differentiated account: Where do the hard constraints for adopting ADM systems in democratic political decision-making lie? Which are the areas where governments can meaningfully apply these systems?
Standards of good political decision-making and ADM systems
On the ambiguity of output-based standards in political decision-making
ADM systems aim to solve a given problem which is why their quality is assessed in terms of producing “good” decisions with regard to some performance measure. This makes them generally compatible with the standard of output legitimacy (Scharpf Reference Scharpf1999) in democratic politics. However, there are at least three important limitations to purposefully using ADM systems for improving political decision-making in terms of output legitimacy: (1) the lack of a ground truth needed for an optimization process; (2) the fragile link between outcomes to preceding political decisions, including in citizens’ perceptions; and (3) the malleability of decision contexts and public perceptions.
Lack of ground truth. Following the idea that AI can serve to inform decisions in ways that optimize the quality of outputs, these systems could be used to produce decisions that maximize the wellbeing and satisfaction of citizens. However, while economists have tried to quantify and measure the aggregate welfare and life quality of societies (see, e.g., Rojas, 2019), output legitimacy deliberately leaves open which outputs lead to the satisfaction and well-being of citizens. There is no pre-defined notion of what good outputs are. Under a pluralist liberal democracy, marked by competing legitimate views, nobody can claim to hold the one correct notion of what is the best course for a polity. There are thus competing conceptions of what is good as a substantive standard by which to evaluate political decisions. Also, many political decisions have (re-)distributional dimensions or involve value questions. This means that while a decision may be in the interest of some citizens, it is not in the interests of others.
Even thornier are decisions about moral issues (Knill Reference Knill2013). Ultimately, politicians must make decisions for the entire society. Yet, citizens may hold entirely incompatible moral beliefs, e.g., regarding same-sex marriage or abortion, where there is little or no room for compromise. Such issues are clearly not a matter of solving a cognitive problem. They are fundamentally about incommensurable beliefs and values. The capacities of ADM systems for producing better decisions are of no use in resolving such value-laden issues. There simply is nothing like a “ground truth” that machine intelligence could be trained upon.
Fragile link between outcomes and preceding decisions. Admittedly, one could object that some political decisions are based on strong consensus about what is desirable—e.g., less crime. However, even where political decisions are about a quantifiable utility, the perceptions and evaluations of citizens are not necessarily tightly linked to the decision itself. Sometimes, it is not clear per se whether a political decision has been the cause of an improvement of some outcome or if other factors have shaped it. This element of uncertainty is present especially where a policy may only show its effect in the long term—and isolating this effect from other potential influences is immensely demanding (Jacobs Reference Jacobs2008). Moreover, citizens may be misinformed or biased when they evaluate a change in outcomes. If, for example, a political decision leads to a broadly desirable objective change, such as a significant reduction of robberies, citizens may nonetheless be dissatisfied, because they are uninformed, or because their evaluations are biased by ideological and identity-based filters (see, e.g., Achen and Bartels Reference Achen and Bartels2017). These filters imply that an improvement of objective conditions achieved by a governing party may not be seen positively by citizens who strongly identify with a different party. Further, even if a political solution to a policy problem is evaluated positively by citizens at the time of the decision, citizens’ retrospective evaluations may change. This may be due to negative long-term effects of the policy or simply because the public has acquired a different perspective on the former decision. What counted as “good” at an earlier time may become unpopular in retrospect.
Malleability of decision contexts. An additional and related limitation to using the capacity of ADM systems for political decision-making stems from a lack of regularity in political decisions. In order for an algorithmic system to “learn” a model which can then be used to decide about new cases, ADM systems require enough data as well as regularity in decision situations to perform well. In politics, it is common to decide on matters which are novel at least in some way. Even where similar decisions have been made before, this might have been under very different circumstances, possibly leading voters to evaluate the decision very differently. Hence, political decision-makers often face rather unique decision contexts with considerable uncertainty about the consequences of their decisions.
This is different from other domains of decision-making where the decision situations are much more structured. Certain business management decisions, for instance, rest on data from a large number of previous events (e.g., investments) with lots of regularity. This enables an ADM system to extract relevant variables for predicting the impact of different decision options. What furthermore simplifies the decision-making, e.g., in a business context is that there are quantifiable value trade-offs—which can be largely or entirely monetarized. In contrast, the impacts that policy decisions have for different areas are not usually amenable to an easy quantification (on this see also Veale and Brass Reference Veale, Brass, Yeung, Lodge, Yeung and Lodge2019). For instance, while predictive policing based on ADM systems might lead to a lower crime rate or less fear of crime by citizens, it may come at the price of restricting civil liberties—a consequence that is hard to quantify.
Altogether, politics is about making decisions under conditions where a ground truth is absent and where objectives are usually contested as well as marked by ambiguity and malleability. Good decisions are therefore not a question of superior knowledge. Because of the lack of clear standards for good decisions in terms of outcomes, procedural standards of legitimacy are a crucial element of democratic politics.
The importance of procedural criteria of good decision-making
To deal with the inevitable fact of pluralism and to still produce collectively binding decisions that find broad acceptance, liberal democracy relies on procedures that structure and contain the conflict between different views in a society. This means that democratic politics is not merely about realizing certain outputs, but mainly about how to realize these outputs—an aspect that is emphasized in a procedural conception of liberal democracy. It maintains that while there is disagreement about what the right or best political decisions are, there can nevertheless be agreement about the rules by which such decisions are taken (Przeworski Reference Przeworski, Shapiro and Hacker-Cordón1999). In that sense, democratic government is not primarily designed to deliver certain outputs, but about guaranteeing the continued equal freedom of all citizens to take part in the will formation and decision-making process (Urbinati Reference Urbinati2014). This freedom entails a guarantee of being able to revise previous decisions and the possibility of ongoing contestation. These provisions foster responsiveness and accountability, and they are supposed to ensure that no position of power permanently dominates over others (Shapiro Reference Shapiro1996). As such, the procedural understanding of liberal democracy conforms to what Scharpf (Reference Scharpf1999) has called input-legitimacy, but it additionally posits that the political process has to conform to certain rules and procedures (throughput-legitimacy) (Schmidt Reference Schmidt2013).
Citizens’ equal freedom of taking part in the process of will formation furthermore presupposes the existence of certain civic and political rights, such as freedom of speech and political equality. Without them, the reversibility of political decisions based on the equal possibility of citizens to participate in public will formation can hardly be attained (Gutmann and Thompson Reference Gutmann and Thompson2004; Knight and Johnson Reference Knight and Johnson2011). From this, it follows that democratic politics needs respect certain limits in the form of rights and protections and the rule of law. These limits imply that only because a decision is supported by a majority of citizens, this does not already make it a “good” decision as it may be in conflict with basic rights (Merkel Reference Merkel2004). In that case, citizens could contest, and a supreme court, for instance, could overrule a decision due to its incompatibility with constitutional law.
All in all, decisions count as “good” according to the procedural standard of democratic legitimacy if they are the result of commonly accepted rules and procedures that regulate the use of public power.Footnote 1 This process allows for correcting previous decisions and thus for learning and adapting. While this may appear similar to the working of ADM systems, such learning is, however, fundamentally different from learning and decision-making by machines (Hildebrandt Reference Hildebrandt2016). ADM systems process inputs aiming to best realize certain objectives or criteria. In contrast, democratic politics, while it too entails a processing of inputs, is not about the optimization of pre-defined goals but about finding out what the guiding goals should be in the first place. Liberal democracy, in that sense, is a social technology designed to deal with societal complexity in a specific way. It allows for realizing commonly accepted collective decisions against the backdrop of a pluralism of conflicting views and preferences. This acceptance is based on procedures that regulate social conflict by leaving open which decisions to take and which goals to set in the future in face of new developments.
The corresponding social and political learning process exhibits features that are not susceptible to computation and optimization. First, openness of the democratic political process is particularly important when considering the unavoidable contingency in politics. It is part of its very nature that there is always the possibility of breaking with the past and seeing things differently. Indeed, for political theorists such as Arendt (Reference Arendt1998), democratic politics is a way of facing and dealing with the inescapable unpredictability and contingency of the world. Accordingly, political action and decision-making are geared toward constantly renegotiating a gap between the past and the future without being able to count on a secure foundation.
Second, this openness of democratic politics is tightly linked to the fact that political will formation and judgment are not simply a question of information processing but are marked by a hermeneutic element of interpretation and reinterpretation of ideas, norms, and values (Hildebrandt Reference Hildebrandt2016). Any political judgment and decision-making will occur against the backdrop of an already existing web of meaning. It is informed by said web, but also acts upon it. This occurs in the mode of language, which is always underdetermined and allows for interpreting and imagining social conditions differently. Societies may therefore change the way that they look at certain political issues—as, for example, the way that gender roles have changed over the last century and may well change further over time.
From this, it altogether follows that no machine can relieve citizens of the burden to form their political will. This is not a question of optimizing a given objective. Individuals have to deliberate—individually as well as with others—about how they think about social issues. Algorithms therefore simply have no place in the process of realizing political decisions as setting the goals and values of a society.
Barriers to using Algorithms in political decision-making
Although knowledge, e.g., about the state of society, the economy etc., may inform political decisions, political decision-making in liberal democracy is not about solving cognitive problems and about using knowledge to best realize given objectives. It is primarily about regulating conflict, achieving mutual understanding, and producing decisions that need to find broad acceptance under conditions of pluralism.
On the one hand, ADM systems do have a clear potential to inform decision-making, as they produce insights from learning and identifying patterns. On the other hand, as applications modeled after rational agents that optimize a performance measure in a given environment (Russell and Norvig Reference Russell and Norvig2016, p. 37), ADM systems can only serve to instrumentally deal with a cognitive task by helping to find the best means for given ends. Building on the discussion of barriers above, such decision-making contexts need to exhibit (1) relatively unambiguous and stable objectives—and therefore without a need of reinterpreting or reevaluating guiding goals and values—and (2) a high degree of regularity in decision situations.
When distinguishing forms of political decision-making in terms of how much they realize these two conditions, as illustrated in Fig 1, one will arguably find systematic differences between policies. Clearly, even within policy areas, different concrete issues can vary in terms of how contested they are. Also, issues for which objectives and value priorities are largely uncontested may well become politicized again—moving them further to the top in Fig 1. Nonetheless, some issues and societal problems will be, on average, more compatible with algorithmic systems than others. In highly value-laden moral policy issues, for instance (A in Fig 1), knowledge and evidence are of little relevance. Deciding about, e.g., abortion law is not a matter of problem-solving and optimizing as there are entirely incommensurable views on what “the problem” is in the first place. The same is true for many other moral issues, such as assisted suicide or the genetic testing of embryos.
The relevance of evidence in political decision-making
In other issue areas, however, finding the best means for achieving general objectives is less contested. This is the case for so-called valence issues (Stokes Reference Stokes1963), i.e., issues characterized by a strong degree of consensus regarding objectives. Certain economic policy issues (e.g., generating employment) exhibit a high degree of societal agreement about what are desirable outcomes. This means that evidence about the probable impact of a policy course may well be relevant for top-level political decisions. For instance, if there is strong evidence that raising taxes in a financial crisis will likely destroy employment, this information about the relation between means and ends is arguably relevant for the decision calculus.
However, while there may be more scope for evidence to inform decision-making on some issues, this is a question of degree. Even where consensual objectives like employment, growth, or security exist, distributional questions and trade-offs between such generally desirable goals remain. Resolving these necessitates prioritization and involves value judgments, which are again not a matter of evidence. Moreover, decision situations in high-level politics will often be incomparable to previous decision settings. Put differently, such decisions are not so frequent, well structured, and repetitive that machines (or humans) could learn how to best realize desirable outcomes under differing circumstances (B in Fig 1). Algorithmic systems are thus of little help in deciding which political course to pursue.
In sum, while we find suitable conditions for employing ADM systems in the bottom right of Fig 1 top-level political decisions will not fall in this area. Rather, it is only once high-level decisions are made and policy goals are defined that evidence becomes thoroughly instrumental for policy-making choices within issue areas. Specifically, this is the case for decisions about concrete policy instruments as the means to achieve a given objective (which has been set based on democratic procedures before). In this setting, objectives are pre-defined and finding appropriate means for a narrowly defined task is likely to show more repetition and regularity in decision situations. On this level of decision-making common to public administration and “low politics,” ADM systems may well be of assistance.Footnote 2 We will therefore turn to the possible role of ADM systems in the selection of implemented policy instruments in the following section.
ADM systems and AI as cognitive enhancements in policy making
Algorithmic decision-making and policy analytical capacity
Based on the preceding considerations, ADM systems have no place at the very heart of democratic politics: the formation of a political will and of setting the guiding goals and values for a society. They can, however, contribute to decision-making on the level of choosing and implementing policy measures for realizing previously defined objectives that emerged from the democratic process. The task is then to find the best solution to solve a given problem or maximize a certain good. Knowledge and evidence play an important role in dealing with such means–ends relations and for increasing the effectiveness and efficiency of policy decisions (Sanderson Reference Sanderson2002).
Human cognitive capacities for instrumental problem-solving alone hardly suffice for reaching good solutions to complex problems: Their capabilities for solving problems are usually small in relation to the complexity of problems, and inferring an optimal solution from given information about a multifaceted problem is compromised by various psychological biases and reliance on heuristics (see, e.g., Elster Reference Elster2007; Tversky and Kahneman Reference Tversky and Kahneman1991; for an application in the field of politics see Stolwijk and Vis 2020). In sum, as Simon (Reference Simon1990, p. 7) has noted, human rational behavior “is shaped by a scissor whose two blades are the structure of task environments and the computational capabilities of the actor.” ADM may help to overcome these limitations as extensive information processing for evidence-based decision-making promises to improve policy outcomes—or at least to avoid policy failures (Howlett Reference Howlett2009, p. 157). The source of such failures, according to Howlett (Reference Howlett2009, p. 161), is often a lack of what he calls “policy analytical capacity,” which involves, as a core component, adequate forms of information management. This analytical capacity stands for “the amount of basic research a government can conduct or access, its ability to apply statistical methods, applied research methods, and advanced modelling techniques to this data and employ analytical techniques […] in order to gauge broad public opinion and attitudes […] and to anticipate future policy impacts” (Howlett Reference Howlett2009, p. 162). Here, the capacities of ADM systems may help to avoid policy failures because they can produce relevant insights through the capacity to synthesize information into actionable knowledge (Höchtl et al. Reference Höchtl, Parycek and Schöllhammer2016; Kettl Reference Kettl2016). By performing scoring and classification tasks that are used for predictions or risk assessments and data-driven policy simulations, ADM systems can serve to assign scores to policy choices indicating their expected success or failure. They very much suit the idea that “governments can better learn from experience and both avoid repeating the errors of the past as well as better apply new techniques to the resolution of old and new problems” (Howlett Reference Howlett2009:154).
Specifically, one can feed ADM systems with data about policy instruments adopted in different contexts and about their consequences so that they “learn” relations between policy instrument choice and policy outcomes. Through systematically harnessing data on a whole range of features that describe the decision situation, one might use ADM systems similarly to medical analyses of drug effects: Processing huge datasets about the use of drugs, including in combination with other drugs, to uncover unknown treatments for diseases as well as interactions between drugs that have undesirable effects (Costa Reference Costa2014). This could work equally well in the area of policy instrument choices—which is akin to “treatments” for dealing with identified societal problems. Furthermore, cumulative data and experience allow for developing ever-more refined models about expected effects of policy instrument choices.
To some extent implemented ADM systems already realize these capabilities. For instance, the US Food and Drug Administration’s (FDA) has piloted a system that informs regulatory activity. The algorithmic tool processes vast amounts of reports of adverse events for the purpose of detecting and addressing undesirable drug effects after launching a drug on the market. The agency aims to then use these results to adaptively inform its rulemaking and policy guidance.
Hence, insights into the suitability of policy instruments can inform higher-level political decision-making. Conceivably, strong evidence that certain instruments are unlikely to achieve desired effects deters political actors from pursuing a certain course for achieving a policy objective. Notably, the use of algorithmic models to inform policy choices occurs largely for risk assessments, resource planning, and fiscal planning. These are decision areas which are comparatively structured, where there are clearly measurable impacts, and where outcomes and trade-offs can be quantified.
These conditions are even more likely to be present on the operative level of administrative decision-making, where most ADM tools are, to date, implemented by government bodies. Taking the example of New Zealand as an advanced adopter, the government’s Algorithmic Assessment Report (2018) lists over 30 systems used by ministries and agencies. These applications either partly automate administrative processes or they produce insights to improve its public services and make more efficient use of its resources. Examples of this include forecasting future service needs, performing recidivism risk assessments in criminal justice, and identifying cases of tax fraud.
Regarding some applications adopted by government bodies, it is easy to see how they may inform policy decisions on a higher level of decision-making. The German city of Mannheim, for instance, monitors and analyzes educational success of students by tracing the effects of various socio-demographic features and uses insights obtained from these analyses to inform municipal policy decisions. In a similar vein, some Danish municipalities predict localized needs for assistance among the elderly, which may then aid in policy planning.
All in all, the kind of knowledge that ADM systems may produce to support decisions could indeed enhance the analytical capacity of the government and help to avoid policy failure. Through detailed monitoring of the performance in a policy area and registering policy instrument choices, they can reveal what works better in some situations than in others.
Obstacles to using ADM systems for evidence-based policy choices
Although the information-processing capacity of ADM systems implies a strong potential to foster better decision-making about adequate policy instruments, there remain several barriers that ultimately limit the value of ADM systems for realizing policy goals. The mere existence of policy-analytical capacity and evidence does not straightforwardly lead to better decision-making. This is because even where objectives are pre-defined—or consensual—the choice of the means to realize these objectives is not entirely a matter of evidence. Only in rather idealizing, rationalist models of policy-making do information and evidence directly guide policy decisions. Indeed, the process of policy making has been described as one that is rather messy and in which learning processes hardly occur (Cohen et al. Reference Cohen, March and Olsen1972). Political actors may furthermore not have an interest in following available evidence to best solve a policy problem (Kogan Reference Kogan, Coffield and Coffield1999; Sanderson Reference Sanderson2002). Kettl (Reference Kettl2016) and van der Voort et al. (Reference van der Voort, Klievink, Arnaboldi and Meijer2019) note that the use of Big Data analytics and the insights that they may realize will not improve policy making per se because decision support based on ADM systems will also be subject to conflicting motives by political decision makers. More importantly, however, other considerations that loom large in the political realm often supersede the instrumental motive of obtaining and using evidence to best deal with a given policy problem. Knowledge and evidence that may serve to attain a policy goal are likely to be evaluated in terms of whether it is in line with ideological goals and will yield political gains.
For instance, if an ADM system identifies that certain tax provisions will lead to more tax evasion, whereas the use of a different instrument would dampen it, this may go against the ideological view of policy actors who favor those tax provisions. These actors will therefore want to dismiss that kind of evidence, while others might use it for their ideological goals. In the same vein, policy actors may adhere to certain policy paradigms in the form of beliefs about what adequate solutions to given policy problems are (Hall Reference Hall1993). Accordingly, they will interpret and evaluate evidence from a specific perspective and dismiss it to the degree that it goes against such a paradigm.
As van der Voort et al. (Reference van der Voort, Klievink, Arnaboldi and Meijer2019) write, political actors may even try to interfere in data collection, processing and the interpretation of information in order to obtain results that support, or at least do not contradict, their ideological views. The authors describe an instructive case in which political goals overshadowed other concerns: an informational dashboard for the city Milan that synthesizes information about the urban and informational landscape of the city. While the data collection and processing linked to this dashboard could produce evidence for optimizing services provided by the city, political actors, in view of the upcoming Expo 2015, partly interfered with this system to make the city appear in the desired light (van der Voort et al. Reference van der Voort, Klievink, Arnaboldi and Meijer2019, pp. 35–36). Political stakes and goals thus trumped the goal of obtaining accurate evidence.
Altogether, this means that ADM systems can become simply another form of expert knowledge which political actors may leverage in order to support their views. This is possible because, even on the level of policy instrument choice and implementation, ambiguity is reintroduced and is met with a considerable degree of discretion (see also Lodge and Mennicken 2019; Veale and Brass Reference Veale, Brass, Yeung, Lodge, Yeung and Lodge2019). Specifically, there are no definite answers when it comes to important technical properties of the adopted information systems: what exactly should be optimized and how; what constitutes an acceptable performance and by which formalizable measure should it be assessed? This means that even though objectives are clearly defined, it is not clear per se what “good” decision-making means when it must be translated into an ADM system (Veale and Brass Reference Veale, Brass, Yeung, Lodge, Yeung and Lodge2019)
Moreover, even if there is a clear notion of what counts as the proper standard of “good” decision-making, the evidence that ADM systems produce ultimately depends on the data which are used—data which are necessarily shaped by social practices and forms of knowing. If there are sedimented relationships in the data—and the social reality that it represents—these can be learned and reproduced by algorithmic systems and lead to unfair discrimination, as has been shown for ADM used, e.g., for criminal justice risk assessments, applicant recruiting or content filtering (e.g., Eubanks Reference Eubanks2017; Noble Reference Noble2018). In predictive policing, for instance, an ADM system uses police control and arrest data to assess the risk of crime in city areas to guide police patrols accordingly. If there is already a practice of unfair discrimination in policing and arresting certain social groups, this will likely be learned and reproduced by an algorithmic system—unless it is explicitly addressed and corrected for.
From this, it follows that existing structures of knowing shape what kind of evidence and quality of decision-making ADM tools can and will produce. As critical algorithm studies have highlighted, algorithmic systems are never neutral technical entities. Not only are they designed by humans, and either deliberately or unwittingly incorporate values and assumptions of their developers (Mittelstadt et al. Reference Mittelstadt, Allo, Taddeo, Wachter and Floridi2016, p. 7; Noble Reference Noble2018, pp. 1–2), they are also implemented in a societal context in which they may acquire pre-existing biases.
Conclusion
The promise of many ADM applications is that their ability to substitute for and surpass human decision-makers as well as various other properties make them ideal servants in a wide range of areas. It is therefore not surprising that various domains of public administration, such as education, welfare, and criminal justice, have already taken up ADM systems and that they are even considered for use at the higher-level of decision-making in democratic politics. Yet, as we have shown above, there are important fundamental considerations regarding when and how these applications can be compatible with democratic politics, and in what sense they can improve it.
An acute awareness of relevant distinctions in the use of ADM systems for decision-making is particularly important from the point of view of political science because politics is, to an important degree, about defining what counts as good decision-making. Since ADM systems embody a certain understanding of what adequate standards of decision-making are, they can be used to renegotiate the conditions of political decisions-making. And there may well emerge novel conflicts as some actors try to advance the adoption of algorithms in politics against those who want to keep it entirely the preserve of humans.
Indeed, as we have mentioned at the outset, political actors, and institutions have already deployed ADM systems—with potentially far-ranging consequences. First, there are signs that the use of machines in democratic politics could find broad appeal, particularly against the backdrop of dissatisfaction with politicians and political parties. It is on this level where the use of algorithms can arguably be the most harmful to democratic politics. When considering the intricacies of politics and the ambiguity inherent to the criteria of “good” decisions in politics, the idea of using ADM at the heart of democratic politics and to “improve” decision-making is misguided. More than that, it is an outright dangerous aspiration to use machines for solving social problems and dealing with political issues. Ultimately, ADM systems have no place in the process of formulating political decisions because politics is not about optimizing pre-defined objectives nor are political decisions a question merely of evidence and knowledge for solving given problems.
Second, ADM systems offer abilities that may be of use in political decision-making that is, at least in part, also about finding solutions to problems. These technologies may well play a role in countering known human deficiencies and biases. We have argued above that the problem-solving performed by ADM systems is most viable where there are (1) unambiguous and relatively stable objectives together with (2) a high degree of regularity in decision situations. Consequently, there do exist areas and aspects of political decision-making where ADM systems may assist and inform human decision-making to improve decision outcomes based on given standards. This mainly concerns the choice of policy instruments where clearly defined objectives and regularity in decision situation imply a major role for evidence and learning.
Third, even if ADM can contribute to more evidence-based decision-making based on a stronger policy-analytical capability, this does not mean that political actors will simply accept insights and recommendations by these applications as a given. Rather, they may try to instrumentalize them for their purposes by exploiting a space of ambiguity that inevitably remains: with regard to what counts as suitable evidence and concerning the question how to translate policy objectives into parameters of a machine. Ideological and political motives may ultimately diminish the role of evidence.
Algorithms may even reinvigorate conflicts over the role of evidence in decision-making. ADM systems are never neutral, but necessarily embody certain assumptions and values. They may furthermore acquire biases from processing data about society that reflect existing power relations and patterns of discrimination. This means that even seemingly technical design choices are imbued with value questions, which may again invite dispute over the suitability of an application and lead to politicization of the technology (Katzenbach and Ulbricht 2019, pp. 6–7).
All in all, ADM systems are potent tools not so much because of their actual technological capacity to solve analytical tasks but rather because they can be leveraged to alter existing structures of political decision-making. This process has only just begun. It therefore promises to be an interesting field of study to see how the greater adoption of ADM in the public sector changes standards of good decision-making. Various disciplines, among which political science has so far remained relatively silent, are already intensively discussing the societal impacts of ADM systems. This should change, as major political principles and values are at stake and political science has quite a few things to say about the role that ADM can play in the future development—and particularly the government—of societies.
Acknowledgments
We would like to thank the anonymous reviewers for their constructive comments. The manuscript also owes to discussions with Anja Achtziger, Julia Felfeli, Adam Harkens, Tobias Krafft, Johannes Schmees, Wolfgang Schulz, Karen Yeung, and Katharina Zweig
Funding
Open Access funding enabled and organized by Projekt DEAL. The authors disclose receipt of the following financial support for the research, authorship, and/or publication of this article: This research has been conducted within the project “Deciding about, by, and together with algorithmic decision-making systems,” funded by the Volkswagen foundation.
Open access
