2.1. The challenge of socially responsible AI in the context of government—Citizen relationships

The special relationship between the public—the ruled—and public governance—their rulers—makes the use of AI technologies in the public sector particularly sensitive. Public service organizations, as the name suggests, are under a duty to serve the public. They obtain their legitimization by the consent of the public, which implies their interests should be driven by the benefit of society and the collective good—the general will (Rousseau, Reference Rousseau1964).

Citizen concerns and controversies have halted several public service AI initiatives after their launch (Misuraca and Van Noordt, Reference Misuraca and Van Noordt2020; van Veenstra et al., Reference van Veenstra, Grommé and Djafari2021). There are multiple examples of cases where the use of AI in the context of public services has received criticism. The Austrian labor administration created a system to categorize job seekers by their likelihood of finding a job which spurred concerns about bias and discrimination (Wimmer, Reference Wimmer2018; Allhutter et al., Reference Allhutter, Cech, Fischer, Grill and Mager2020; Lopez, Reference Lopez2021). The Dutch government developed an AI system that links multiple data sources and provides indications of possible fraud by welfare beneficiaries. In response to that, civil society organizations and individuals convened and filed a freedom of information request asking questions about its workings and use (Wieringa, Reference Wieringa2023). The Dutch courts found that the system violates citizens’ rights by being untransparent and challenging privacy (Bekker, Reference Bekker, Spijkers, Werner and Wessel2021). In their examination of possible adverse consequences of algorithmic decision-making (ADM) within the public sector, Rinta-Kahila et al. (Reference Rinta-Kahila, Someh, Gillespie, Indulska and Gregor2022) investigated an ADM system implemented by the Australian government to automatically calculate and collect welfare overpayment debts from citizens. They found that the system inadvertently led to significant distress among both citizens and employees of the public service organization. The destructive effects ultimately harmed the government too, both financially and in terms of its reputation and citizen trust. Overall, key concerns expressed about public sector AI relate to training on inappropriate datasets that may perpetuate or even amplify biases (Lopez, Reference Lopez2021), harming service impartiality (Rothstein, Reference Rothstein2013), and blackboxing AI’s inner workings (Asatiani et al., Reference Asatiani, Malo, Nagbøl, Penttinen, Rinta-Kahila and Salovaara2021). This urgently raises the issue on how we can ensure socially responsible and legitimate use of AI in public services by taking into account the special relationship between the citizens and their government.

In the exploration of theoretical frameworks for investigating socially responsible AI in public services, multiple alternative theoretical lenses were considered. Specifically, three promising theories were examined, including stakeholder theory, institutional theory, and structuration theory. Stakeholder theory can be viewed as inherently managerial; a framework for managerial and organizational behavior (Freeman, Reference Freeman1984). It has been applied in the field of public service research but has received varying appraisals. Although Flak and Rose (Reference Flak and Rose2005) found no conceptual mismatch between stakeholder theory and the government’s objective of providing policy and services for citizens and organizations, Donaldson and Preston (Reference Donaldson and Preston1995) describe the theory as merely one of the private sector, governed by fundamentally different principles and implications than any public sector organization. Its key limitation, compared to social contract theory, is its limited capacity to comprehend the specific roles and reciprocal obligations in a governmental context beyond executive responsibilities. We also examined institutional theory and structuration theory that explore the creation and enactment of structures, their relationship with context, actions, and actors (Scott, Reference Scott2004; Giddens, Reference Giddens2014). Institutional theory suggests organizations strive for legitimacy to ensure long-term survival, which is a concept aligned with social contract theory (Meyer and Rowan, Reference Meyer and Rowan1977). Meanwhile, structuration theory recognizes how actors operate within the constraints of social structures, offering insights into power dynamics, legitimacy, and organizational responsibilities (Cohen, Reference Cohen1989). However, as this study is not focused on structure dynamics but rather on socially responsible AI in the context of citizen—government relationships, social contract theory was deemed to be the more suitable theoretical framework.

2.2. Need to reposition AI in public services: A social contract theory perspective

Public service organizations execute governmental rights and obligations. They are required to provide the same services to all people regardless of social, ethnic, or religious background (Aucoin, Reference Aucoin2012). According to Jørgensen and Bozeman (Reference Jørgensen and Bozeman2007), equal treatment of citizens, neutrality, and impartiality are considered critical public values. This implies that public organizations cannot choose whether they want to offer a service or not—unlike private companies. But just like a public service organization cannot pick and choose whom they want to engage with, citizens are also bound to the public service organizations as sole providers for specific services (Junginger, Reference Junginger2016). These power relationships between governmental institutions and the people are part of the “social contract.”

In this research, we take a Social Contract Theory perspective for socially responsible AI in public services to explore the relationships and power dynamics between the involved actors. The term Social Contract Theory encompasses a large body of theories and should be understood as a collection of conceptual streams rather than a single specific theory. In general, a social contract describes a hypothetical agreement in which the members of a society endorse and comply with the fundamental rules, institutions, and principles of that society. It is an agreement made between citizens and the state, through which they accept the authority of the state in return for benefits that only a sovereign power can provide, delineating a structure of power relationships between governments, institutions, and the people (Heywood, Reference Heywood2015). Power can be understood as the capacity to make formal decisions that are binding on others (Heywood, Reference Heywood2015). According to D’agostino (Reference D’agostino1996), a social contract is concerned with determining whether a ruling power is legitimate and therefore worthy of loyalty. In consequence, this engenders a responsibility for the rulers to act in the interest of the ruled but also ascribes certain rights and obligations to the latter (Figure 1).

Figure 1. Relationships within the social contract.

The social contract formalizes the effort to create consensus on shared values and ideals and ensure ethical practice (Jos, Reference Jos2006). The ultimate aim of a social contract approach is to demonstrate that social, moral, or political rules can be rationally justified. According to Rawls (Reference Rawls2001), the social contract is a model of rational justification that transforms the problem of justification into a problem of deliberation. It consists of general parameters and dynamics, set to represent reasons for endorsing and complying with social rules, principles, and norms. Yet justification does not rely on some exogenous reason or truth but is rather generated endogenously by a form of agreement. These specific dynamics between citizens and public service organizations render social contract theory as a suitable lens when studying the adoption of AI by the public service.

According to the idea of a social contract by Rousseau (Reference Rousseau1964), legitimacy to governments and institutions is assigned from the people they govern—the sovereign. Rousseau describes the sovereign as the collective grouping of people who by their consent enter into a civil society. David Gauthier (Reference Gauthier1986) argues that any system of moral constraints must be justified to those to whom it is meant to apply. Applying this understanding of mutual duties and obligations requires a reevaluation of privacy concerns, data processing and transparency considerations, which are common topics in the discourse about responsible AI implementation.

Citizens expect that their data will be used by the authorized organization for its intended purposes and to their benefit, which can be seen as an implied social contract (Perreault, Reference Perreault2015). This points toward an implicit expectation from the individual to be able to exercise their right as the sovereign to question governmental decisions. Furthermore, this also examines how a social contract, which is not a formalized contract but rather a conceptual one, can be broken. For example, Xu et al. (Reference Xu, Teo and Tan2006) found that a social contract in the case of technology use in the public sector is considered breached if citizens are unaware of unauthorized data collection or data processing, or data transfer to other parties without their explicit consent, which illustrates a form of power abuse. This level of examination can be understood as the macro-lens of social contract theory.

At the next level, which we label as the meso-lens, one needs to consider the interaction between the individual and society. This relationship has been touched by almost all political debates (Heywood, Reference Heywood2015). Applied within the research of socially responsible AI, it can help to better investigate the balancing act between individual rights and benefits, and the common good. If we stick with the example of personal data, which reasons do individuals have to share personal data, which provides benefit for everyone within a society? Why are they prioritizing either societal cooperation or personal gain? These reasons individuals have for agreeing to some rules or principles need to be understood as their own reasons, not necessarily as “good reasons” from the impartiality perspective. Individuals may care about what they perceive to be impartially good or some other non-individualistic notion, but what they care about will differ from one another. Rawls (Reference Rawls1996) highlights that a society cannot reasonably be expected to have similar conceptions of the good, and heterogeneity needs to be considered. And the same is true for the perspective of the government organization. Does the organization respect individual freedom, or does it exercise its power for the benefit of the collective? If a social contract is to endorse interrelated normative desiderata (e.g., liberty, equality, and welfare, which are all guiding principles for a working society), a deliberative process that draws on a diversity of perspectives will outperform one based on a strict normalization of perspectives. A human-centered approach that acknowledges and incorporates the tensions between the collective good and the individual good forms a suitable conceptual framework.

The final tier of examination within the framework of social contract theory is found in the micro-lens. This level explores the intricate processes and mechanisms that underpin the attainment of consensus among the various involved stakeholders. It places particular emphasis on the dynamics and interactions between citizens and public organizations, scrutinizing the immediate communication and exchanges among these different actors to construct an agreement. Among the most widely discussed mechanisms are consent (Rousseau, Reference Rousseau1964; Hobbes, Reference Hobbes2016; Locke, Reference Locke2016), bargaining (Nash, Reference Nash1950; Harsanyi, Reference Harsanyi1976), aggregation (Harsanyi, Reference Harsanyi1976), and equilibrium (Buchanan, Reference Buchanan1975; Gauthier, Reference Gauthier1986). These mechanisms delineate various forms of normative authority for self-binding, predicated on the notion that individuals possess fundamental normative powers over themselves. If parties can indeed bind themselves by wielding this normative power, then the outcome of the social contract results in an obligation (as asserted by Hobbes, Reference Hobbes2016; Hume, Reference Hume1963). It is worth noting that some of these agreement mechanisms have faced scrutiny from contemporary social contract theorists, with consent, in particular, standing out as a key example (Weale, Reference Weale2020).

3.1. Prototype

This study is part of a larger research project on the responsible application of AI in the public sector, which follows an action design research (ADR) approach (Sein et al., Reference Sein, Henfridsson, Purao, Rossi and Lindgren2011). Taking an ADR approach entails close collaboration with practice. In a series of iterations, we developed together with the public organization an interactive prototype consisting of a user interface, mimicking a public service agency portal (Table 1). The prototype depicts a predefined interaction sequence starting from a notification about the optional use of an AI-based prediction, different types, and levels of information about the prediction, and consent options.

Table 1. Prototype overview

The development of the prototype was driven by the social contract lenses framework, which informed the design decisions on multiple levels. On the macro-level, the prototype needed to take the existing power structures into account and reflect on the rights and duties of each actor in this process. The existing rights and duties of both parties—the governmental organization as well as the citizens—needed to be presented in a sufficient manner. One element in this specific case was the transparency and acknowledgment of which personal data about the citizen would be accessible to the organization per the nature of the organizational power, in case of an agreement to use. Simultaneously, it also needed to be clear to the citizen that the right to object to the use of this personal data exists without any negative consequences or need for justification. On the meso-level, the participation and contribution of an individual to the collective good by agreeing to the use of personal data should be outlined. In particular, this needs to address deliberation of social morality, rather than the moral obligation of the citizen but also requires trust in the honesty and integrity in the public service organization. Doran et al. (Reference Doran, Schulz and Besold2017) assert that to achieve trustworthiness and an evaluation of the ethical and moral standards inscribed on a machine, explanations should provide insight into the rationale of the AI system and enable users to draw conclusions based on them. In this specific use case, the purpose, and anticipated benefits of such a system for the general society, as well as in particular to the individual need to be presented. For the individual citizen, the benefit would result in the potential avoidance of unnecessary appointments. For the organization, the use of personal data, and in consequence, the use of the algorithmic decision-support system would result in improved processes, and by that, more efficient resource utilization. Finally, on the microlevel, the prototype had to support the immediate user–system interaction to consent or dissent to the request. This means that the interaction and communication sequence needed to enable the users (the citizens) to understand their role in the process, grasp the governmental request and the consequences of either agreement or disagreement, and act accordingly.

3.2. Recruitment

For this study, we recruited 20 participants aged between 18 and 65 years, reflecting the distribution of the general population on sick leave based on the official Norwegian statistics office (Statisisk Sentralbyrå, Reference Sentralbyrå2022) (Table 2). The number of participants was defined considering the relevance and diversity in participants’ backgrounds and experiences as well as the qualitative research design, which entails engagement with participants to elicit rich qualitative insights (Creswell and Poth, Reference Creswell and Poth2016). Another criterion for determining the number of participants is theoretical saturation. Theoretical saturation occurs in data collection when new information ceases to emerge and themes become repetitive (Strauss and Corbin, Reference Strauss and Corbin1998). The key thing is reaching the point of data saturation, where new participants are not providing substantially new insights. Data collection continued even after initial similarities in responses and to balance depth of insight with practical considerations, we stopped reaching out to new participants after we completed 20 interview sessions with prototype walkthroughs. This number has been proven to be sufficient to obtain meaningful results through user studies (Faulkner, Reference Faulkner2003). As themes became repetitive, 20 participants provided sufficient diversity and depth, and this was deemed appropriate since the aim of the study is to gain a deeper understanding of participants’ perspectives within a specific use case and context (Marshall et al., Reference Marshall, Cardon, Poddar and Fontenot2013). The recruitment criteria included gender and education. For the gender criteria, we were only able to differentiate between two genders (female and male) as these are the only defined genders in the official statistics. Moreover, we also aimed to match the statistical distribution of the educational level, including participants with high school education (Videregående), vocational school level (Fagskolenivå), and university education (Universitets- og høgskolenivå). The sessions were conducted both online via a video conferencing and screen-sharing application as well as in person, depending on the availability of the participants. Before the study commenced, participants received information regarding the aims of the particular study and the overarching goals of the research project, and were also briefed on the voluntary nature of their participation. Additionally, they were provided with an overview of the study’s procedures and were reminded of their right to withdraw from the study at any point.

Table 2. Overview of participants

This research study was examined and approved by the Norwegian Center for Research Data (Norsk senter for forskningsdata—NSD) under the reference number: #931033. This approval process involved a comprehensive review of the research protocol, the recruitment process, informed consent procedures, and data handling to confirm that the study adheres to ethical guidelines and legal regulations. The approval confirms that the study has undergone a thorough ethical review and met the necessary standards and requirements, ensuring the protection and well-being of our participants throughout the research process.

3.3. Data collection

The data collection included three consecutive stages (Table 3). In the initial stage, we collected general data about the participants: age and gender, current occupation, and highest educational level. Further, we asked the participants to provide a self-assessment for two dimensions on a scale from 1 to 5 (low–high). The first dimension was defined as “prior knowledge about AI” with an average self-reported rating of 2.20. Overall, the participants had some rudimentary knowledge about AI, but lacked a deeper technical understanding. The second dimension we asked about is: “frequency of technology use.” Participants provided an average self-reported rating of 4.65, demonstrating exposure and general familiarity with the use of technologies such as computers or smartphones. Next, we collected data on the level of trust toward the Norwegian government and governmental organizations and on the use of AI within public services. These questions addressed the macro as well as the meso level of the social contract framework. The data collected through these questions provide indications about the current state of the government–citizen relationship and about participants’ understanding of the current balance between individual and collective benefits from public services.

Table 3. Data collection stages

In the second stage, a moderated user study in the form of a task-based interaction with the prototype was conducted. Participants were presented with a short scenario and task that led them to a decision whether they would consent to the use of a new AI-supported prediction system in relation to the public service use case. While performing the task, the participants were encouraged to share their thoughts and feelings with the study moderator. This “think-aloud protocol” method describes the concurrent verbalization of thoughts while performing a task (Ericsson and Simon, Reference Ericsson and Simon1984). This helped us to better follow the participants’ line of thinking and achieve a clearer understanding of their reasoning, thoughts, and concerns

In the third and final stage, we followed up with questions about the experience with the interactive prototype. After completing their interaction with the prototype, the participants were again asked to rate their level of agreement with a set of predefined statements in relation to the scenario, from 1 (strongly disagree) to 5 (strongly agree). The predefined statements were related to the perceived competence and efficiency of the AI-enabled system, the anticipation of negative consequences of its use in public services, and the understandability of the system. Further, we also asked the participants about their levels of comfort and trust toward the specific system they tried. Additionally, we asked open questions about the different explanatory elements used in the prototype.

3.4. Data analysis

To ensure a comprehensive analysis, we categorized the collected data into three classes: scale ratings, spoken word feedback, and interactions with the prototype (Table 4). Each class of data was first analyzed individually followed by a synthetization with the other two classes. The goal of the analysis was to identify themes that would help us to better understand the stances and concerns of participants. Our comprehensive data analysis was performed through the social contract lenses framework. This encompassed an evaluation of power dynamics, a nuanced assessment of collective versus individual interests, and an exploration of how participants perceived and experienced the various agreement mechanisms. First, we evaluated the answers to the predefined questions given by the participants. We began by aggregating the scale ratings in a spreadsheet, which allowed us to create a comprehensive overview of the data’s diversity and trends. This also allied us to facilitate the identification of distinct groups, enabled us to gauge participants’ perceptions and opinions, and helped us in quantifying and monitoring changes occurring throughout the study.

Table 4. Data analyses classes

In a next step, we conducted an in-depth analysis of the qualitative data obtained from both the think-aloud protocol, consisting of spoken responses, and the feedback participants provided in response to open-ended questions regarding various types of explanations. To carry out this analysis, we reviewed the study recordings and thoroughly examined interview transcripts multiple times. The objective was to identify common patterns, perceptions, and recurring themes within the data collected from participants. Throughout this analytical process, we selectively highlighted noteworthy incidents, quotes, and specific expressions shared by the participants and engaged in thorough discussions among the research group members to collectively interpret these findings. This category of data yielded a wealth of valuable information concerning the concerns and understandings of the study’s participants.

In a final step, our investigation extended to the examination of participant interactions with the prototype, based on the recordings as well as notes taken during the research sessions. We focused on determining whether specific information elements were accessed, how participants engaged with this information, and the duration of time spent processing it. By merging this analysis with the findings from the previous stages, we aimed to fortify our analytical examination and uncover emerging concepts and recurrent themes through cross-referencing the observations and findings.

4.1. Macro-lens—Government—Citizen relationship

The macro-lens turns particular attention to the aspects of power dynamics. The power granted to governments by the legitimation of the public enables rulers to fulfill governmental duties. Believing that a government and its institutions will act in the interest of the citizens is an integral part of a social contract and can be seen as a fundamental requirement for a society to work.

A major finding is that among the 20 participants, only one did not provide consent for the use of the AI tool. Interestingly, most participants expressed a rather positive stance right from the start while the prototype interaction mostly enhanced the positive attitudes toward the use of AI in the public sector. Specifically, when the participants were asked about their agreement with the statement “I think I would be comfortable with the use of AI for public services,” 10/20 gave a rating of 4 or 5, and 8/20 gave a rating of 3. Only 2/20 gave a rating lower than 3. According to the given ratings, we assigned the participants to three categories: skeptic (below 3), neutral (3), and comfortable (above 3). After the interaction with the prototype, we followed up on this rating with the statement: “I think I would be comfortable with the use of such a tool within public services,” aiming to assess whether the interaction with the prototype had any effect. We found that 40% changed by increasing their comfort, approximately 40% of the participants did not change and 20% did change by lowering their rating. Specifically, we found that after having interacted with the prototype, half of the participants that started neutral converted to being comfortable, one of the two “skeptics” became “neutral,” and two of the “comfortables” became more comfortable. Finally, among the participants who lowered their rating one “neutral” became “skeptic” while the others remained in the “comfortable” category but reduced their expressed level of comfort. According to the framework, these findings relate to an unscathed perception of power relationships between citizens and the governmental organization, from the citizens’ perspective.

Exploring the generally positive attitude toward AI in public services further, we found that the participants linked this stance to their overall trust in the government. Several participants provided revealing articulations about their trust in the government and how this trust affects their choice to consent to the use of the AI system: “I don’t have time to read all this, but I trust the government”; “If that would be like a private company, e.g. if it’s [name of a telecom company], I trust them somewhat, but these companies sell the information given to other companies for marketing and whatever.” These statements indicate a deep belief in the integrity of the Norwegian government and its executing organizations. However, some of the participants also expressed reflective comments on how self-aware they are about their level of trust in the Government: “[…] I’m a typical Norwegian, super naïve to the government. I really think they are trying to do the best, even if they don’t”; “Maybe I’m naive, but I trust the government.” and “I trust the government and I think they are trying to do the best for the people in Norway.” This sentiment shows a clear self-awareness about the high level of trust in the government that manifests a feeling of comfort but also potentially a concern about ensuring that their trust is not betrayed, that is, the social contract breached.

When applying the macro-lens, the existing trust in a government reveals a well-established working relationship between citizens and governmental organizations. The fact that some participants even agree to the use of personal data and AI, without reading through the full information indicates the credibility of public organizations.

4.2. Meso-lens—Interplay between individual and collective good

Another theme that was found in our study concerns the interplay between individual and collective needs and the respective deliberation from citizens. As part of the social contract, citizens are aware of their role as part of a society, but also defend their own individual interests. The meso-lens allows investigation into the different understandings of the individual and the collective good.

We observed participants being aware of the objective of governmental organizations acting in the interest of “the people,” with one participant expressing this understanding as “I really think they are trying to do the best, even if they don’t do the best for everyone, all the time.” Another participant reflects on the responsibility to not perpetuate biases, misuse personal data, or discriminate against marginalized groups: “Whenever you work with AI, there’s just all this data that needs to be collected and dealt with and I fear that it’s easy to cut corners. And I think you’ve seen this with the big tech companies—they make something cool and then it turns out that it’s racist […]. Which then makes me concerned that the same problems are not quite solved yet.” Furthermore, participants were also reflecting on the efficient use of public resources, which can be seen as a benefit for society but also, by extension, for the individual citizens in regard to sensible spending on their tax money. One participant mentioned a personal example: “For my own sick leave, I was so annoyed about all these dialogue meetings, and I thought it’s so expensive for the government and it’s so unnecessary. How can they think that I should go back to work when they are paying for my “Stråling” [radiotherapy]? It was so stupid and inefficient, so if this [the AI tool] could help to just not spread out all the money it would have been better.”

However, participants were also expressing reflective thoughts from their individual perspectives. For instance, when asked about potential negative consequences of the use of the tool, one participant said: “No, […] as long as I have the opportunity to ask for a meeting myself if I want to have one,” highlighting that the use of such a decision-support system must not restrict any legal individual rights. Another perspective was formulated by a participant, who raised the concern that by pooling the data of the people into groups, there is a risk of losing the intricacies of the individual case, resulting in oversimplification instead of addressing the distinct need of a particular human being: “It’s easy to put people in these boxes on the “Information effects” [referring to the section in the prototype]. So it was actually the scale on the different age groups, that made me think that this would have a negative effect.”

These instances reveal that on the meso-level of the social contract theory framework, citizens continuously shift between considerations about the collective good and the individual need, depending on the context. A clear articulation of the AI value for both society as a whole and individuals is important.

4.3. Micro-lens—Humans in the loop and transparency

Through the micro-lens, the actual processes and mechanisms to reach an agreement among the involved actors is analyzed in detail. This lens shifts attention to the interplay and exchanges between citizens and the public organization, examining the immediate communication and interaction between the different actors to model an agreement. It allowed us to investigate if and how citizens are informed about the intended use of algorithmic models and their personal data and the mechanisms to consent, dissent, or contest.

In our investigation, the micro-level of the social contract theory framework, relates specifically the role of human actors during the process and the perceived transparency. The participants expressed that they feel reassured by having humans involved in decisions alongside AI systems. Within the public service scenario of the study, the leave duration prediction is used as an additional information source for the responsible case worker. It is not an automated decision-making system. The general concept of having humans involved during the process has been brought up by participants in different variations. One of the participants said: “I will be comfortable at least if it’s not only the tool that’s part of the decision but with people.” Another participant stated: “It would be much easier for me if the element of the case handler was more visible.” One participant explained that a fully automated system would be a concern: “[…]decisions are made with too little discretion because they fully trust the system.” The common sentiment from these statements can be understood as an implicit expectation that a machine should not be left alone to make a decision, but a human actor would still retain the ultimate power of decision. However, two participants did express their opinions to get humans out of the loop to ensure impartiality. One of them said: “I think some people are very pushy and begging, and maybe they get more. And some people don’t ask for much, so they can have real problems and in a way to me it’s fairer if it’s based on this [AI system], and not if I’m yelling or crying.” This hints toward a perception that having a human within the process may also be a potential weakness that could lead to less fair or equal treatment of citizens.

Some participants related their reasoning for having an HITL to their need to ensure contestability. Participants expressed the concern that if a decision were made without a human involved in the process, it would be difficult for them to find a person to raise such a dispute. This was expressed explicitly by one participant: “Where should I complain to?” Another participant explained: “AI’s rating may be incorrect. With a person, you can explain what is wrong, but you cannot AI.” This involvement of a human point of contact and decision maker relates to the microlevel of the social contract lenses framework, as it provides insights into potential agreement modeling mechanisms. It highlights that there exists an expectation to have a human actor available within these processes, rather than a pure system–citizen relationship.

Further, our analysis also revealed findings relating to the transparency provided by the prototype. Although the overall impression of the explanations provided was positive, and the prototype interaction enhanced the positive stance toward AI for most participants, some of them also provided comments that indicate that AI remained relatively opaque for them. Multiple participants commented on difficulties understanding the text. Some of the comments were attributed to the use of concepts that require some basic understanding of statistics: “Maybe like this sentence in here, it’s probably a bit difficult to understand for all people that are not working with statistics.” Also, participants remarked that some of the wording was difficult to understand due to its legislative content: “I don’t think normal people would understand this; it’s like a lawbook” and “Heavily written, very classic bureaucratic language. “Duration estimate”—I do not think many people understand that much of.” But also specific words were perceived as hard to understand: “I don’t understand the word information effect.”

Similarly, the process chart received mixed feedback. For some participants, the chart helped to understand how the system would be used within the overall context and which role it would play in relation to the general process: “I really love this”; “I love to see how the process is with and without the tool.” But others found the process chart difficult to understand: “Hard to figure out the chart” or “I don’t understand anything about this.” Similarly, although some participants found the data table useful: "Yes, I think it is useful that it says what the purpose of collecting information is, what kind of information is collected,” other participants found the data table excessive, mentioning: “I would just close it right away ‘cause I would think I don’t have the time to read this or understand it so I wouldn’t read it” or “Too much to read.” Interestingly, some of the participants expressed surprise when they saw the data table, as they did not expect that the government would have all this data. Further, some participants also wondered who else might have access to the data: “Do they have this information?,” “Will my employer see this information?,” and “I feel an employer can use it against me and in further hiring processes. As an employee you already have a weaker position.” These findings may indicate that the existence and availability of information and explanations contribute toward a positive stance about AI, although citizens do not fully understand the information provided.