One of the great strengths of Fleur Johns’ approach is her conceptual starting point: that sensing and knowing are intimately connected but distinct, and that therefore “sensing practice … encompasses those ways of knowing, or claims to knowledge, that are mobilized in the course of perception.”Footnote ¹ To borrow John Berger’s phrase, sensing practice is a “way of seeing” and whether we are viewing European oil paintings or the human-readable rendering of an algorithmic reading of data from a satellite, it is only possible to fully understand this process by recognizing that “sights” cannot be understood separately from society.Footnote ² Johns’ exploration of the questions of power and agency that are posed by an investigation into the implications of adopting new sensing technologies by the International Atomic Energy Agency (IAEA) and the UN High Commissioner for Refugees (UNHCR) is important and timely because it opens up a wider discussion about the role played by Machine Learning (ML) in a wide range of social contexts, prompting us to ask about the social relations through which the technology itself is produced and used.

This response probes the implications of Johns’ reflections by posing further questions, which might contribute to outlining a broader research agenda concerned with these questions across a range of disciplines. In particular it aims to bring to bear insights from history and sociology to complement an inquiry framed within the context of international law.

An initial set of questions concerns how the adoption of these technologies affects the relationship between expertise and agency within organizations, and particularly whether the adoption of ML exacerbates tendencies towards the centralization of power and reduces the autonomy of agents in “the field.” In her case studies of the IAEA and the UNHCR, Johns shows how the adoption of ML technologies changes the status of these field agents, whether IAEA inspectors or the UNHCR's local officers in refugee camps. In the case of the IAEA, a move towards remote sensing, and in particular the use of satellite imagery, requires the employment of new experts who are able to do their work a comfortable distance from the contested sites subject to the IAEA's inspection regime. As Johns notes, using satellite data to remotely view changes at monitored sites requires many acts of inference and interpretation to render the data comprehensible by wider audiences. There is a shift going on here, she argues, between inspection as performance, complete with moments of high drama, such as the stand-off between IAEA inspectors and Iraqi officials in a Baghdad car park, and inspection as remote data analysis, which changes the “telescopic” sensory regime of the past for a panopticon built on statistical inference.

Thus the IAEA's increasing reliance on satellite data, she argues, will increase the relative weight and importance of previously “unknown” analysts who can interpret this data within the organization and potentially beyond it. However, given that the IAEA's officials still have to persuade others that their assessments of compliance or noncompliance by states are correct, and propose appropriate strategies for action, it seems likely that the data analysts will have to learn skills in diplomacy and working with the media (in other words to come out from “behind the scenes”). Alternatively, they will have to partner with other colleagues who are experienced in this kind of work. While Johns is no doubt right to warn that even when the IAEA's work moves towards legal or policy action “the relationship between a certain action and a particular assemblage of data will typically not be made apparent in any public arena,” could this be a spur towards action either within the IAEA itself, or within wider civil society, aimed at rendering such relationships visible and open to scrutiny and public accountability rather than accepted as a fait accompli?Footnote ³

In case of the UNHCR, the picture which emerges is one of de-skilling and the stripping away of the powers of discretion and agency from aid workers who were previously encouraged not only to view themselves as capable of reasoning and decisionmaking, but urged to treat refugees as conscious and rational human beings rather than simply data. In Johns’ piece the UN aid workers appear to be relieved rather than frustrated by their relegation to simply appendages of the iris-scanning machinery, as the apparent infallibility of the device reduces their responsibility for making unpleasant decisions, such as refusing cakes of soap, wheat flour and tarpaulins to desperate and vulnerable people on the grounds that they have already had “enough.” The description of the encounter between the aid workers and the woman accused of being a “recycler” is particularly chilling, as officials and refugee exchange jokes about how the decision encoded in the iris-scanner will hopefully persuade her husband to give up on the “exhausting exercise” of trying to obtain further assistance. As Johns notes, in the case of women living with violent partners the fact that the “computer said no” is unlikely to be any comfort or protection.

While it is tempting to focus on the ways in which expensive machinery such as iris-scanners contribute towards the centralization of expertise and agency, it is also important to note that there are often counter pressures, particularly for organizations working in remote and poorly-resourced locations. One possible reading of the 1992 and 1994 UNHCR manuals which Johns counterposes to the agency's later practice, is that these are the pragmatic distillation of experience that imperfect communications and limited access to resources within refugee camps means that aid workers will have to rely on their skills in negotiation and persuasion as much as coercion and that this includes the ability to exercise discretion and make their own judgments without constantly having to refer back to a higher authority for guidance. Internet and phone connections may have improved since 1992, but the constraints of time and distance have not been erased, and there will remain many situations where it is aid workers’ capacity to act on their own initiative and using the limited resources available “in the field” that is ultimately more important than their status as representatives of the wider agency. Disruptions to electricity supplies and data connections can temporarily render the iris-scanner nothing more than an expensive lump of metal and plastic, rather than a link to the centers of power. Moreover, as the 1992 and 1994 UNHCR manuals recognized, refugees are not disembodied body parts presenting themselves for identification, but human beings whose collective social agency aid workers ignore at their peril. Displaced people may have been torn away from their homes or countries, but this does not mean they are simply a mass of individuals: as the reference to the challenges of dealing with “leaderships” in the 1992 documentation acknowledges, the ways in which refugees interact with the UNHCR and its claims to authority over them are shaped both by existing social relations and new ones which come into being in refugee camps themselves.

It could be argued that a mirror image of the reduction of refugees to data points by the UNHCR's use of the iris-scanner can be found in IAEA officials’ apparent perception of “citizens” as a vast network of sensors, complementing and enriching the data collected by machines. As Johns notes, combining data from satellite imagery with other geospatial data “drawn from a mostly undifferentiated pool of ‘citizen sensors’” is problematic on a number of levels.Footnote ⁴ While her focus is largely on the opaqueness of the machine learning algorithms which produce the satellite imagery, there are also other difficulties with the “citizen as sensor” metaphor. A crucial one of these is the fact that human beings have rights which may be violated if they are perceived by governments which are attempting to circumvent the Treaty on the Non-Proliferation of Nuclear Weapons (NPT) as “whistleblowers” about such activities.

Nevertheless, Johns notes enthusiasm among IAEA officials for “citizen sensing” methodologies. In her brief account it isn't completely clear what kinds of tasks IAEA officials imagine could be delegated to “citizens” but it seems likely that discussions at the IAEA's 2014 conference were prompted by widespread interest among journalists and human rights organizations in the possibilities of mobilizing volunteers to collect and verify information about human rights violations in contexts which are difficult or dangerous for field researchers to access. Some of the most well-known examples of large scale experimentation in these kinds of practice relate to the civil war in Syria, which was both intensely mediated and presented numerous difficulties for data gathering by human rights organizations and UN bodies.Footnote ⁵

Along with other projects which collect and attempt to verify data about human rights violations, an important method used by Bellingcat and The Syrian Archive involved the process of triangulation across different types and sources of data, such as social media reports and satellite data. Organizations such as Amnesty International which use social media data for similar purposes are likely to also attempt to triangulate further by conducting fieldwork and seeking eye witness testimony (both in person and mediated) to events.

All of these types of data can pose significant challenges in terms of interpretation, which would also be faced by the IAEA. Verifying social media data from contextual information in the content of images or video is time-consuming and thus expensive and human rights organizations and journalists working in this field often find that the relevant metadata is either missing, incomplete, or falsified (sometimes in order to protect the content creator and sometimes as a deliberate act of misinformation).

There are also significant ethical issues involved in relying on data sourced from social media: the degree to which information shared on social media platforms is “public” is frequently unclear to users, and it is hard to judge their expectations of which audiences are viewing the information they publish and what the consequences may be if they are identified as having shared that information with UN institutions, for example. In many ways, citizens sharing information with the IAEA about activities of their governments which may contravene the NPT would be in a similar situation to other “whistleblowers” reporting on human rights violations and could face severe legal sanctions or become the target of human rights abuse themselves, even if they were an unwitting source of the information.

Secondly, Johns’ article highlights another area of similarity between the IAEA and the UNHCR: the hopes invested in technologies as a means to render pressing social and political problems more tractable. Just as the iris-scanner is meant to relieve aid workers of the burden of thinking they are responsible for people whose needs they can never properly meet, so the data-mining methods which make remote sensing possible become a solution for the problems of the IAEA's lack of resources in the face of intensifying nuclear proliferation.

Finally, a discussion of the relationship between the advance of quantification in general, and statistical methods in particular (of which ML is a branch), and the development of the modern state system would be one way to broaden the scope of Johns’ investigation. As Benedict Anderson notes in his work on the role of print technologies in the rise of the capitalist state,Footnote ⁶ the process of making censuses and maps was for the field officers of the European colonial powers a way of visualizing “that system of places … that allots to each his or her proper role and function,” to borrow Ranciere's phrase. It is no accident that across many cultures the words for order in the sense of a mathematical array and order in the sense of a social hierarchy and division of functions are the same. One of Anderson's key insights was that the officers of the colonial state changed the societies they were ruling in the process of fitting them into the rows and columns of a census table: treating people as members of the national or ethnic category which labeled them in the data spurred the development of new identities and nationalist ideologies.Footnote ⁷ This was not a one-way process, however, whereby European states simply imprinted themselves on “native” societies, but the outcome of complex social and political struggles within and between colonizer and colonized, and the internalization of the census-makers’ schemata by their subjects was often disrupted by the emergence of new horizontal solidarities of social class overriding the vertical divisions of language, ethnicity, or religion.

The question of the role of individual states and the international order of states in legitimating and imposing new technologies of sensing, measurement, and analysis is also pertinent to the case studies Johns explores in her article. Both the IAEA and UNHCR are governed by, and answerable to, specific states, whether through the hierarchy of the NPT treaty (which is based on the self-serving claim by the existing nuclear powers that they can be trusted to act as guardians of world peace against “rogue” states), or as the donors which fund the UNHCR's work.