Technology has blurred the line between real and fake imagery

The premise of manipulating visual media is not new. Until relatively recently, though, the ability to do so remained in the hands of those with considerable digital editing expertise. Rapid and substantial technological advances mean that today almost anyone can manipulate digital images. The most recent technological leap involves the use of artificial intelligence, specifically generative adversarial networks (GANs) to synthesise images, like the one used by the prankster student to create Andrew Walz. The accessibility of such technologies – which allow users to manipulate and synthesise images both easily and cheaply – brings opportunities for education, art, and science, but also potential to use fabricated visual imagery for nefarious purposes (see Silbey and Hartzog Reference Silbey and Hartzog2019 for an insightful discussion on neutralising the destructive force of deep fakes).

The successful use of fake visual imagery for harm, of course, somewhat depends on whether people can sort the real from the fake. New studies have begun to answer this question, examining the extent to which observers can detect various types of digital manipulations (e.g., Farid and Bravo Reference Farid and Bravo2010; Robertson et al Reference Robertson, Kramer and Burton2017; Shen et al Reference Shen, Kasra, Pan, Bassett, Malloch and O'Brien2019). In our own labs, we have explored people's ability to sort authentic from manipulated images of real-world scenes (Nightingale et al Reference Nightingale, Wade and Watson2017, Reference Nightingale, Wade, Farid and Watson2019, Reference Nightingale, Wade and Watson2022). People's ability to determine if an image is real or fake depends on the type of manipulation applied to the image (e.g., airbrushing, shadow alterations, object insertion), but in general people perform only slightly above chance on this task (Nightingale et al Reference Nightingale, Wade, Farid and Watson2019). Somewhat counterintuitively, in our research we have found no strong evidence that individual factors, such as having an interest in photography or experience of editing images, are associated with increased accuracy in distinguishing real from manipulated photos. In one study, we tested 15,873 individuals aged 15–75, and found that older adults categorised images less accurately than their middle-aged and younger counterparts, which is not overly surprising given that visual processing declines with age (Nightingale et al Reference Nightingale, Wade and Watson2022). Perhaps more importantly, we found that despite their mediocre ability to sort the genuine photos from the fakes, participants of all ages tended to express confidence in their judgements (Nightingale et al Reference Nightingale, Wade and Watson2022). These results suggest that people are not only poor at detecting manipulated photos, but also often unaware of just how poor they are.

A relatively new type of image manipulation – face morphing – has raised significant concerns about the opportunity afforded by advanced digital editing software for committing identity theft. Face morphs are created by digitally combining images of two or more individuals to create a new image that resembles each of the original identities. The wide availability of face morphing software means that this type of fake image now poses a serious threat to border controls and other face recognition systems used in security settings (see Pikoulis et al Reference Pikoulis, Ioannou, Paschou and Sakkopoulos2021), and worryingly, several studies show that people cannot reliably detect face morphs (Kramer et al Reference Kramer, Mireku, Flack and Ritchie2019; Nightingale et al Reference Nightingale, Agarwal and Farid2021; Robertson et al Reference Robertson, Kramer and Burton2017, Reference Robertson, Mungall, Watson, Wade, Nightingale and Butler2018). In one recent study, participants attempted to determine whether two face images depicted the same person or not (Robertson et al Reference Robertson, Kramer and Burton2017). Over 49 trials, each participant viewed three types of face pairs: (1) two facial images of the same individual, (2) faces belonging to two different individuals, and (3) one individual's face, alongside a 50/50 morph combining that same individual with another person's face. Participants incorrectly endorsed the 50/50 morphs as being a ‘match’, on average, 68 per cent of the time with error rates under 10 per cent for the same individual and different individual face pairs.

In the previous few years, researchers have turned their attention to an even more sophisticated technology for producing manipulated media: deep fakes (Agarwal et al Reference Agarwal, Farid, Gu, He, Nagano and Li2019). The advent of AI-synthesised content has signalled a giant leap forward in the creation of photorealistic fake visual media. Deep fake images are typically synthesised using GANs, a type of machine learning that works by pitting two neural networks (a generator and a discriminator) against one another in an iterative back-and-forth process. This GAN structure can be used to synthesise any type of fake image, as well as video and audio. Highlighting the photo-realism of the outputs of synthesis machines, studies have begun to reveal that people are frequently fooled by deep fakes and cannot reliably distinguish between real and synthetic faces (Hulzebosch et al Reference Hulzebosch, Ibrahimi and Worring2020; Lago et al Reference Lago, Pasquini, Böhme, Dumont, Goffaux and Boato2022; Nightingale and Farid Reference Nightingale and Farid2022) or videos (Groh et al Reference Groh, Epstein, Firestone and Picard2021; Hughes et al Reference Hughes, Ferguson, Hughes, Hughes, Fried, Yao and Hussey2021), yet show overconfidence in their ability to recognise deep fake content (Sütterlin et al Reference Sütterlin, Lugo, Ask, Veng, Eck, Fritschi, Özmen, Bärreiter, Knox, Schmorrow and Fidopiastis2022).

Fake imagery influences thought, intent, and behaviour

The research evidence is clear: Distinguishing between doctored and authentic visual media is difficult, and research dating back to the early 2000s illustrates why we should be concerned. Fake media can have detrimental effects on human memory, cognition, and behaviour. Studies have shown that doctored photos of prominent public events can change not only what people recollect about those events but also people's attitudes and behavioural intentions (Frenda et al Reference Frenda, Knowles, Saletan and Loftus2013; Nash Reference Nash2018). Sacchi et al (Reference Sacchi, Agnoli and Loftus2007), for example, created doctored photos of the 1989 Tiananmen Square protest in Beijing and the 2003 Iraq war protest in Rome. For the Rome event, aggressive-looking demonstrators and police officers wearing riot gear were inserted into an original image of the peaceful demonstration. During a memory test, people who viewed the doctored Rome photo were more likely to state that the protest involved physical confrontation, significant injuries, and damage to property, and indicated they would be less inclined to participate in future protests, than people who viewed the original photo. Doctored images can also distort how people recall self-involving, significant, childhood experiences (e.g., taking a hot air balloon ride, Hessen-Kayfitz and Scoboria Reference Hessen-Kayfitz and Scoboria2012; Wade et al Reference Wade, Garry, Read and Lindsay2002) or recent everyday actions (e.g., shuffling cards or counting to 20, Nash et al Reference Nash, Wade and Lindsay2009; Nash and Wade Reference Nash and Wade2009; Wright et al Reference Wright, Wade and Watson2013). Doctored videos have been used to induce people to falsely confess to committing an undesirable act (e.g., cheating in a gambling task) or to provide erroneous testimony about another person's actions (Nash and Wade Reference Nash and Wade2009, Wade et al Reference Wade, Green and Nash2010 Wright et al Reference Wright, Wade and Watson2013).

More recent research has demonstrated the potential for doctored imagery to change consumers’ memories of what they have purchased and the brands they prefer. Under the guise of conducting consumer research, Hellenthal et al (Reference Hellenthal, Howe and Knott2016) instructed participants to compile a basket of 12 food items made by their preferred brands. The researchers then took a photo of the participant with their ‘personal brand lifestyle basket’. Approximately a week later, participants viewed a doctored version of the photo in which 4 of their 12 selected items were replaced with similar items made by different brands. Participants were asked whether they were comfortable with the photo being included in their brand profile. The next day, participants completed a surprise memory test in which they were asked to indicate which items they included in their original basket. Participants displayed the classic ‘misinformation effect’ – frequently remembering items that were merely suggested to them in the fake photo. Their brand preference ratings also changed after viewing the doctored photo, with many participants indicating a positive shift in attitude and behaviour towards brands that were suggested to them.

It is clear, from over 20 years of empirical research, that doctored images can have powerful effects on cognition and behaviour. Interestingly, psychologists conducting doctored photo research in the early 2000s were of the belief that ‘most of us will never be confronted with images of ourselves doing things we have never done, or in places we have never been’ (Strange et al Reference Strange, Gerrie and Garry2005, 240). Today we are not so sure, given the numerous technologies that can capture, archive, and visualise images of our personal experiences, and given the widespread availability of digital editing software (not to mention the emerging synthesis technologies, but see Murphy and Flynn Reference Murphy and Flynn2021).

People's limited ability to detect fake visual media, paired with its powerful effects, makes it highly effective for nefarious purposes. Furthermore, the prevalence of social media platforms allows content – both real and fake – to be rapidly disseminated across the world. We have already witnessed the harm that synthesised and manipulated images can have, for example, in creating non-consensual sexual imagery, committing financial fraud and identity theft, and fuelling misinformation campaigns (e.g., Kalpokas & Kalpokiene Reference Kalpokas and Kalpokiene2022; Sleigh Reference Sleigh2021; Wakefield Reference Wakefield2022; Westerlund Reference Westerlund2019). We now turn to an important question that has received relatively little attention from researchers, particularly social scientists: What can be done to improve the detection of fake visual content?

Improving detection of fake imagery

Why are people limited in their ability to detect fake images?

An important question that remains largely unanswered is why people have such limited ability to determine when an image is real and when it is fake. One obvious, potential explanation is that fake images are now so sophisticated and realistic that there are no perceptible clues to alert people that the image has been manipulated or synthesised. Yet, in studies involving images that do contain detectable signs of manipulation, including changes that are physically impossible (such as a scene containing cast shadows that are inconsistent with the lighting source), people still frequently fail to notice that those images are manipulated (e.g., Nightingale et al Reference Nightingale, Wade, Farid and Watson2019). Based on these findings, it is reasonable to think there may be value in training people to detect signs of manipulation. However, given the evidence outlined above, which suggests limited benefits of training interventions, more empirical work is needed to advance our understanding of exactly when and why people fail to successfully use such signs.

When thinking about why people fail to notice signs of digital manipulation, a good starting point is to consider the limits of human perception. Decades of cognitive science has shown that people's capacity to perceive the visual world is finite, with seminal studies demonstrating that people can fail to notice even highly conspicuous events unfolding right in front of them (e.g., Neisser Reference Neisser and Pick1979; Neisser and Becklen Reference Neisser and Becklen1975). Perceptual failures have been shown in change blindness and inattentional blindness studies, where people are surprisingly unaware of significant changes to, or the presentation of, stimuli outside of their focus of attention (e.g., Rensink et al Reference Rensink, O'Regan and Clark1997; Simons and Chabris Reference Simons and Chabris1999). One of the most famous examples of inattentional blindness is the ‘invisible gorilla’ study (Simons and Chabris Reference Simons and Chabris1999) in which participants observed a video of a ball game while counting the number of balls passes made between the players in the game. When engaged in this task, approximately half of participants failed to see a person dressed as a gorilla walk through the middle of the ball game. Furthermore, these perceptual failures are affected by the observer's perceptual load; when people are tasked with processing a lot of information, they are less likely to detect changes in scenes (e.g., Carmel et al Reference Carmel, Thorne, Rees and Lavie2011). Thus, it remains possible that attention is another crucial factor that impacts whether or not people notice when an image has been manipulated.

It is also important to think about the challenge of distinguishing between authentic and manipulated media in a digital world where the internet, and particularly social media, offers endless content (more than 3.2 billion images are shared online each day; Thomson et al Reference Thomson, Angus and Dootson2020). Research drawing on cognitive and evolutionary theory, along with behavioural economics, shows us that when people have access to vast amounts of information, the way they search that information shapes the decisions they make (Hills and Hertwig Reference Hills and Hertwig2010). Technological developments afford an ever-increasing ability to store and share information, yet the psychological limits on people's capacity to process information remain unchanged, resulting in a state of information overload (Henkel et al Reference Henkel, Nash, Paton, Lane and Atchley2021; Hilbert and López Reference Hilbert and López2011; Hills Reference Hills2019; van den Bosch et al Reference van den Bosch, Bogers and de Kunder2016). With such overload, people must select what to attend to, what to believe, and what to share. However, not all information is equal: through evolution, humans have developed cognitive heuristics that make certain types of information more attention-worthy, such as negative information and information that is consistent with existing beliefs (Hills Reference Hills2019). As such, it might be that some manipulations are detectable in principle, yet in a world overloaded with information, human perceptual limits lead people to overlook types of evidence that would indicate foul play.

]Alternatively, it might be that people simply do not know what to look for, and rely on unhelpful strategies when trying to verify the authenticity of an image. In a recent study, participants were asked to distinguish between manipulated and genuine photos of real-world events, and to report the strategies they used to determine whether an image had been manipulated or not (Nightingale et al Reference Nightingale, Wade and Watson2022). Overall, people's success was similar regardless of whether or not they reported using a specific strategy, yet there were some interesting differences when looking at the specific types of strategy used. For example, those who reported paying careful attention and systematically ‘zooming in’ to look at different parts of the image were more accurate than those who did not report using this strategy. Although this notion of paying attention might seem obvious, only 2 per cent of participants (263/15,873) mentioned it. This finding suggests that people might be able to improve their detection of manipulated images simply by changing the way they approach the task, echoing Hills and Hertwig's (Reference Hills and Hertwig2010) finding that search strategy can play a crucial role in decision accuracy.

The need for an interdisciplinary theoretical framework

An important next step in improving visual media authentication is to develop a theoretical framework for understanding how various factors – including individual, cognitive, environmental, and cultural – influence people's ability to detect manipulated images. As mentioned above, a small but rapidly growing body of empirical research spanning multiple disciplines speaks to this issue; much of this work could inform theory development.

Within cognitive psychology – our own discipline – one framework in particular could guide theoretical thinking: the source monitoring framework (SMF; Johnson et al Reference Johnson, Hashtroudi and Lindsay1993). Briefly, the SMF aims to explain how people distinguish between mental experiences that result from perception (i.e., memories of real events) versus mental experiences that result from internal processes (i.e., memories of dreams or thoughts). The SMF posits that people can determine the source of their mental experiences by evaluating the characteristics of those experiences. For example, when a memory or image comes to mind, one might consider how familiar, detailed, or coherent it is. If the mental experience has the characteristics typically associated with a memory of genuine experiences (i.e., it is sufficiently familiar, detailed, coherent), then the individual is likely to conclude that it is indeed a memory of something that really happened, rather than something that was merely imagined or thought about. Moreover, according to the SMF, people typically rely on two types of judgement processes to evaluate and classify their mental experiences – a slow systematic reflection and reasoning process, or a rapid, automatic heuristic process (Hasher and Zacks Reference Hasher and Zacks1979; Johnson et al Reference Johnson, Hashtroudi and Lindsay1993). As you might expect, source misattributions (i.e., mistaking an imagined or internally generated event for a genuine memory) are more likely to occur when people rely on a rapid, heuristic, decision process.

We can apply the SMF judgement process to the task of distinguishing between genuine and fake visual imagery: If real and fake images differ in systematic and detectable ways, then people may engage in either a careful, systematic search of an image to detect clues that are indicative of a fake image, or they might rely on a more rapid and automatic judgement process to determine the image's authenticity. From a SMF perspective, we might predict that various extraneous factors could influence a person's ability to accurately evaluate an image and to determine whether it has been manipulated or not. One such factor is a person's political perspective, yet the evidence is mixed. Some research shows that people are more likely to buy into fake news, and mistake fictitious for genuine stories, if the false information aligns with their political beliefs or worldview (Frenda et al Reference Frenda, Knowles, Saletan and Loftus2013; Greene et al Reference Greene, Nash and Murphy2021; Walter and Tukachinsky Reference Walter and Tukachinsky2020; Zhou and Shen Reference Zhou and Shen2022). Other research suggests that susceptibility to fake news is less about how closely information aligns with an individual's political ideology and more about the extent to which an individual engages in analytical thinking (Pennycook and Rand Reference Pennycook and Rand2019). Adding further complexity, in another study, partisan-motivated reasoning affected participants’ susceptibility to believing political-based misinformation, however, more so for authentic video content than deep fake video content (Barari et al Reference Barari, Lucas and Munger2021). According to the SMF, when false information aligns with an individual's own views, beliefs, and stereotypes, it is likely that they will either automatically feel that information to be true, or through motivated reasoning conclude that the information is likely to be true (Mazzoni and Kirsch Reference Mazzoni, Kirsch, Perfect and Schwartz2002). In a similar way, it seems reasonable to expect that a person's personal views, beliefs, and stereotypes might affect their ability (or effort) to detect image manipulations. Indeed, research has already shown that people's expectations and preferences can influence how they perceive visual information (Balcetis and Dunning Reference Balcetis and Dunning2006; Bruner and Potter Reference Bruner and Potter1964). To date, few studies have explored what makes people better or worse at detecting manipulated images, and the majority so far have involved images that depict unfamiliar people partaking in fairly mundane events. The images are not manipulated to serve a particular political goal, or to comment on culture or society, or to evoke an emotional reaction in the observer. Therefore, it remains possible that in real-world scenarios, where visual media are often manipulated to serve a specific goal, observers’ own goals might decide whether they perform better, or worse, when distinguishing authentic from manipulated images.

Another important factor that warrants greater attention from researchers is the context in which the image is viewed, and its apparent source. Research has shown that media platforms vary in terms of their perceived credibility, and the extent to which people trust any particular source might influence their credulity toward images appearing on that platform (Metzger et al Reference Metzger, Flanagin and Medders2010). Computer science and communications experts have started to address this question, and the data from one study suggest that the reported source of an image, and other contextual factors such as how many ‘likes’ it has received, in fact does not significantly affect observers’ perceptions of image credibility (Shen et al Reference Shen, Kasra, Pan, Bassett, Malloch and O'Brien2019). The data did, however, reveal that observers’ attitudes and individual factors, such as their photo-editing experience, affected their perceptions of image credibility.Footnote ¹

Ethical challenges when seeing is not believing

Finally, the issue of sophisticated fake visual media raises a number of ethical challenges. Consider the so-called liar's dividend: perhaps one of the most concerning consequences of how easily people can manipulate and synthesise visual digital content. In a world where practically any image, video, or audio can be manipulated, it is easy to dismiss anything as fake. Soldiers pictured committing human rights violations, a CEO captured in an embarrassing photo, or a politician at a party they had claimed they did not attend: All of these people could, with enough plausibility to satisfy at least their most willing audiences, argue that those images are fraudulent. We have seen this strategy used in recent years, with former US President Donald Trump denying the authenticity of the 2005 recording of him bragging about sexually assaulting women (Fahrenthold Reference Fahrenthold2016). Below we highlight the need to consider how society deals with future technological developments, to help us to secure the benefits of that technology while minimising its possible threats.

One consideration is how to balance the practice of open code and software distribution with the ethical sharing of image manipulation and synthesising technology. Open science initiatives encourage scientists to make their methods, data, and analytical and computer code openly available, which serves to enhance scientific rigour and researcher integrity as well as encourage the collaborative development of technology. The scientific community should, however, more carefully consider when this sharing is ethical and when there might be good reason to keep certain resources out of the public domain. New technologies, including GANs, quickly become widely and freely available on sites like GitHub, often with walkthroughs for implementation. On the one hand, and for the most part, access to such technology is non-problematic and allows for further advances to be made and the potential to develop use for good. For example, through the use of deep fakes in the documentary ‘Welcome to Chechnya’, LGBT individuals were able to testify anonymously about their suffering and persecution in Russia (RD 2020). On the other hand, the open access also extends to malicious actors who wish to deploy the technology for harm – for example, to generate images that can be used to scam a victim or to create videos to support false claims posted on social media. The balance between open and ethical sharing is a complex issue and one that requires interdisciplinary discussion to ensure the development of sensible and useful guidelines.

Another, much broader, consideration is how the research community might develop appropriate guidelines for the ethical development and use of new technologies. The market has exploded with new applications using GANs to create deep fakes – either for free or at a relatively low cost (Cole Reference Cole2018). One application – FakeApp – introduced in 2018, which allows users to create deep fake videos at the press of a button, gathered great interest with hundreds of thousands of downloads in the first month of its release (Marino Reference Marino2018). Although the complexity of training a GAN still prevents many from creating their own models, the development of applications like FakeApp opens up the market to everyone. As such, the potential for misuse is wide; one of the most common abuses so far being the creation of non-consensual sexual imagery. In 2019, research conducted by a cybersecurity company, Deeptrace, revealed that 96 per cent of the deep fake videos online at that time were of a pornographic nature, and the victims overwhelmingly women (Ajder et al Reference Ajder, Patrini, Cavalli and Cullen2019; Wang Reference Wang2019). The ethical and moral concerns surrounding these new technologies are highlighted in the steadily growing number of publications on this topic from fields such as law, information technology, and political science (e.g., de Ruiter Reference de Ruiter2021). We believe that there is much that researchers from a range of disciplines can contribute to this discussion.

A final consideration is for the giants of the technology sector to understand how their platforms are used for sharing and weaponising content, and to put substantial effort into preventing such misuses. Business media experts have posited that social media companies are doing a substandard job of keeping harmful content, such as COVID-19 vaccine misinformation, off their platforms (O'Sullivan et al Reference O'Sullivan, Duffy, Subramaniam and Boxer2021). In a recent study examining 30 anti-vaccine Facebook groups, researchers discovered that just 12 individuals accounted for sharing 70 per cent of anti-vax disinformation within these groups (Center for Countering Digital Hate 2021). Of course, this study considered only a subset of Facebook groups, but it does pose an interesting question: if researchers can find those responsible for posting this vaccine disinformation, why can't Facebook? The better question is perhaps why won't they, as opposed to why can't they. Meta (previously Facebook) reported that 97 per cent of its total revenue from October to December 2021 came from advertising (Johnston and Cheng Reference Johnston and Cheng2022). The business model underpinning such success involves gleaning as much data as possible from site users, to build detailed profiles ripe for ad targeting. One way to keep users returning to social media sites is to show controversial and evocative content that captivates interest (Kim Reference Kim2015) – fake content can achieve this goal extremely effectively, given that it is free from factual constraints (Lewandowsky and Pomerantsev Reference Lewandowsky and Pomerantsev2022). Deep fakes might be particularly powerful when it comes to captivating users’ attention, especially given humans’ ability to quickly recognise and understand visual content (e.g., Greene and Oliva Reference Greene and Oliva2009; Isola et al Reference Isola, Xiao, Parikh, Torralba and Oliva2013). Therefore, legislators should consider reasonable policy and regulation for ensuring that social media companies are accountable for real-world harms that might result from their services. Modest regulatory changes should incentivise companies to introduce safeguards, and as a result, help toward restoring trust in our digital world.

Ultimately, the potential consequences of fake imagery mean that it is worthwhile examining new ways of improving people's ability to sort the fake from the genuine. These attempts stand to be useful, even if they were only to equip people to weed out the poorer attempts at manipulation. With the pace at which technology is improving, it is perhaps overly optimistic to think that people could learn to reliably detect the most sophisticated fakes that are now readily disseminated across the internet. Instead, within the research community, we should continue to raise awareness of the current and emerging threats, with the aim to encourage more research in this area, including the development of improved computational methods of detection – or face the possibility that people will be fooled by scams far worse than that of the made-up congressional candidate, Andrew Walz.