Utility and media choice

A central parameter in the study of political media is the process by which the public selects media to consume. The degree to which media exposure happens incidentally or as the result of an explicit choice varies with both the individual and the larger sociotechnical context, and while incidental and explicit exposure are both large components of the media diet, the latter is particularly relevant in the “hyperchoice” context of contemporary online media (Arceneaux and Johnson Reference Arceneaux and Martin2013; Messing and Westwood 2012).

Faced with either a blank search bar or a “feed” of articles on a social media platform (e.g., Facebook) or general interest web portal (e.g., MSN News), the citizen must take some action. She can “scroll” through the feed – a prominent form of online behavior akin to browsing newspaper headlines but far more extensive in terms of content (Settle Reference Settle2018) – but the central act of media consumption only takes place after she makes a selection or “clicks.” Indeed, fewer and fewer people navigate directly to media companies’ homepages, and even then they have to select an article to read or video to watch (Guess Reference Guess, Brendan and Jason2018). This shift in the location of media choice has changed the economics of the media industry to what Munger (2019) describes as “Clickbait Media.”

The increasing centrality of consumer choice has not been met with sufficient attention by scholars of media effects. While the wider political communication and journalism literature has addressed this topic (Hamilton Reference Hamilton2004; Webster Reference Webster2014), and there are a number of excellent books on the topic in the context of cable news (Arceneaux and Johnson Reference Arceneaux and Martin2013; Prior Reference Prior2007; Stroud Reference Stroud2011), the political behavior literature has not fully adapted to the centrality of “clicking” in online media. ^{Footnote 1} . Still, Sood and Lelkes (Reference Sood and Yphtach2018) identify a key theoretical limitation. The robust evidence of the preference for information from congenial sources conflates two possible causes: the preference for congenial information and the belief in the greater accuracy of congenial sources.

Here, we present results from an experiment that transcends this limitation. We collect a sample of Italian Facebook users through a paid advertisement. The sample is thus not representative of the general population but should be quite representative of the actual population of interest: people who click on links on Facebook. Italy is a useful case study because it has in many ways been ahead of the rest of the West in terms of disillusionment with the media establishment, and the ensuing rise to power of anti-establishment politicians who embrace the credibility-through-virality that clickbait entails. ^{Footnote 2}

We first present subjects with a series of non-experimental choice tasks to estimate their individual “preference for clickbait,” and the distribution of this propensity across the sample population. This portion of the design corroborates – in a novel national context – the findings in Munger et al. (Reference Munger, Mario, Jonathan and Joshua2020) that the elderly, the less educated, and the less digitally literate have a higher “preference for clickbait”; we also find this to be higher among women, frequent internet users, and supporters of the ruling coalition.

For the experiment, we present subjects with a similar choice task, but also inform them that they will read the article they rank highest. Subjects are then randomly assigned into one of two treatment conditions: being told there will be a quiz on the topic discussed in the article or being told there will be a quiz and that providing answers that are “closer to the truth” will increase their chance of winning a lottery for a financial prize; the control group is simply told they will read the article. ^{Footnote 3}

We find no evidence of direct effects of the addition of the financial incentive, but this masks significant and offsetting heterogeneous treatment effects. The treatment increased the preference for clickbait among the elderly, the less educated, and less digitally literate, while at the same time it decreased the preference for clickbait among those on the other ends of these respective distributions. The heterogeneity in “preference for clickbait” thus increases in the financial incentive for accuracy. This finding contrasts with the results in Prior and Lupia (Reference Prior and Lupia2008) and Prior et al. (Reference Prior, Sood and Khanna2015), which find that increasing financial incentives decreases heterogeneity in civic knowledge and perceptions of the economy. Even more importantly, the result provides strong evidence for the hypothesis that people who prefer clickbait headlines do so because they consider them more likely to be accurate. More generally, this finding supports a theory of media choice that more heavily weights perceived source accuracy than preference for congenial information.

The troubling implication of this finding in the current context is that people may not be opting to consume low-quality news sources because they are being cognitively lazy (Pennycook and Rand Reference Pennycook and Rand2019) or because they lack the savvy to differentiate low and high quality news. Instead, our results suggest that – for a certain subset of elderly, pro-government, less educated individuals – they are opting to consume low-quality news sources because they actually trust them more.

Theory and hypotheses

Clicking on a link is central to the process of online media consumption and this is reflected in the behavior of online media firms. This creates the market dynamics of “Clickbait Media” in which each online news story must compete with thousands of other stories for the audience’s attention. Hundreds of new online media firms employ a novel business model enabled by two interrelated features of Clickbait Media: the sharing or retweeting of posts along homophilous social networks and the quantified accretion of these decisions in the form of the number of likes (or shares, or retweets) associated with a given post.

The structural reasons for the emergence of Clickbait Media are complex and outside the scope of this paper. But there exists an intrinsic connection between the business model of Clickbait Media and the style of headline that gives it that name. We argue that this style serves as a signal to readers that the media firm is not aligned with the legacy media establishment. However, it is also a signal of their business model. While the traditional media relies on “credibility” derived from journalistic standards, the “credibility cascades” of Clickbait Media obviate these standards.

The role of social recommendation in online media choice is thus larger today than in previous media contexts, but other theories of media choice remain very much relevant. Sood and Lelkes (Reference Sood and Yphtach2018) summarize the state of this literature, explaining the empirical regularity of consumers’ preference for congenial sources. The key argument is that researchers need to conceptually differentiate preference for congenial sources from preference for congenial information. The former is well-established but often naively assumed to be caused by the latter; Sood and Lelkes (Reference Sood and Yphtach2018) make the case that the preference for congenial sources is also a function of perceived accuracy.

The other major change produced by contemporary online media is the massive expansion in the number of media firms and articles from which to choose. The expansion of the media choice set reveals previously unobservable heterogeneities in consumer preference; considering Prior’s argument that the roll-out of cable television revealed that many Americans had a high “preference for entertainment” and thus opted out of viewing any television news if given the option of more and better entertainment programs (Prior Reference Prior2007).

In the early days of the internet, online headlines were no different from newspaper headlines. But the development of “clickbait” headlines by web native media firms like Upworthy in the early 2010s introduced a new dimension of preference heterogeneity. Munger et al (Reference Munger, Mario, Jonathan and Joshua2020) demonstrate that this “preference for clickbait” varies widely among the US population. Republicans, the elderly, and the less educated tend to prefer clickbait headlines, holding the topic constant and in the absence of source or social recommendation cues. Possible explanations for this variation are simply that these people are less “digitally literate” (Hargittai Reference Hargittai2001; Hargittai, Piper, and Morris Reference Hargittai, Piper and Morris2018) or lower in analytical reasoning (Pennycook and Rand Reference Pennycook and Rand2019), and also posited as explanations for the related propensity to consume “fake news” (Guess, Nagler, and Tucker Reference Guess, Nagler and Tucker2019).

The key limitation of these studies is the same described in Sood and Lelkes’ summary of broader theories of selective exposure: an inability to differentiate between theoretically distinct mechanisms underlying media preferences (Sood and Lelkes Reference Sood and Yphtach2018).

Our approach to address this problem is to adapt the framework used in Prior and Lupia (Reference Prior and Lupia2008) and Prior et al. (Reference Prior, Sood and Khanna2015) and experimentally add incentives for correct answers, thereby raising the relative salience of the motivation for accuracy. In Prior et al. (Reference Prior, Sood and Khanna2015), the behavior of interest is how people respond to survey questions that probe their perception of economic conditions. As in our context, these respondents face incentives that can be in conflict: the desire to be accurate and the desire to report congenial information. In these studies, the addition of an incentive for accuracy decreases the heterogeneity of the responses, suggesting that in the absence of the desire to report congenial information, people’s estimates would be centered on the truth.

The situation with contemporary media choice is different. Our theory is that some people have a preference for clickbait that is linked to a sense that legacy media is aligned with the distrusted political establishment, and that it is only from new online sources that employ the distinctive style of partisan emotional clickbait that can be trusted. Some other people have a preference for non-clickbait based on the analogous idea that only information from vetted and reputable media outlets can be trusted. ^{Footnote 4} Our hypothesis is that the addition of incentives for accuracy will increase the heterogeneity in media choice, along the dimensions that predict preference for clickbait in the pre-experimental phase.

First, we aim to corroborate findings about this preference for clickbait from the US context: it is higher among the elderly, the less educated, and the less digitally literate. Here, we follow the conception of “digital literacy” proposed by Guess and Munger (Reference Guess and Kevin2020): information discernment combined with the basic digital skills necessary to attain it. This is an active and unsettled area of research, but at the time we conducted the experiment, we combined the 6-question “abbreviated battery” proposed in Hargittai, Piper, and Morris (Reference Hargittai, Piper and Morris2019) with the 8-question “social media” battery described by Hargittai, Piper, and Morris (Reference Hargittai, Piper and Morris2019) was the most current and relevant measurement of the concept. See Appendix Section 10 for a full discussion; note that there was an error in our survey instrument and that we only in fact included 5 of the 6-question battery.

Next, we test our main hypothesis:

Sampling and demographic characteristics

We recruited survey subjects using a Facebook ad campaign. The ad simply promoted a survey about “news consumption” and offered the possibility to enter a raffle to win 250 to all participants who finished the 15-minute survey. The campaign ran between September 28 and October 15, 2018. Overall, around 540,000 people were exposed to the ad, 16,000 people clicked on the ad, 4,104 started the survey, and 1,754 finished it. Most attrition was in the very early stages of the survey and was not related to treatment conditions. The campaign was run using Facebook’s tool for promoting pages, which automatically “learns” the optimal target population to maximize the number of clicks to the link. As a result, we did not specify who was exposed to the ad, though we do have ex-post information on demographic characteristics.

Appendix Table 1 presents the full demographic breakdown. Women are overrepresented (72%), but the sample is diverse in terms of age and other relevant covariates. Although not representative of citizens, voters, or even Facebook users, the population contains sufficient variation in the demographics of interest. The other variables listed in Appendix Table 1, and included in our analyses, are defined in the Appendix. Interspersed with measurement questions, we also include an attention check. We ask people to answer “65” to a question to show they are not replying randomly. Of the 1,754 initial respondent, 1,537 passed the attention check. In our analysis, we restrict our focus to people who passed the (pre-treatment) attention check.

Experimental design

Prior to the experiment, we asked respondents to rank a set of fictional headlines in order from “most prefer to read” to “least prefer to read,” to estimate each individual’s taste for Clickbait Media. The treatment headlines vary on two dimensions: they are either “Clickbait” or “Non Clickbait”, and are either “Pro Government” or “Anti Government.” We also include a “Neutral” control headline that is dryly factual and non-partisan.

We follow previous research on clickbait and create partisan emotional clickbait headlines by the addition of phrases like “This will make you furious…” or “You won’t believe this! …” to headlines with a strong partisan valence. The full list of headlines (and their English translations) is found in the Appendix.

We took several steps to mimic the style of Italian media. We first studied clickbait-news outlets, following examples provided by the main Italian website for debunking fake-news butac.it. We selected only topics that were politically controversial, where each side (Pro and Anti government) had a clear stance. We then wrote headlines changing the slant and the style to provide something that would look like it was published from either a slanted newspaper (e.g., La Repubblica or Il Giornale) or from a non-traditional news outlet (e.g., InformareXResistere or Matteo Renzi News). While a reader who is not accustomed to Italian politics might not catch the tics that define the slant and style of the headline, we believe that any Italian reader could easily guess these characteristics by the headline alone.

The “Pro Government” headlines suggest a positive development for the government, for instance, that “The government finally provided a cut to expenses.” While “Anti Government” give a negative spin to the event, for instance, saying “The Government managed to cut just a few expenses.”

In addition to the four slanted headlines, Neutral headlines report events without a clear slant or clickbait style. They are designed to imitate sources with a sober style, with no implicit judgment on the Government. All five articles are on the same topic, hence people were asked to choose between different ways of spinning the same story.

The pre-treatment section consisted of two sets of 5-headline choices that needed to be ordered in terms of preference to read. We then use this ordering to construct the individual-level measure of preference for clickbait (CB-Score): we assigned 4 points if a clickbait headline was ranked first, 3 if ranked second, and so on. The maximum CB-Score is 14, which would result if a user ranked the two clickbait headlines in the top two positions for both batches (i.e., 4 + 3 + 4 + 3 points, adding up to 14). The mean value of this variable is 6.7, the median is 7. This variable is a strong predictor of the likelihood of choosing clickbait in the second part of the survey and is a crucial control for ex-ante taste for clickbait.

Appendix Table 2 shows that older, less educated, more pro-government, less digitally literate people are more likely to choose clickbait. This finding nicely replicates previous research from the USA (Munger et al. Reference Munger, Mario, Jonathan and Joshua2020).

Next, we implement our experimental manipulation in order to understand the mechanism that explains variation in taste for clickbait. In this section, subjects had to rank five headlines with the same structure as above, but they were told that they would then read the article they put in the first position.

Our experimental manipulation involves changing the message that subjects receive before selecting a headline. All of the articles cover a cut in expenditure for pensions in the House of Representatives (Camera dei Deputati), which was one of the main campaign issues for the largest party in government (Movimento Cinque Stelle) during the electoral campaign. We have three treatment conditions:

No Incentive: “Attention! In the next section we will give you the titles of five new articles. This time, we will also display the text of the article to read.”

Intrinsic Incentive: No Incentive + “After that, there will be a small quiz on the theme of the article.”

Monetary Incentive: Intrinsic Incentive + “There will be a second drawing for an Amazon voucher to reward the correct answers. The closer your answers are to the real quantities, the more likely you are to win an extra prize of €250.”

The wording of the stimuli is designed to map tightly with our theoretical interest. We say explicitly that the quiz will be on the topic of the article, not on the article itself. We also say that the answers have to be “vicine al vero”: close to the truth, not close to the article. The text of the articles varied in style but contained all of the same information necessary to answer the quiz questions correctly; crucially, subjects were unaware of the content of the article before selecting it based on the headline.

The monetary incentive is exactly as large as the raffle prize with which we recruited subjects. The five headlines can be found in the Appendix.