The Macedonian Fake News Industry and the 2016 US Election

ABSTRACT During the 2016 US presidential election, Americans were exposed to an onslaught of disinformation on social media. Many of the most viral posts originated from Veles, a small town in central Macedonia. During fieldwork in Veles, where we interviewed several residents and disinformation creators, we found that the epicenter of this viral phenomenon was Mirko Ceselkoski, an autodidact social media expert, teacher, and mentor to Veles’ fake news operators. We interviewed Ceselkoski and registered and attended his online course—the same course numerous Veles residents took offline. Our research confirms (1) the pivotal role Ceselkoski had in the creation of this industry; (2) the economic motivation driving the fake news disseminators; and (3) the manner in which the mostly young people in their early twenties with little English fluency were able to generate so much traffic and disseminate so much disinformation.

D uring the final three months of the 2016 US national presidential election, the 20 topperforming false election stories generated more engagement (i.e., shares, reactions, and comments) on Facebook than the 20 top-performing stories from major news outlets such as the New York Times, the Washington Post, and NBC News (Silverman 2016). Many of these hoax stories-such as the claims that Democratic candidate Hillary Clinton had "sold weapons to ISIS" and that Pope Francis had endorsed the candidacy of Donald Trump-originated from websites whose IP address pointed to a little town in Macedonia called Veles.
News stories published by The Guardian in late August 2016 (Tynan 2016) and Buzzfeed a few days before the election (Silverman and Alexander 2016) were the first to expose this phenomenon. Since then, this small fake news industry has been covered extensively in the media (e.g., Cvetkovska 2018;Soares 2017;Subramanian 2017). Nevertheless, there is still much we do not know, including why so much of this disinformation originated in Veles and what made its disseminators so successful.
To answer these questions, we conducted fieldwork in Veles, interviewing fake news generators and local residents familiar with this industry. We also interviewed and investigated Mirko Ceselkoski, a self-taught, viral marketing specialist whose name has been associated with the fake news business since the story's inception. Our findings reveal the mechanisms behind the fake news production and the ease with which fake news may be generated and disseminated. They also show that the driving force behind the Veles fake news business is economics rather than politics.

ALL ROADS LEAD TO VELES
In recent years, fake news has become a buzzword that encompasses several related phenomena, including propaganda, news satire or parody, misinformation, and disinformation (Tandoc et al. 2018). However, in the 2016 electoral context, we can state that fake news coming from Veles was a form of disinformation-a deliberate effort to create and share false information (Wardle 2017).
Immediately before the 2016 election and its aftermath, mainstream reporters began exposing the source and magnitude of the fake news industry. These news reports showed that most disinformation articles and posts demonized Hillary Clinton and glorified Donald Trump (Silverman 2016); they also provided evidence that individuals exposed to those stories tended to believe them (Silverman and Singer-Vine 2016). As a result, it became almost a truism that if not for fake news, Trump would not have been elected president (e.g., Parkinson 2016; Subramanian 2017).
The scholarly response to this question has been more measured. Reviewing the information environment to which voters were exposed in the months before the election, Jamieson (2018) argued that it indeed is likely that disinformation swayed the election in favor of Trump. Likewise, Gunther, Beck, and Nisbet (2019) argued that disinformation may have aided Trump by leading many former Obama voters away from Clinton in 2016. Conversely, measuring exposure to and recollection of fake news, Allcott and Gentzkow (2017) concluded that it is unlikely that fake news gave the victory to Trump. Moreover, given that the Macedonian stories were rarely among the most significant during the campaign, Faris et al. (2017) argued that disinformation from partisan sources are the main propagators that swayed public opinion.
Fake news stories that exploded on social media in the run-up to the 2016 presidential election can be traced to many sources. Some were produced by American citizens Tynan 2016) and some originated in Russia (US Senate Judiciary Committee 2017). Yet, many reports indicate that the epicenter of the fake news industry was the sleepy little town of Veles in Macedonia (Cvetkovska 2018; see also Silverman and Alexander 2016;Subramanian 2017). According to most accounts, the hoaxers in Veles were primarily poor, apolitical young men in their teens and twenties who, under the tutelage of the web-marketing specialist Mirko Ceselkoski, became adept at luring American conservatives to their websites and Facebook posts and made clickbait fortunes. A more recent and somewhat conflicting account found that some sites emerged as early as September 2015. They were created by Trajche Arsov, a local attorney who began publishing hyperpartisan content with the assistance of two American conservative writers and a Russian nationalist .
In the face of uncertainty about the origins and impact of the fake news phenomenon, and given the potential importance of the subject for the functioning of democratic societies, we decided to conduct our own fieldwork in Veles. Our aim was to better understand who these fake news creators were (and still are), what motivates them, how they learned to generate and disseminate political content, and what accounts for their success.

FIELDWORK IN VELES
In the initial stage of research, the first author traveled to Veles in May 2018 and conducted her fieldwork, adopting an ethnographic approach common in the study of communities of practice. Originally viewed as a group of people that share a passion or a vocation and whose interactions allow them to gain knowledge and grow professionally (Lave and Wenger 1991), communities-of-practice research now commonly encompasses additional stakeholders within a greater community (Thomson et al. 2013). This methodology requires a researcher to engage with the professional community as well as the community that surrounds it, such as novices, family members, and locals familiar with these professionals.
Interviewees were identified through a "snowball" method, beginning with two local residents whom the author encountered on arrival in Veles. This led to others with increasing knowledge of and familiarity with the industry. Eventually, nine interviews were conducted-five with locals who were familiar with actors in the fake news industry but were not themselves operatives; three with fake news creators; and one with a foreign diplomat who has closely followed the Veles fake news industry. Because most locals spoke little English, most of the interviews were conducted with the assistance of a volunteer translator from Skopje, who escorted the first author during her time in Veles. Most subjects refused to allow the interviews to be recorded. Therefore, the research is based on copious notes taken by the first author during the interviews.
The original plan was for the first author to spend a week in Veles and interview as many people as possible. However, after being asked to "pay" for an interview with sexual favors or €5,000 to replace their business if their page resulted in getting shut down, she decided it was no longer safe or viable for her to stay in Veles and terminated the field research. 1 Veles is a town of 45,000 residents about a one-hour drive southeast of Skopje. The city is quite poor, with an average monthly net wage of $380 USD (well below the national average), a 22% poverty rate, and a 24% unemployment rate (Ilievski, Ghorgjievska, and Zlatkov 2016). The residents who were first approached for this study-all individuals of limited means-admired their compatriots who had pulled themselves out of poverty through the dissemination of fake news. Notably, everyone in Veles with whom the author broached the subject-whether at a family meal or the local café or gas station-indicated that they knew (or at least knew of ) operatives in the industry. This was especially true for young people, who were more than willing to support the current research by sharing the names and addresses of disinformation disseminators.
In one case, the first author was escorted by a local resident to the home of one operative. The home, modern and well maintained, was easy to identify among the more neglected houses on the street. The author's host explained that he used to be friends with two brothers who work in the industry but that since they became wealthy, they no longer valued his company. According to locals, these two brothers were among six Veles residents who had become highly successful as fake news operatives, earning more than $1 million USD in 2016. Local rumor had it that one person still received $500 USD monthly from President Trump in appreciation for his help during the election.
In numerous conversations, the first author probed residents of Veles-including fake news operators themselves-about whether they had any ethical problems with fake news dissemination, or even remorse. The answer was unequivocally no. "We can't afford anything, and if the Americans can't tell the difference [between real and fake news], it's their fault," said one young woman. Another man told us, "It's the Americans' fault for not understanding things. So what if we make money off their inability to discern [fact from fake]." Several residents said repeatedly that they wished for "a normal life," and this was one way to achieve that goal. This moral elasticity is not surprising, given the fact that ample research shows individuals are more willing to break the law with the decline of wages (Machin and Meghir 2004). Hence, for an extra monthly income of $300 USD, an American may be dissuaded from starting a business that spreads shady, false "We can't afford anything, and if the Americans can't tell the difference [between real and fake news], it's their fault," said one young woman. stories; however, for a resident of Veles, where that sum can sustain his family for a month, it may be enough.
Could it be that Macedonians themselves have difficulty separating fact from fiction? According to a foreign diplomat stationed in Skopje who spoke to us on condition of anonymity, "Macedonia has the lowest discernibility rate in understanding true and false news due to the poverty and poor journalism [in the country]." This assertion was not supported by our research. When we asked a group of disinformation creators if what they were doing was illegal, their response was that none of their activities were illegal. However, a few laughed nervously, revealing discomfort with the question.
An alternative explanation may lie in Macedonia's political culture and the way that its citizens view themselves and the law. Mungiu-Pippidi (2005) found that Macedonia, like many former Eastern Bloc countries, is characterized by a scarcity of human capital and a slowness to internalize the rule of law. Of the Macedonians surveyed, 53% said the law should not be a barrier to certain behavior; 85% agreed that some people are above the law. Moreover, among the five Balkan nations she surveyed, Mungiu-Pippidi found Macedonia to have the highest levels of interpersonal mistrust, with 72% of respondents stating that only kin can be trusted. Although Mungiu-Pippidi's (2005) research may be dated, it is unlikely that social attitudes have changed substantially in the intervening years. As such, whereas this explanation alone may be insufficient to explain how a small town in Macedonia became a hotbed of fake news production, it may be that Macedonian norms and values facilitate immoral behavior in the face of economic hardship. After all, as a friend of our translator said when probed, "We're too poor to do anything. We have no future working here. We work very hard, and this is a way for our children to make money." Multiple websites report that disinformation creators are young, single men; although some are, our interviews revealed that many are parents supporting families. One individual we talked to showed us his personal Facebook page with photographs of his wife and three smiling children. He told us that if he were to be identified, he would be at risk of losing his family's livelihood.
According to our research, whereas most of the people who operated fake news websites and Facebook pages in 2016 did not become millionaires, many did very well. The three operatives interviewed in this field research reported earning additional income around €1,000 a month-an amount similar to that reported in numerous news stories (e.g., Agence France-Presse 2018). Yet, we learned that to make money, first one needs money. "It's the circle that everyone wants to be part of but can't unless they have 500 euros," said one young woman, who was saving up to take a course that would teach the skills to run her own websites and Facebook pages. This was the Facebook Marketing University, the course established and maintained by Mirko Ceselkoski that apparently launched most of the young fake news disseminators into their career.
Mirko, as locals call him, has become a media sensation. Most reports on the Macedonian fake news industry place him at the center of it-the person who taught more than 100 Veles residents how to make money through the Internet (e.g., Subramanian 2017). Our own research confirms Ceselkoski's central role. Locals repeatedly referred to his course, recounting how his alumni became wealthy. In addition to the young woman mentioned previously, another interviewee-an older woman-said she was putting aside money for her son to take the course. Moreover, scanning the Facebook profiles of the three fake news generators that we interviewed, as well as 10 others we identified, we found only one name that appeared as a Facebook friend on all 13 lists: Ceselkoski.
Enjoying his newfound fame, Ceselkoski-who styles himself as "the man who helped Donald Trump win the 2016 US election" (Ceselkoski 2018)-has now opened his course to the general public, including both Macedonians and outsiders. We paid the $797 USD fee and took his course, which-to the best of our knowledge-no previous reporter or scholar has done. We also conducted two interviews with Cesekloski himself via longdistance video link and in person.

THE FACEBOOK MARKETING UNIVERSITY
Most news reports about the Veles fake news industry tell a similar story-one in which a few teenagers, through some mixture of computer savvy and luck, "hit the jackpot." To illustrate this, Ceselkoski said that some years ago, a woman invented a story about the "hidden meaning" of the colors printed on the flat end of toothpaste tubes. So many people visited her website that it crashed, resulting in enough views to earn about $30,000 USD. Mirko laughed and said, "She made it all up in her house, and now people don't believe the actual toothpaste companies." Nevertheless, the interviews and course materials reveal that, for the most part, success was actually the product of studying the market, learning from competitors, conducting other forms of research, and enduring much trial and error. 2 Ceselkoski's course, the Facebook Marketing University (Ceselkoski 2018), is not an introduction to fake news production and dissemination but rather a primer that teaches students how to utilize the Internet for financial gain by creating sensational content fitting a pay-per-click world. Ceselkoski boasted that 1,200 students already are making "a collective $10+ million per month" and that "maybe you have heard that a team of individuals from Macedonia somehow helped US President Donald Trump win the 2016 US elections. It was my crew and my students from Macedonia" (Module 1-001). Other than this specific reference in the introductory module, Ceselkoski avoids any references to political disinformation throughout the course. In fact, if he was not associated with the 2016 election except for a few references throughout the course, we could not differentiate between the product he sells to similar workshops and Internet marketing academies.
The goal is to create one's own Facebook page with 500 thousand to 1 million followers. This is accomplished strategically.
"We're too poor to do anything. We have no future working here. We work very hard, and this is a way for our children to make money." PS • January 2021 21 In Module 1-003, students begin with a careful analysis of Facebook pages that are already popular and learn how to emulate them. It is crucial to start with a general topic that already has numerous followers and then find a niche within that topic that is likely to be popular while also narrow enough that people will want to engage with the content. Thus, if the topic is health, one must narrow it down to diabetes or anxiety; if the topic is dogs, the page must relate to a particular breed; and if it is politics, one can create a fan page for a certain politician.
Much of the knowhow is the outcome of trial and error. Ceselkoski told us in an interview that some students began with health and celebrity pages but quickly learned that political pages generated greater engagement and were more profitable. They first tried ads against both Trump and Clinton, finding that those against Clinton were more viral and generated more revenue.
Scholars (Mele et al. 2017) and practitioners (Martin 2019) alike concur that conservatives are more susceptible to fake news and more likely to share it (Guess, Nyhan, and Reifler 2020). 3 In addition, students are advised to list 10 competing Facebook pages and manually "spy" on them daily to see where they are successful and how they can be imitated (Module 1-010). Having hundreds of fake news sites and web pages to follow, this is an easy learning lesson.
To continue growing, the module recommends monitoring the performance of posts and spending modest amounts (i.e., up to $10 per day), running different ads, and using A/B testing to see which outperforms others in terms of shares and organic traffic. Narrowing the audience is key to any campaign; therefore, if the aim is to reach Trump supporters, Ceselkoski recommended targeting "veterans, or people who like veterans, or family of veterans: wives, children, maybe brothers and sisters of US soldiers" (Module 1-004). The emotional engagement increases the chances that these individuals will like the page and share its content.
Fake news travels faster and deeper than regular news (Vosoughi, Roy, and Aral 2018), but the key is that most followers need to believe the false story, which requires trust. Therefore, Ceselkoski suggests establishing the Facebook network or web page months in advance, feeding it with relevant content. Once trust is established, one can begin publishing more outrageous content. Ceselkoski told us in our first interview that the ratio should be "80% truth, 20% not truth." The idea behind this ratio is to strike just the right balance. If all stories were true, then-in comparison to regular media-the website or Facebook page would offer no added value and generate no traffic. If, alternatively, all stories were false, even readers with low media literacy would figure out that they are being lied to and abstain from clicking on links. Thus, when the fake news is wrapped in true stories, it establishes and sustains the credibility of the medium. It is that credibility that seduces readers to click on an incredibly interesting story that otherwise would have been flagged as false.
Although learning the technical aspects of viral marketing is a time-consuming task, it can be mastered with practice. Writing content is a different matter altogether. To strike a responsive chord (Schwartz 1974), one must be immersed in the language and culture of the targeted audience. It is clear from our own fieldwork -and supported by numerous media accounts-that most of those operating disinformation sites are not fluent in English (indeed, some barely speak it) and are not well versed in American politics and culture. How is it that they were able to create content that was so engaging?
First, the most effective tool recommended by Ceselkoski is BuzzSumo, a content-marketing research platform. For $99 USD a month, users can learn which content is most shared on social media; see how given domains are performing, including their own; find influencers related to particular topics; track competitors; and even sign up for alerts so that one can ride a new viral trend or topic as it happens (Module 1-009). Operators are required to copy only from the best. Second, to lure readers to one's web page or Facebook page, provocative clickbait titles are required because these exploit an insatiable appetite and curiosity among readers (Anand, Chakraborty, and Park 2017). "It is a hook and you get a chance of less than a second to catch the reader's attention and persuade them that this text is worth their time" (Module 1-019). To succeed, Ceselkoski recommends using numerous free title generators and pretesting them, like one does with stories and posts. The titles do not have to be false, only seductive or sensational enough to urge readers to click on them. For example, a title such as "I supported Hillary until I saw this," which is not false, may outperform a title such as "Hillary had two strokes." The key is spending a few dollars to test which title generates more traffic and then invest in the one that outperforms the other.
Third, the content itself can be obtained from various sources: competitors' Facebook pages; Google's keyword planner; alerts and hot-trend indicators; public reviews and comments on news sites and blogs; and answers to questions on sites such as Quora and Yahoo Answers (Module 1-010). By emulating and even plagiarizing the content of successful competitors, one saves the time and effort of pretesting stories andbecause they are not real news organizations-one does not have to fear a lawsuit for stealing competitors' stories. Moreover, by copying and pasting from these sites, a foreign content creator, who is a non-native speaker, can generate articles that "feel American" in their sentence structure, idioms, and content.

CONCLUSION
Initial accounts on the Veles fake news industry tell the story of people living in conditions of economic hardship who harnessed their recently acquired knowledge of online marketing to make a living-and even acquire wealth-by exploiting conservative Americans' hunger for negative stories about Hillary Clinton (Silverman and Alexander 2016). A more recent account, still under investigation by authorities, found links to American conservative writers working with Trajche Arsov, a Macedonian lawyer, and a Russian nationalist . Speaking with operatives, locals, and Mirko Ceselkoski himself in our own field research, They first tried ads against both Trump and Clinton, finding that those against Clinton were more viral and generated more revenue.
we found no evidence to contradict the latter account but much information to confirm the original narrative.
We found that Ceselkoski began teaching locals his marketing knowhow as early as 2011, four years before Arsov allegedly began creating his own sites. Locals with whom the first author initiated conversations led us to operatives, and both groups revealed Ceselkoski's central role in the creation of this industry. According to Ceselkoski-and corroborated by numerous news stories-he instructed hundreds of students, thereby extending his influence far beyond that of anyone else. When asked whether it is possible that Russian operatives were planting useful content for his students to use, Ceselkoski said that the posts were a "domestic product of Macedonia," but he admitted that many came from US sources, slightly modified.
It is obvious that Ceselkoski has an interest in keeping the title "the man who helped Donald Trump win the 2016 US election," giving him reason to deny any story that does not support this narrative. All of the individuals we interviewed seemed to corroborate his story. Alternatively, it is plausible that the Russian Internet Research Agency (IRA) clandestine operation was so effective that Ceselkoski and all others were unaware of it or refuse to admit that they were manipulated by the IRA. We cannot rule out this possibility, but all of the local operatives insisted that they were behind it all with a single goal in mind: making a profit. As we approach the next US presidential election, we should remind ourselves that if the Macedonians are those making a profit, it must be democracy that is paying the bill.