1. Quantifying change points at scale

I produce a high-level encapsulation of the behavior associated with reported activities of rebel groups in a given year by using a computational text model to assign news articles to one of two group-specific topics. I then aggregate these into a yearly summary of reported activity patterns. In the sections that follow, use the term “frame” to describe the high-level summary produced by the content of, and proportion of articles ascribed to, these topics. Concrete examples of the output can be seen in the Proof of Concept section of the Appendix. The approach conceptualizes “change” via revealed preferences, which capture actions and thus incorporates strategic decisions as well opportunism, disobedience, shirking, or misjudgment (Beshears et al., Reference Beshears, Choi, Laibson and Madrian2008).Footnote ¹

To identify themes, I use structural topic model (STM), an unsupervised topic model. STM is well suited to modeling dynamic changes because it can incorporate document-level covariates (such as time) and has demonstrated validity for short texts on a narrow range of issues (Roberts et al., Reference Roberts, Stewart, Tingley, Lucas, Leder-Luis, Gadarian, Albertson and Rand2014, Reference Roberts, Stewart and Airoldi2016). Although rarely applied in the conflict literature, STM has been used by scholars of comparative politics to identify and quantify trends in opaque processes such as corruption (Pan and Chen, Reference Pan and Chen2018); individual preferences in consensus-based institutions (Baerg and Lowe, Reference Baerg and Lowe2020), and institutional support for human rights abuses (Bagozzi and Berliner, Reference Bagozzi and Berliner2018; Magaloni and Rodriguez, Reference Magaloni and Rodriguez2020).

A two-topic model, such as the one used here, is essentially a unidimensional text scaling model with the discovered topics serving as scale anchor points. Although scholars have, with reason, critiqued unidimensional scaling models as an oversimplification of complex dynamics (e.g., Klar, Reference Klar2014), such scales are widely used to produce high-level summaries of political actors and estimates of how their goals change over time (e.g., Slapin and Proksch, Reference Slapin and Proksch2008; Grimmer, Reference Grimmer2010; Lauderdale and Clark, Reference Lauderdale and Clark2014; Becker and Malesky, Reference Becker and Malesky2017; Egerod and Klemmensen, Reference Egerod and Klemmensen2020). The parsimony of a two-topic model allows the algorithm to efficiently highlight change points without the need for researchers to tune more than 200 separate models, as they would if they customized the number of topics to each group.

1.1. Data and output

I use the source articles collected in the Uppsala Conflict Data Program (UCDP) disaggregated Georeferenced Event Dataset (GED) version 21.1 (Pettersson et al., Reference Pettersson, Davies, Deniz, Engström, Hawach, Högbladh and Öberg2021). This allows the project to benefit from their systematic selection, vetting, and deduplication processes (see Sundberg and Melander, Reference Sundberg and Melander2013). I modeled articles rather than features of events based on the rationale that the texts embed both activities and context. For example, a group may be described as rebels (context) who engage in clashes with government forces (activity) or as local militants (context) who kill civilians and regional policemen (activity).

I subset the GED to conflicts classified as intrastate, extrasystemic, or internationalized intrastate conflict. I then derived a list of the non-state organized armed groups using the UCDP Actor Dataset (Pettersson et al., Reference Pettersson, Davies, Deniz, Engström, Hawach, Högbladh and Öberg2021). This produced articles associated with 191,252 violent events from 1989 to 2020, representing the activity of 352 non-state organized armed groups (actors) across 393 unique conflict dyads. As text model requires enough data to identify patterns in the text distributions, I removed actors with fewer than ten events and those for which fewer than ten words remained after preprocessing.

The 10-event threshold produced 2180 group years for 258 unique armed [non-state] conflict actors, representing 190,688 recorded events from 151 uniquely named conflicts. I created an actor-specific corpus of news articles from the GED for each of these actors and estimated a two-topic STM with splined year covariates. This generated 258 separate models, one for each non-state armed actor with enough content to model. Within each actor corpus, I assigned an expected thematic proportion from that actor's Topic One and Topic Two to each source article. To identify and summarize trends, I assigned each actor-specific Topic One the value of −1 and Topic Two the value of +1. I then used the topic ratio assigned to each article to position each on the [−1,  1] two-dimensional scale.

This analysis produced a set of scaled UCDP source article texts for each conflict actor. To quantify evolution, I took the average yearly position of scaled articles. A yearly topic ratio of −1 indicates that all the articles for that year are associated with the actor's Topic One, and the inverse. At the other extreme, a topic ratio around 0 indicates that news coverage of the group was equally divided between the two topics.

The magnitude of the difference between the topic ratio at year y _t and y _t−n quantifies the presence and magnitude of “change” between a given year, t, and n year(s) in the past. This produced a record of the descriptive evolution of 242 groups and 1818 group-years.Footnote ² This approach to measuring and summarizing changes in armed conflict actors is flexible and customizable.Footnote ³ Given yearly and lagged topic proportion data, researchers can decide how to operationalize a “change” threshold tailored to their research design. As well, the design allows the model to identify topics that are specific to each actor, and for the proportion of the topics to change non-linearly over time.

In the analysis that follows, I operationalize “change” as occurring in a year during which an actor experienced a change in their topic ratio of ≥|1| relative to the previous year. Substantively, this threshold suggests that in one year, reporting on the group's behavior shifted from being predominantly associated with that group's Topic One (or Topic Two) to the group-specific Topic Two (or Topic One). This is a high threshold, capturing rapid change in the descriptions of an organization's behavior.

A group-level distribution of topic proportion changes based on a one-year lag can be seen in Figure 1. The figure shows the per-group distribution of yearly topic changes for each of the GED's 174 armed conflict actors that have more than three active years and ten recorded activities. The X-axis shows the distribution of one-year lagged topic proportion changes. Militant groups with fewer changes have distributions that are concentrated around 0 and thus have a more peaked distribution. Conversely, groups with more framing instability have a flatter distribution, with more proportion changes near 2 and −2. Conflict actors are grouped by quartiles of activity levels. The first quartile contains actors with the least activity, while the fourth quartile represents the most active armed groups. Actors in the first quartile are associated with [10, 25] violent event records, actors in the second quartile are associated with (25, 75] violent events, those in the third are associated with (75, 219] events, and groups in the fourth quartile are associated with 219 or more violent events. As one would expect, for all groups, the distribution of year-on-year topic ratio changes is concentrated around zero. This indicates that most group years have very little change in the framing scale relative to the previous year.

Figure 1.

Group-level distributions of one-year topic changes.

The Supplementary Appendix provides a close analysis of the results for several groups, to illustrate the outcome and suggest face validity for using a text-based measurement strategy. The cases are chosen to cover a range of operational and situational environments. Additionally, the Appendix addresses four points of concern—selection effects, media access, model specification, and face validity—and describes in general how a researcher could analyze the results for specific groups of interest. The Appendix also discusses the opportunities (and limitations) for researchers to qualitatively interpret the meaning of the summary words in the group-specific scale. In the next section, I highlight how the measure can be used quantitatively, as part of a large-N statistical model.

2. Application and replication

To demonstrate how the modeling strategy presented above can enhance existing research into the dynamics and consequences of armed conflict, I extend analysis from Nilsson and Svensson (Reference Nilsson and Svensson2021). Their research design operationalized “uncertainty” via an indicator coding whether the non-state actor in a conflict dyad put forward Islamist claims or motivations in their first year of operation. In addition to demonstrating the utility of my approach, I fill a measurement gap in the literature as dynamic measures of group and conflict level uncertainty were previously unavailable to researchers of civil conflict.

I re-estimate their analysis using changes in the yearly topic ratio as an alternative operationalization of uncertainty. Rapid changes in the depiction of activities of the rebel group should reflect underlying instability in the conflict environment and be suggestive of observer uncertainty about the motivation and operational profile of the non-state actor. Following Nilsson and Svensson's logic that uncertainty prolongs conflict, such representational instability should be associated with conflicts that are more resistant to termination.Footnote ⁴ The primary independent variable is an indicator that captures whether there have been any years with a rapid (as defined below) change in topic proportions. This is a time-invariant measure, intended to closely match the replicated model. The unit of analysis for the model is group-year.

I replicate the central finding of Nilsson and Svensson: conflict dyads in which the armed non-state actor begins the conflict with an Islamist claim are less likely to terminate. I then introduce three measures of change: the first, and lowest, threshold is a binary indicator for whether the group had any one year changes in topic proportions of at least |1| (i.e., half of the range of the group-specific scale); the second is an indicator for whether the group had a shift of at least |1.5| (i.e., two-thirds of the scale's range); and the third is an indicator whether the group experienced a single-year representational change of magnitude |2| (i.e., the entire possible range).Footnote ⁵ Figure 2 summarizes the binary variables that capture whether a specific group was subject to a low (at least |1| on the [−1,  1] scale), medium (at least |1.5| on the scale), or high (at least |2| on the scale) change period across all active years. The Appendix presents a time-varying specification of the model that uses the number of changes, whether there was a recent change, and the length of time between “change” years as independent variables. Additionally, the Appendix presents an analysis of whether the results are robust to more restrictive modeling thresholds.

Figure 2.

Distribution of new binary variables.

The original study and the extension have different temporal scope, which warrants discussion. Nilsson and Svensson's dataset spans 1979–2013, whereas my measure of uncertainty is based on the UCDP GED, which spans 1989–2020. The results below cover 1989–2013, the intersection of the data.Footnote ⁶ The data adaptation needed to incorporate my measure is described in detail in the Appendix.Footnote ⁷ I keep all other parameters the same as in Nilsson and Svensson's analysis.

Figure 3 presents results for the replication and extension of the model of conflict termination. Adding the change measure strengthens the underlying finding that uncertainty prolongs conflict. Armed non-state actors that underwent periods of rapid framing changes are associated with conflicts that are significantly more intractable than conflicts associated with groups that did not experience a representational change. The “Low Change” case of a one-year shift of at least magnitude |1| on the scale [−1,  1] scale is associated with a conflict being 43 percent less likely to change. Likewise, armed groups with a one-year change of at least magnitude |1.5| (the “Medium Change” groups) are associated with a 50 percent decreased likelihood of termination. Conflicts in the “High Change” condition whereby the actor changed across the entire scale (and thus had a one-year change of magnitude of |2|) were 30 percent less likely to terminate, although estimated with lower precision than the original study. These findings are not driven by correlation between framing shocks and Islamist motivations: there is a very low correlation between Islamist motivation and having experienced rapid descriptive change.

Figure 3.

Replication of termination model with change. Termination propensity is modeled via a Cox proportional hazard model to capture the effects of time. In these models, hazard ratios are interpreted relative to 1. A positive value indicates an increased likelihood of termination, whereas a negative value shows a reduced likelihood of termination. To account for the possibility of repeat events the specifications are stratified on termination episodes.

Thus far, the analysis has used a binary measure of whether a militant group experienced a period of framing change. This is the most conservative approach because it is agnostic to the precision with which the text-based measure identifies the date of the change. However, using more precise measurements may produce additional insights into the connection between group transformation and conflict termination. The Appendix extends the analysis by introducing variables that incorporate both time and frequency of group transformation.