Concepts are the building blocks by which we make sense of the social world. Although they reflect the world, they also impose order on that world. Why do some words evoke endless head-scratching, efforts at definition, and debate, while others seem to be self-evident?
Conceptual disagreement is inseparable from, and perhaps necessary for, scholarship. It has been an explicit and central component of fields such as analytic philosophy in the twentieth century, but simmers in the background in other fields. A major step forward occurred in 1956, when W. B. Gallie (Reference Gallie1956) published a foundational article identifying some concepts as “essentially contested.” In the intervening decades, this article has been cited over 6,000 times, and a sizable literature has developed centered on concepts such as democracy, justice, and power that seem to resist authoritative definition (Collier, Hidalgo, and Maciuceanu Reference Collier, Daniel Hidalgo and Olivia Maciuceanu2006).Footnote 1 Contestation is what motivates writers such as David Collier and Giovanni Sartori to take concept analysis seriously.
To many observers, concept ambiguity in social science is problematic. If scholars disagree on what democracy means, it is difficult to cumulate knowledge on the subject, for concepts are the categories by which we classify phenomena and organize research. In this spirit, strategies have been proposed to generate greater clarity and consistency in the conceptual world, variously referred to as concept formation or concept reconstruction.Footnote 2 Others appear to be less concerned with conceptual shiftiness. Just as there is conflict about democracy, so there must be conflict over the word democracy. Conflicting meanings within ordinary language cannot help but reverberate in scientific language, because the latter is never entirely removed from everyday speech.Footnote 3
Regardless of perspective, most commentators find conceptual contestation interesting and important. Yet no attempt has been made to measure the degree of conceptual disagreement that exists or to compile a list of concepts identified as essentially contested. It is unclear how one might distinguish contested from uncontested concepts or test propositions about the causes of contestation. The concept of a contested concept is itself contested.
Measurement is important if scholarship on conceptual contestation is to proceed systematically. Although authors may have an intuitive sense of how contested their own terminological terrain is, they probably do not have a good sense of how “their” concepts compare and contrast with others. After all, contestation is relative, and without an overview of the terrain, it is impossible to make such judgments or to inquire into the (systematic) sources of contestation.
This chapter represents an effort to treat concepts as units of analysis in a large-N study, an approach that is increasingly common with the digital tools of natural language analysis. We begin by introducing an approach to measuring conceptual contestation (as a continuous rather than binary concept) within social science. Next, we explore four factors that may help to explain variation in conceptual contestation: value, abstraction, normativity, and discipline. The concluding section identifies a path for future research.
A Measure of Conceptual Contestation
A concept consists of a term, a definition (intension), and a set of referents (extension). Contested concepts, according to Gallie, are characterized by “endless disputes about their proper uses” (Reference Gallie1956: 169). For present purposes, contestation will be understood as a situation in which the same term is employed in different ways, creating the potential for confusion.Footnote 4
With this understanding, we turn to the problem at hand: How can the notion of conceptual contestation be made empirically tractable. How might we know when a concept’s meaning is disputed?
Population and Sample
A first step is to identify a population of concepts that is potentially subject to the dynamic of contestation. Although scope conditions are not explicit, we infer from work on contested concepts that the intended population extends to common nouns that seek to identify phenomena “out there” in the world of human attitudes and behavior. Democracy is prototypical in these discussions, but one might also include institution, clientelism,Footnote 5 corporatism,Footnote 6 or populism. Purely methodological concepts (e.g., randomization) or philosophical concepts (e.g., consequentialism), as well as proper nouns and other parts of speech (adjectives, adverbs, verbs, articles), are therefore excluded. We restrict our purview to social science, though contestation arguably extends across fields.
To identify a sample of such concepts, we canvas several recent encyclopedias of social science (Kuper Reference Kuper2004; Kurian Reference Kurian2010), retaining only those terms that satisfy the foregoing desiderata. One might easily extend this canvas to additional reference works. However, we do not believe that this would yield many additional concepts, since there is a high degree of overlap across lexicons. Nor is there any reason to anticipate that a larger sample of concepts would render different underlying patterns.
The result of our canvas is a set of 383 nouns that are widely employed across the social sciences (see Table 14.A.1 in the Online Appendix).
Identifying Contestation
We can envision several approaches to identify degrees of contestation across these concepts. One could examine stated definitions of concepts for multiple or competing meanings. Alternatively, one might compare the phenomenal realm that corresponds to rival definitions of a concept. Variation in the categorization of cases according to the same concept might indicate a low level of conceptual agreement.
A third approach is to enlist algorithms for textual analysis (Grimmer and Stewart Reference Grimmer and Stewart2013) to analyze conceptual contestation. Such analysis requires a comprehensive database of texts to analyze. Such databases (e.g., Google Books, JSTOR, Scopus, Web of Science [WS]), however, are not open source and thus can only be accessed through the bespoke search engines that each database provides. Among these data sources, WS offers several analytic advantages in terms of coverage and search fields. One can conduct finely honed queries with Boolean operators across published articles from across the academic world, which number 171 million records from 1900 to the present. Although WS omits most conference papers, doctoral dissertations, master’s theses, books, book chapters, and work published in languages other than English, it encompasses all International Scientific Indexing (ISI) journals and thus seems adequate for our purpose. We do not anticipate that patterns of conceptual contestation would differ across a larger corpus.Footnote 7
To identify the presence of conceptual contestation, we search WS for phrases that indicate issues of conceptualization with respect to a given concept (X).Footnote 8 Such phrases include “concept of X,” “conception of X,” “meaning of X,” or “definition of X.” All of these locutions suggest that issues of conceptualization demand scholarly attention and are likely nontrivial and nonobvious. Although “concept of X” et al. are proxies, we believe they are pretty good proxies for the phenomena of interest. The number of articles in WS with at least one of these phrases may therefore be regarded as a plausible measure of conceptual disputation.
An advantage of this approach is that it casts a fairly wide net, encompassing the views and opinions of any researcher, not just those preoccupied with concepts. Search results should include conceptual papers focused explicitly on the meaning of X, along with empirically focused articles that confront challenges of terminological ambiguity. Our unit of analysis is the article, which avoids overcounting multiple occurrences of a phrase in the same article.
We find that variations in the structure of the query are not very consequential. For example, dropping one of the query phrases results in only a slight attenuation of the yield since many articles include more than one query string and thus remain in the search results. Results are therefore highly robust. Likewise, adding additional query phrases (e.g., “conceptualization of X,” “understanding of X”) has little impact on our results since most of these instances are already captured by other query phrases.
Variations in the grammatical form of the term are sometimes consequential for a particular concept. Here, one must pay close attention to how usage and meaning vary across cognates, for example, republic, republics, republican, republicans, republicanism. It seems clear that republic and republican can be different concepts, warranting separate queries. So, republicanism is preferred to republican, as republican returns many false positives associated with the American Republican Party. In our process, we explore plausible cognates, giving preference to those that maximize search results without introducing error. However, since most concepts have a dominant cognate, these sorts of judgments are impactful only for a small number of concepts and have little impact on aggregate results.
We might have chosen to measure contestation as the share of mentions of X that are contested, in order to account for a concept’s prevalence. However, this approach presents its own measurement problems. Consider rarely used terms (in English) such as autogolpe or commodity stabilization schemes. Because of their unfamiliarity, they are more likely to be defined and thus will be picked up by our proxy measure of conceptual contestation even though they are not contested. Likewise, ubiquitous terms such as democracy or justice may be used without attention to conceptualization where they are employed peripherally. (Only key terms in an article justify an excursus into definition.) Accordingly, we measure conceptual contestation as absolute counts, rather than as relative frequencies. We recognize that the measure may incorporate the prevalence of a concept to some degree.
Since our focus is on social science, we discard results from any non-social science discipline that appear in the top five disciplines identified by each search. Although a small number of results from non–social science disciplines are inevitable – after all, the distinction among fields is not crystal clear – it is not great enough to impact aggregate search results reported for each term.
Data Description
Histograms of the counts produced by queries across all terms listed in Table 14.A.1 are displayed in Figure 14.1, with the linear scale in the left panel and the logged scale in the right panel. (Prior to logarithmic transformation, we add 1 to the scale.) As is common with bounded scales, the modal outcome is 0. In these cases (nearly 10 percent of the sample), no article in the entire WS database contains any of the four locutions regarded as proxies for conceptual contestation. We regard these terms as uncontested. The distribution skews to the right, which suggests that a smaller number of terms are continually and repeatedly contested. We regard these terms as highly contested.
Histogram of contestation.
However, we find no obvious discontinuity in the distribution that would suggest a qualitative distinction between terms that are weakly and strongly contested. Accordingly, we regard contestation as a matter of degree, a continuous scale. (All 383 terms in the sample are maintained in subsequent analyses.)
Looking across our sample, we find that articles indicating signs of conceptual contestation comprise a small portion of the total mentions of a subject. For every thousand articles mentioning X in WS, six (on average) refer to the concept, conception, meaning, or definition of X. This conforms to our intuitive notion of the subject: most social science research is empirical, and only a small number of published papers devote substantial attention to matters of conceptualization.
Validity
We turn now to a consideration of measurement validity, along the lines that Adcock and Collier have sketched.Footnote 9 Convergent validity cannot be assessed, as there is no extant measure of conceptual contestation to serve as a reference point. In this setting, face validity must suffice (though one might also assess causal validity by the patterns revealed in the next section).
To explore the matter, let us consider the concepts that form the long right tail in Figure 14.1. Those with a score of over 300 include community, creativity, culture, democracy, development, education, family, gender, human rights, justice, leadership, learning, market, participation, politics, power, race, rationality, religion, representation, social capital, sovereignty, the state, trust, and violence. As it happens, these concepts are frequently perceived in the literature as contested, a judgment that resonates with our own intuitions and, we imagine, with the intuitions of many readers. We regard this as a strong signal of face validity.
To provide a more systematic test, we drew fifty concepts randomly from our sample. These were presented to a panel of five political scientists who have written about concepts (and thus are versed in the notion of contestedness) but had no knowledge of our project. We asked each expert to code the degree of contestation that they associate with that concept on a 1 to 7 scale. After combining these ratings into a single estimate with an item response theory (IRT) model, we find a modest but highly significant positive correlation (Pearson’s r = 0.48). (For further details, see Online Appendix B). This also bolsters our assumption that our proxy is measuring what we intend it to, albeit with some noise.
In a final validity test, we identified a stratified random sample of ten terms from our sample of 383, displayed in Table 14.1. Of these ten, we chose five randomly from among those appearing to have elicited little or no contestation (a score of 0 on our WS query-based measure), and five terms from among those that elicited a great deal of contestation (over 500 hits on our query-based measure). For each term, we randomly chose five articles employing that term, using the WS query (see Online Appendix E). We classified the degree of semantic inconsistency across the five articles in a four-part ordinal scale: 1 = strongly consistent, 2 = weakly consistent, 3 = inconsistent, and 4 = strongly inconsistent. See Table 14.1.
| Contestation score | Semantic inconsistency | Meaning(s) | ||
|---|---|---|---|---|
| Low contestationa | 0 | 1.2 | ||
| Arms race | 0 | 1 | Competition between nations for superiority in the development and accumulation of weapons. | |
| Caucus | 0 | 2 | A meeting at which local members of a political party register their preference among candidates running for office or select delegates to attend a convention. A conference of members of a legislative body who belong to a particular party or faction. | |
| Isolationism | 0 | 1 | A policy of remaining apart from the affairs or interests of other groups, especially the political affairs of other countries. | |
| Martial law | 0 | 1 | Military government, involving the suspension of ordinary law | |
| Stagflation | 0 | 1 | Persistent high inflation combined with high unemployment and stagnant demand in a country’s economy. | |
| High contestationa | 945 | 3.4 | ||
| Citizenship | 804 | 4 | Generally, the position or status of being a citizen of a particular country. However, more specific attributes corresponding to different concepts of citizenship differ (e.g., responsive, active, dual, flexible, fungible, fragile, reparative). | |
| Community | 1302 | 3 | (1) A group of people living in the same place or having a particular characteristic in common. (2) A feeling of fellowship with others, as a result of sharing common attitudes, interests, and goals. | |
| Gender | 753 | 2 | The male sex or the female sex, especially when considered with reference to social and cultural differences rather than biological ones, or one of a range of other identities that do not correspond to established ideas of male and female. | |
| Justice | 907 | 4 | Generally, a just, impartial, or fair behavior or treatment. However, more specific attributes differ (e.g., sustainable, distributive, restorative, procedural, social, institutional, spatial). | |
| Power | 959 | 4 | Generally, the capacity or ability to direct or influence the behavior of others or the course of events. However, specific concepts of power invoke various features of the concept (e.g., soft, hard, structural, power-over, power-in, power-with). | |
aScores in this row represent averages across the category.
This coding exercise seems to confirm that terms identified as having low contestation in our query-based methodology call forth much less semantic disagreement. The mean score for low-contestation terms in Table 14.1 is 1.2 while the mean score for high contestation terms is 3.4. This final validity test is thus in sync with other tests (discussed earlier), offering further reassurance that we are measuring what we purport to measure.
Explanations
Why are some concepts more contested than others?
We focus on four characteristics of concepts that may shed light on varying levels of contestation: scholarly value, abstraction, normativity, and discipline. After introducing the rationale for each factor and its operationalization, we conduct an analysis in which these variables are employed as predictors.
Value
A common-sense explanation for why some terms are more contested than others lies in the fact that some terms are more valuable than others, that is, more useful for the work of social science. Arguably, overall interest in a concept drives the market for discussions of conceptualization. The more valuable a term, the greater the need for explications of its meaning.
This seems reasonable from the point of view of the production of knowledge since the definition of well-traveled concepts matters more than the definition of poorly traveled concepts. If we get commonly used concepts wrong or we neglect to note important ambiguities, a large literature is affected. Conceptual muddles are less impactful where the literature is less extensive.
To measure overall value, we conducted a query of WS centered on the number of social science articles that mention X (in any context). Terms that are more prevalent – receiving more overall hits – are assumed to be of greater value to social science. Because of its highly skewed distribution, we employ the logarithmic transformation of this variable in the analyses that follow.
Importantly, including this variable helps us to account for elements of concept prevalence that might have confounded our counts of conceptual disagreement, though as we suggest above the two are not perfectly correlated. An article that registers a term as contested will also register that same term as valuable. However, the rate of contestation is so small (6/1,000, on average, as noted) that it does not raise concerns about circularity between inputs and outputs in the analyses that follow. Indeed, model-fit statistics are nearly identical if overall mentions are purged of conceptual mentions. Moreover, the fact that attention to conceptualization is minuscule as a share of total mentions suggests that the tail is not wagging the dog. Authors are not driven to write articles about X because X is contested.
Another potential objection to our measure of conceptual value is that correlations might be driven by stochastic processes. Specifically, total mentions of X may be associated with conceptualizations of X because – for unknown reasons – a certain share of studies of a given concept will focus on matters of conceptualization. We think it more plausible to suppose that if there is a correlation between mentions of X and conceptualizations of X, it is the product of a causal relationship, and the most likely cause appears to be overall interest in X as a conceptual vehicle, as argued.
Abstraction
What makes a concept difficult to define is, arguably, its indistinct relationship to the things in the world it purports to describe. Ambiguity increases as the distance between term and referents increases, that is, as it becomes more abstract.
By abstraction, we mean (1) that a concept has a large scope (extension), and (2) that many intermediary concepts lie “beneath” it (in a taxonomic sense). For example, democracy applies to a wide range of phenomena (e.g., countries, cities, school boards, games, families) and sits above lower-order concepts such as participation and turnout that may be, by virtue of their greater concreteness, less prone to contestation.
Granted, concepts lying at the very top of a ladder of abstraction such as phenomena or entities are not always hotly contested. We surmise that this is because concepts at an extremely high level of abstraction hold very little interest for social scientists – or anyone else, for that matter. They are uncontested because no one cares to contest them: There is nothing at stake. Because of their trivial status, these sorts of terms are unlikely to become fodder for specialized social-science definitions, falling outside our population of interest. With this caveat, it seems reasonable to suppose a monotonic relationship between abstraction and contestation.
To measure abstraction, we enlisted six independent raters, who coded each term in Table 14.A.1 along a five-point scale. We then combined these codings through an IRT model to produce an interval scale, as described in Online Appendix C.
Normativity
What makes a concept contested is not simply its level of abstraction but also its normative valence, which Gallie (Reference Gallie1956) refers to as “appraisive.” If a concept has a strong positive or negative valence, we may expect people – lay speakers as well as scholars – to fight over its meaning.
Some will want to expand the meaning of X so that it applies to new phenomena not originally envisioned as part of its extension. For example, concepts with positive valence such as human rights, which originally referred to a small set of basic rights, have been applied to all sorts of phenomena – education, employment, even democracy – that were not envisioned by early users of the term. The same expansion of meaning occurs with concepts carrying a negative valence such as fascism or genocide.Footnote 10 Since not everyone adopts these semantic expansions, there is plenty of ground for contestation.
Note also that control over meaning may have important policy repercussions. Whoever defines the meaning of human rights, fascism, and genocide may affect the attitudes of citizens and the actions of governments across the world. This feature of normatively charged words may also fuel conceptual contestation.
As with abstraction, normativity is measured in an ordinal fashion relying on codings by six independent raters, which are combined through an IRT model to produce an interval scale (see Online Appendix C).
Discipline
Plausibly, some disciplines are more preoccupied with matters of conceptualization than others are. Indeed, the tradition of “concept studies” referenced at the outset is dominated by anthropologists, linguists, philosophers, political scientists, and sociologists (along with those from cognate fields such as communications and law). These same fields are often taught through key concepts, which are thought to provide a useful overview of the subject. In other fields, a “conceptual” approach to knowledge would be regarded as heterodox, or the important terms would pertain to methodology or pure theory and thus fall outside our purview. This suggests that there might be something about certain fields that lends itself to conceptual contestation.
To test this proposition, we code each concept in our dataset according to its home turf, classified as anthropology/archaeology, economics/business, political science/law, psychology/education, or sociology/demography. In cases where a concept is employed regularly across several of these disciplinary classifications, it is coded accordingly, though most concepts are judged to have a single disciplinary home.
Analysis
Having stated four hypotheses about the sources of conceptual contestation, we are now in a position to test their impact. Since the outcome of interest – our proxy measure of conceptual contestation – is strongly right-skewed, we employ the logarithmic transformation in the following tests (illustrated in the right pane of Figure 14.1). This measure is regressed against variables measuring value, abstraction, normativity, and discipline, as described. Benchmark analyses, shown in Table 14.2, use linear models with robust standard errors. Online Appendix A displays descriptive statistics (Table 14.A.2) and intercorrelations (Table 14.A.3).
| Model 1 | Model 2 | Model 3 | Model 4 | Model 5 | |
|---|---|---|---|---|---|
| Concept characteristics | |||||
| Value | 0.635Footnote *** (260.209) | 0.543Footnote *** (18.518) | 0.532Footnote *** (15.514) | ||
| Abstraction | 0.820Footnote ***(15.177) | 00.304Footnote ***(6.090) | 0.307Footnote ***(6.168) | ||
| Normativity | 0.181Footnote ***(2.797) | 00.189Footnote ***(5.140) | 0.179Footnote ***(4.369) | ||
| Disciplines | |||||
| Anthropology/archaeology | −0.095 | ||||
| (−0.435) | |||||
| Economics/business | −0.333Footnote ** | ||||
| (−2.516) | |||||
| Political science/law | −0.110 | ||||
| (−0.713) | |||||
| Psychology/education | 0.016 | ||||
| (0.088) | |||||
| Sociology/demography | 0.138 | ||||
| (1.234) | |||||
| R2 | 0.653 | 0.361 | 0.0185 | 00.712 | 0.719 |
Notes:
Outcome: conceptual contestation (log). Constant not shown. N = 383.
Estimator : ordinary least squares, t statistics in parentheses, robust standard errors.
*** p < 0.01, **p < 0.05, *p < 0.10
Value is evidently a very strong predictor of conceptual contestation, accounting for nearly two-thirds of the variance in Model 1 and robust in all specifications. Abstraction is also a fairly strong predictor, though its impact diminishes when value is included in the specification. (The two variables are positively correlated: Pearson’s r = 0.59.) Normativity explains little variance on its own but is highly robust in all specifications.
Thus, three core characteristics of concepts explain over 70 percent of the variability in contestation, as shown in Model 4. It is difficult to assess their relative contribution because interrelationships among these three factors are ambiguous. Plausibly, value is endogenous to abstraction and normativity; if so, estimates in Model 4 are biased upward for the former and downward for the latter.
Interestingly, disciplines add little to the benchmark model, as registered by total model fit, which is virtually indistinguishable across Models 4 and 5. Among disciplines, economists appear to be somewhat less interested in conceptual matters. This fits with our priors, but it is not an especially strong relationship. (Note that because codings for disciplines are not mutually exclusive, collinearity is reduced, and it is therefore possible to include all five dummy variables together in a single model. Results for these variables when tested individually are very similar to those reported in Table 14.2.)
We should acknowledge the possibility that other factors, untested here, are at work. That said, many features commonly associated with contestation such as openness, complexity/multidimensionality, and acknowledgment of competing views are difficult to distinguish from contestation itself. Indeed, they are often presented as definitional rather than explanatory.
Importantly, all of the chosen regressors in Table 14.2 are subject to caveats if viewed as playing a causal role. Note that the data-generating process is difficult to reconstruct and variables of theoretical interest are of uncertain exogeneity, as discussed in Online Appendix D. Yet, even if regarded as descriptive, patterns illustrated in Table 14.2 are nonetheless informative.
Conclusion
We have taken a broad and empirical approach to understanding the extent and explanations of conceptual disagreement. In doing so, we have chosen to focus on a few aspects of an extraordinarily complex subject. We have flattened it out, so to speak, leaving aside many of the nuanced insights and interpretations pertaining to specific subfields, methodologies, writers, theories, time periods, and concepts generated by the large field of studies focused on the history and meaning of concepts (referenced at the outset).
Our intention is not to replace this traditional approach to concepts but rather to complement it. Insofar as conceptual contestation is worthy of commentary, it is also worthy of systematic measurement. Through careful measurement we may gain a better grasp of what we are talking about when we say a concept is contested, and a better sense of the sources and implications of that contestation.
After introducing our approach to measurement, we turned our attention to four potential sources of conceptual contestation: value, abstraction, normativity, and discipline. The first three factors appear to explain most of the variability in conceptual contestation. “Little” concepts – those that are less valuable for the work of social science, less abstract, and less inflected by normative concerns – generate less confusion than “big” concepts.
Arguably, little concepts also do less work, or less important work. Important theories are often framed with contested concepts. This is a reminder of the integral role big concepts play in the conduct of social science.
We believe that an empirical investigation into concepts, such as the one pursued here, may be fruitful for addressing many additional questions. For example, are social sciences (at large) more enmeshed in conceptual debates than the natural sciences? Are conceptual papers more likely to appear in certain journals within each discipline? Is there an association between qualitative work and preoccupation with conceptualization? Is there a trade-off between conceptualization and measurement? Do patterns in ordinary language mimic those in the specialized language area of social science?
For these questions and many others, the methodology set forth in this study offers an empirical handle. There is a lot we can learn about the changing shape of the social science universe through focused queries using platforms such as WS, Scopus, Google Scholar, Semantic Scholar, and OpenAlex, which was launched in 2022.Footnote 11 For insight into everyday language, platforms such as Google Books beckon. Eventually, digital semantics may provide more precise measures of conceptual disputation,Footnote 12 and causal relationships may be investigated with experiments and natural experiments (see Online Appendix D). In the coming years, we envision an expansive research agenda that builds upon the classic texts of concept analysis, as we have done in this modest study.
Open access