What turns a concept into a valuable, indeed a valid, fact-finding container?
Sartori, Reference Sartori1970, 1039
What criteria should guide political scientists in choosing between alternative systematized concepts? Failure to establish criteria for good concepts may lead to unsuitable concepts or to equivocality and confusion, and thus to a less well-functioning science. This is why political scientists have come up with criteria and constructive suggestions not only for “measurement validity” but also for what is here called “concept validity,” ensuring that measurements represent good concepts (Gerring Reference Gerring1999; Reference Gerring2012; Goertz Reference Goertz2020).
Existing lists of criteria for good concepts include standards such as resonance, consistency, differentiation, causal utility, operationalization and so forth. Such lists, however, usually omit criteria for deliberating between and pondering different normative considerations. This article highlights the importance of filling this lacuna and suggests one particularly important criterion: a “concern” criterion. This criterion pertains to the extent to which a chosen systematized concept reflects the worrisome element(s) of the political world.
Hesitation in setting criteria for normative deliberation derives from skepticism about debates that involve values, which are deemed inimical to standardization and harmonization. In particular, hesitation is connected to the idea of “essentially contested concepts” (Gallie Reference Gallie1956), which points to the fundamentally unresolvable nature of some concepts. More than a half-century ago, W. B. Gallie famously pointed out that normative disagreements often lead to this very contested-ness.Footnote 1 Following Gallie and contemporary scholars, however, “essentially contested concepts” do not force us into “agreeing to disagree” but rather to acknowledge the importance of well-reasoned contestation (Gallie Reference Gallie1956, 188–92; see Collier, Hidalgo, and Maciuceanu Reference Collier, Hidalgo and Maciuceanu2006). The point emphasized here, therefore, is that even when concepts involve evaluative and normative assumptions, contestation about these assumptions needs to be disciplined by some criteria. A concern criterion, I argue, should be considered as one such criterion.
The structure of the article is as follows. The second section outlines the challenge in political science of deciding between alternative concepts and concepts’ structures. It locates the contribution of the paper within the semantic-pragmatic approach to conceptualization (Brady, Collier, and Box-Steffensmeier Reference Brady, Collier, Box-Steffensmeier and Goodin2011).
The third section reviews existing criteria for good and useful concepts and their shortcomings, especially the avoidance of criteria for good normative considerations, in general, and of a concern criterion, in particular. It presents the concern criterion as one possible (nonexhaustive) complementary criterion. In some cases, the section shows, it is used naturally and implicitly. An example discussed in this section is the way that political scientists in previous decades have conceptualized “political polarization.” Concern, it is argued, has rightly served the prioritizing of some conceptions of political polarization over others. The section concludes by drawing out the lessons from such cases and the importance of making this criterion explicit.
The fourth section examines the theoretical and practical implications of the concern criterion, highlighting its advantages and explaining why it is particularly well suited to fill the gap and complement existing criteria. In essence, it argues that this criterion can help mitigate deep disagreements over values and significance by shifting attention from political ideals to underlying concerns. Although the identification of such concerns may itself be contested, it is likely to be less disputable than (positive) ideals—a point emphasized by thinkers such as Karl Popper and Judith Shklar in other contexts. The section then turns to further practical implications, illustrating how a concern criterion can be applied not only in deliberating between concepts but also in prioritizing dimensions within multidimensional concepts in political science. The fifth section addresses three central objections to the concern criterion, and the sixth concludes.
As suggested in the article, a concern criterion could help scientists in deliberately structuring value-laden and contested concepts in political science, such as democracy, corruption, social cohesion, poverty, equality, and so forth. However, the criterion is only complementary to other criteria and is not sufficient by itself. Moreover, it is only necessary for concept validity in cases of concepts that are meant to avoid harmful consequences and so not for every concept. Importantly, I argue that the criterion is especially relevant in a worrisome political world or under particularly worrisome circumstances.
Deciding between Concepts (and Concepts’ Structures)
Political science, like any science, relies on concepts. This article adopts the semantic–pragmatic approach to conceptualization (Brady, Collier, and Box-Steffensmeier Reference Brady, Collier, Box-Steffensmeier and Goodin2011)—the Sartori-Collier tradition—which holds that the scientific use of concepts requires an explicit explication of meaning and often a pragmatic choice among alternative meanings.Footnote 2 Indicators in political science, according to this approach, are designed to represent systematized concepts or “conceptions.”Footnote 3 Measuring democracy, for example, requires an explicit explication of which conception of democracy is being measured; otherwise, measurement becomes vague and problematic.
This approach is contrasted with the “classical syntactic–statistical” approach (Brady, Collier, and Box-Steffensmeier Reference Brady, Collier, Box-Steffensmeier and Goodin2011) common in psychometrics and to a lesser extent in political science. This approach relies on latent variables and constructs as opposed to concepts. Such constructs receive their characterization by revealing underlying statistical routines that elucidate similarities and differences between entities, using methods such as factor analysis, principal component analysis, cluster analysis, and so forth (see Briggs Reference Briggs2022). By using this approach, disputes about the meanings of a concept are avoided, and statistical commonalities between bundles of attributes determine what a term stands for.
In the semantic–pragmatic approach, however, we address the process of constructing good concepts as based on reasons, judgments, and argumentation—and not on statistical regularities alone.Footnote 4 Importantly, reliance on judgment does not imply subjectivity or relativism: conceptions can be intersubjectively justified through public reasons and defensible criteria (Schedler Reference Schedler2012). Reliance on judgment also does not eliminate the need to use the rest of the criteria concerning conceptual validity (elaborated in the next section). For instance, conceptions that are not operationalizable, and hence do not fit as objects of science, would not make for good candidates. Note how the challenge addressed here is not one of avoiding bad or disordered concepts—a challenge elaborated in Schedler (Reference Schedler, Badie, Berg-Schlosser and Morlino2011), Sartori (Reference Sartori1970; Reference Sartori1984), and Collier and Mahon (Reference Collier and Mahon1993)—but of choosing among multiple well-functioning alternatives. Meeting this challenge, recently termed “the fundamental problem of conceptualization” (Graefrath and Jahn Reference Graefrath and Jahn2025), requires additional evaluative criteria.
In addition to choosing between alternative conceptions, a further role for concept validity and for employing criteria for good concepts is to define the relations between different aspects of one concept. Many political concepts, and hence their measurement devices, are multidimensional, requiring specification of their constituent elements and logical structure,Footnote 5 which is reflected in indicator selection and aggregation schemes (Goertz Reference Goertz2006; Reference Goertz2020).Footnote 6 The semantic–pragmatic approach entails that data and statistical routines, although they may indeed be helpful, are not sufficient in determining structures and that therefore good arguments are required.
Basic Criteria for “Good” Concepts and Their Limitations
Although much attention has been devoted in political science to measurement validity at the indicator level—focusing on reliability and validityFootnote 7—I address, as should be clear, a higher-level question: the basis on which competing conceptions and structures are decided. What constitutes concept validity? This question is distinct from and prior to indicator-based validation (hence, the epigraph from Sartori).
To cope with this challenge, political scientists have come up with lists of recommendations for crafting good concepts and well-suited logical structures (Gerring Reference Gerring1999; Reference Gerring2012; Goertz Reference Goertz2020). John Gerring’s list is prominent and contains seven basic criteria: resonance, domain, consistency, fecundity, differentiation, causal utility, and operationalization. Footnote 8 However, Gerring’s list (as do Goertz’s suggestions) does not refer explicitly to criteria of normative significance, in general, and to concerns in particular. To illustrate the need for these complementary criteria, I first analyze the example of political polarization, an increasingly dominant concept in political science.Footnote 9 This case demonstrates that the concern criterion is not merely theoretical and hypothetical but reflects a natural tendency, warranting attention and discussion.
Let me describe the case. As is well known, two prominent conceptions (or groups of conceptions) of political polarization have been used by political scientists over the past few decades. The first identifies polarization as the size of the gap between issue–positions (Abramowitz and Saunders Reference Abramowitz Alan and Saunders2008; Fiorina, Abrams, and Pope Reference Fiorina, Abrams and Pope2008; McCarty, Poole, and Rosenthal Reference McCarty, Poole and Rosenthal2006). The second identifies polarization as the depth of resentment or other negative emotions between political camps (Iyengar, Sood, and Lelkes Reference Iyengar, Sood and Lelkes2012; Iyengar et al. Reference Iyengar, Lelkes, Levendusky, Malhotra and Westwood2019; Iyengar and Westwood Reference Iyengar and Westwood2015; Mason Reference Mason2018). For convenience let us call issue-based/ideological conceptions “poliarization1” and affect-based conceptions “polarization2.”
Polarization2 is discussed, for instance, in Iyengar et al. (Reference Iyengar, Lelkes, Levendusky, Malhotra and Westwood2019, 130): “America, we are told, is a divided nation. What does this mean?… While previously polarization was primarily seen only in issue-based terms, a new type of division has emerged in the mass public in recent years: Ordinary Americans increasingly dislike and distrust those from the other party.” They imply that this new type of division that has emerged in the political world warrants a new systematized concept of political polarization in political science.
In line with the Adcock-Collier multilevel framework, different sets of indicators were developed to represent the two types of conceptions. In this case, it should be noticed that the two are totally distinctive: they do not even share a common element (as pointed out by Schedler [Reference Schedler2023], the two do not share “a semantic core”). Thus, although there might be correlation and causality between them, we can imagine a society very polarized under one conception and much less polarized under the other. The question about causal relation and correlation between the two is empirical by nature, but the two alternatives are conceptually distinctive because they represent different ideas of political polarization.
When discussing political polarization, scientists can take into consideration both conceptions, but in practice they tend to choose between them in their day-to-day work.Footnote 10 When political scientists increasingly favor one conception over another in their descriptions and explanations, it becomes a convention. Thus, in our small-scale check, it appears that when political scientists referred to “political polarization” in 1980–82, they actually meant polarization1, but by 2020–22 they increasingly meant polarization2 or both (figure 1). Concepts are evolving and changing, and this evolution is determined by choices within the disciplinary community.
Illustration of the Changing Meaning of Political Polarization
Notes. Based on 50 articles addressing the issue of political polarization: 25 articles from 1980–82 (group 1) and 25 articles from 2020–22 (group 2). In group 1, 25 out of 25 articles understand political polarization as ideological, issue-based polarization. In group 2, 9 of the 25 articles address polarization as purely issue based, 10 focus solely on affective polarization, and 16 on both.
These are the 50 most-cited articles from the relevant years, given that the word “polarization” was part of their title and they were published in one of the top 10 political science journals, according to Google Scholar’s rating. In case there were fewer than 25 matching articles per time section, the ‘top 10’ journals filter was removed, and the next most-cited article was chosen.
Addressing this change in preferences that has prompted an increasing turn among scientists to polarization2, we turn to Gerring’s list of seven basic criteria for conceptualization. Although comprehensive, Gerring’s list, I argue, does not fully account for what underpins this growing preference.
Resonance is “the degree to which a definition makes sense, or is intuitively clear” (Gerring Reference Gerring2012, 117). Polarizations1 and 2 both resonate well and intuitively. Far from idiosyncratic, both are easy to grasp; in particular, both make sense in the political science domain. Polarization1 makes sense because questions of ideology and issue-positions in general are traditionally at the core of political science; nevertheless, emotions have also become respected residents in the discipline, and so in political science polarization2 is far from an unwelcome conception. Nor do consistency and differentiation pose a challenge to either conception, for both can be used consistently (to carry the same meaning through cases) when definitions are provided, and both are distinguishable from other conceptions.Footnote 11
Turning to the criterion of operationalism, both conceptions are conventionally operationalized in political science. In Mason (Reference Mason2018), for instance, polarization1 is operationalized by computing left and right “constraint”Footnote 12 and polarization2 by social distance computations.Footnote 13 Thus, it is not reasonable to prefer one of the conceptions based on this criterion. Adhering to either of the two conceptions, we can operationalize and use the concept of political polarization in scientific descriptions and explanations.
The criterion of causal utility assumes that one of the conceptions might fit better in explanations in political science; that is, one could perform better as an explanans (an attribute that can explain other attributes). Goertz (Reference Goertz2020, 91) refers to this as “causal power” and provides a useful analogy: “We can ask what copper is. One property of copper is its reddish color. However, we would not consider a definition of copper based on its color very adequate for most purposes; much better would be a conceptualization of copper based on its atomic structure… because its atomic structure explains more.”
This might appear to be a justification for why political scientists are turning increasingly to polarization2. However, the two may explain different phenomena (and we need a guide to which of the explained phenomena is more significant). Moreover, when we use a variable to explain a phenomenon, in many cases the very phenomenon (the explanandum) is itself a conceptual artifact. If we use the two polarizations to explain the level of democracy—for instance, as a test of causal power—we need to have deemed the particular conception we are using of democracy as a worthy one beforehand. What would justify that conception of “democracy”? If we base the explanandum’s worthiness on causal utility as well, we encounter pointless endless chains of explanations (and possible circularities with variables changing roles and explaining each other).
At root, another criterion must have been at play alongside causal utility. Political scientists increasingly preferred polarization2 because it was perceived as more relevant per se and not instrumentally only; that is, important for explaining another phenomenon. For polarizations1/2 to be perceived as more or less valid conceptions, they ultimately should make for a worthy explanandum (that scientists care about) and not only well-functioning explanans.
This last point carries us to the final criterion—fecundity—which Gerring (Reference Gerring2012, 124) describes as “reducing the infinite complexity of reality into parsimonious concepts that capture something important—something real—about that reality.” He adds, “I shall call this criterion fecundity, though it might also be referred to as coherence, depth, fruitfulness, illumination, informativeness, insight, natural kinds, power, productivity, richness, or thickness.”
In our case, choosing between competing conceptions boils down to this question: Which of the two (or more) conceptions of political polarization captures the aspects of the political world most significant to the term “political polarization”? The criterion of fecundity hence brings us to the very heart of “conceptual validity.”
However, the criterion of fecundity must be examined more closely, because insight, productivity, richness, and so on, are not the same. It should be noted that fecundity is partly a relation between a conception and the world (richness, something real) and partly a relation between a conception, the world, and us (insight, something important). Researchers who adhere to the classical syntactic–statistical approach would claim that the relationship is always between a conception and the world only, and not about us. Hence, they assume that methods that track statistical regularities can reveal which of the two conceptions is more fecund (see Gerring Reference Gerring2012, 124–25). The semantic–pragmatic approach, in contrast, adheres to the view that fecundity in political science can seldom be revealed by mere empirical knowledge about the world. It usually demands that a conception be regarded as convincing and insightful for us. According to this approach, there is in many cases a residual that no statistical data can determine (Max Weber refers to this as “cultural significance”).Footnote 14
Returning to the case of political polarization, it seems indeed that political scientists took polarization2 to be more “fecund”—in the sense of more significant for us—and hence increasingly valid. But the hypothesis highlighted here is that what has made polarization2 more significant for us is precisely its being worrisome. The scientific community started to “care” more about polarization2 because it increasingly became a matter of concern.Footnote 15 If indeed this has been the decisive criterion at play, we should be fully transparent and ask bluntly whether “concern” is a plausible criterion and in which specific cases.
The concern criterion prioritizes those conceptions that best reflect our worries about the world (other things being equal). Referring to Goertz’s earlier example, if copper had been a matter of concern per se, would it not be relevant for its definition to ask, What is it about copper that makes it a matter of concern? Suppose that it was its very specific and unique reddish color? In this case, wouldn’t it make sense to define copper by its specific red color, instead of its atomic structure? This is the basic idea that is highlighted here, which is absent in previous lists of criteria.
Taking the Concern Criterion Seriously
Theoretical Significance
Famously, with some concepts in political science, two (or more) competing conceptions can be insightful and convincing, each on its own terms; hence, the cases of “essentially contested concepts” (Gallie Reference Gallie1956). In some cases, “fecundity” and “insightfulness” are based on normative convictions. Indeed, in political science, we often as a community accept this, because other conceptions, implicitly or explicitly, are based on values and normative convictions; that is, they are based on how democracy, populism, equality, and freedom ought to be understood. This seemingly trivial point is at best underdeveloped and at worst played down or even suppressed in discussions of concept-validity criteria. The reason is probably that social and political scientists tend to view normative considerations as making things even less determinate and messier and therefore as hardly helpful when judging between alternative conceptions. Many consider normative convictions as a part of the problem, because they contribute to what makes the controversy seemingly endless.
Nevertheless, and as I highlight, normative discussions—which are sometimes inevitable in conceptual validity contestations—should not be regarded as absolutely open-ended and indeterminate. Scientists are better off being aware of the options when pondering normative presumptions. They should neither avoid nor discard or, so it is suggested, the possibility of better and worse normative arguments. One way for political scientists to better ground conceptions, from a normative point of view, is by reference to political philosophy/theory. Political theory is relevant because the main mandate of this branch of political science is, after all, conceptualizing based on well-grounded normative considerations (Gaus Reference Gaus2000). See, for example, suggestions made by Crasnow (Reference Crasnow2021) regarding democracy and Cohen Kaminitz (Reference Cappelen2023) regarding social progress: both urge political scientists to consider political theory when judging between competing conceptions.
However, not every concept that serves political scientists is thoroughly dealt with in political theory. Moreover, some concepts developed in political theory may be regarded as unfit for political science purposes (based on criteria such as operationalization, causal utility, and so forth). Also, within political theory there is not always agreement about competing conceptions. But whereas political philosophers and theorists sometimes avoid choices, highlighting—and even celebrating—their disagreements, the choice between conceptions by political scientists is a necessary component of ordinary, everyday scientific practice.
Because political philosophy is insufficient in some cases, the concern criterion can be of aid. An important attribute that makes it especially appealing when coping with contestation is its tendency to navigate debates into less harshly contested territories. This might work even in cases when the contestation is normative by essence. Arguably, political scientists tend to reach agreements about concerns more easily than they reach agreements about ideals. This very same intuition led Karl Popper, for instance, when discussing the design of political institutions in his book, The Open Society and Its Enemies (first published in 1945), to distinguish between “utopian” and “piecemeal” engineers:
The piecemeal engineer will … adopt the method of searching for, and fighting against, the greatest and most urgent evils of society, rather than searching for, and fighting for, its greatest ultimate good.
In favor of his method, the piecemeal engineer can claim that a systematic fight against suffering and injustice and war is more likely to be supported by the approval and agreement of a great number of people than the fight for the establishment of some ideal. (Popper [1945] Reference Popper1994, 148–49)
Judith Shklar’s explication of liberalism follows the same intuition. Her “liberalism of fear’ is a conception that focuses on the avoidance of evil, cruelty, and intimidation. Similarly to Popper, “putting cruelty first” is explained and justified not only through its inherent significance but also due to its tendency to overcome disputes: “Because the fear of systematic cruelty is so universal, moral claims based on its prohibition have an immediate appeal and can gain recognition without much argument” (Shklar Reference Shklar and Rosenblum1989, 30) The concern criterion imports this rationale into the process of choosing between contested conceptions. Such a criterion at the very least has the potential to mitigate contestations.
Pragmatic Implications and Further Examples
What would be the pragmatic implications of accepting the concern criterion? As exemplified by the political polarization case, in some cases this criterion naturally motivates scientists without any explicit declaration. In other cases, it plays an explicit role (e.g., Schedler Reference Schedler2023) along with other criteria. In addition to making this path explicit in cases of concept formation and when deciding between competing conceptions, the concern criterion could play a role when ascribing logical structure to the different elements of a concept (see fn 5).
The concept of democracy, for instance, is usually conceptualized and measured by breaking it down into a few elements. Coppedge et al. (Reference Coppedge, Gerring, Altman, Bernhard, Fish, Hicken and Teorell2011), for example, disaggregate “democracy” into six conventional aspects: electoral, liberal, majoritarian, participatory, deliberative, and egalitarian. They explicitly avoid the advocacy of a particular systematized concept and the elucidation of the weights and aggregation scheme, suggesting instead, as a prerequisite, developing the most comprehensive and valid set of indicators for each aspect. But when it comes down to the question of how to aggregate these elements, it is far from obvious on what grounds it should be done.
Ascribing equal weights to the different elements of a concept is but one option, which sometimes (allegedly) serves social scientists as a way of avoiding judgments and decisions. But unless an equal-weight scheme is backed by a well-grounded belief that the elements are equally fit or valid in representing the concept, it should be deemed an arbitrary decision. Fleurbaey and Blanchet (Reference Fleurbaey and Blanchet2013, 240) refer to a similar lacuna in welfare indexes, namely the absence of a “theory of weights.” Gerring’s list can serve scientists in assessing the relative fitness or validity of elements and ascribing structure and weights in accordance. The concern criterion may complement such a process.
Thus, when scientists adhere to the semantic–pragmatic approach, they are required to judge, to decide, to prioritize these six conceptions of democracy or part of the six. The concern criterion will lead them to ask, in addition to other questions, to what extent each of the six aspects addresses what is worrying about (an absence of) democracy in the political world. According to the criterion, the aspect of democracy that most addresses our greatest concerns should be ascribed a heavier weight, less compensability by other aspects, or both.Footnote 16
The same procedure is required with the explication and structuring of other multidimensional concepts such as political corruption, poverty, political violence, and so forth. To explicate their meanings and to structure them validly, thereby complementing other considerations, this question should be asked: What is it about them exactly that makes them a matter of concern? The implication of this suggestion is far-reaching. Adhering to decisions about the structure of concepts can be regarded as a way of calibrating our measurement instruments. As nicely put by Schedler (Reference Schedler2012, 24), “in social science we don’t build instruments, we knit rules.” These rules result in different measurements of the same world phenomena, and our decisions regarding conceptions and structures determine the sensitivities of those measurements. Hence, when we accept a concern criterion and assign heavier weights in an aggregation scheme to attributes that we see as matters of greater concern, we craft the measurement in accordance with our worries and sensitivities.
Importantly, our concerns about an aspect of a concept may grow stronger or become weaker in accordance with two different kinds of change: (1) a change in the world, such as a deterioration in liberal attributes, a worsening in resentment between political camps, and so on, or (2) a change in our normative convictions, a change of values when attributes become more concerning regardless of their prevalence in the world. Advocating for the concern criterion is advocating for the relevance of both changes to conceptualization, so long as they lead to a greater concern.
The Concern Criterion: Limitations and Caveats
Social and political scientists may raise various caveats concerning the concern criterion. A first line of issues arises with the question of why concepts (the “fact-finding containers”) should be determined according to contingent normative convictions at all. Is it legitimate that concepts and measurements change in response to a change in values and to a change in the world? Does that not put standardization and stability of concepts and measurements at risk or sacrifices the “objectivity” of measurements? A second line of issues concerns legitimacy and trust: Who is to decide what is troublesome and what is not? By accepting the concern criterion, do we put trust in political scientific measurements at risk? What if we fail to reach an agreement about concerns? A different (and opposite) line of criticism suggests that this criterion is too minimal or too conservative, because it undermines the role of ideals in conceptualization.
A response to the first line of issues is linked to wider discussions in the literature in the philosophy of science that shake off the shackles of strict positivism and the fact/value dichotomy in social science. From the point of view of the philosophy of science, combining values and facts in social science is deemed almost trivial. The myth of a social science that builds on facts alone is considered not only untenable but also unwelcome. Distinguishing epistemic from non-epistemic values, philosophers point to both as inherent to social science (and sometimes also to the “pure” sciences).Footnote 17 See, for instance the fact/value entanglement argument made by Putnam (Reference Putnam2002); see also Anderson (Reference Anderson1995; Reference Anderson, Gärdenfors, Woleński and Kijania-Placek2002); Douglas (Reference Douglas2009; Reference Douglas2016), and Kitcher (Reference Kitcher2001).
Alexandrova (Reference Alexandrova2018) points to a related acknowledgment that many claims in social (and medical) science are “mixed claims”; that is, they are partly empirical and partly normative. Being mixed, Alexandrova points out, is by no means at odds with being scientific or objective (on objectivity as a concept that does not necessarily exclude non-epistemic values, see Alexandrova Reference Alexandrova2018, 433; Crasnow Reference Crasnow2021; Douglas Reference Douglas2004; Reference Douglas2009; Kitcher Reference Kitcher2001).
Distinguishing the role of non-epistemic values in the pure sciences from their role in the social sciences, John Dupré (Reference Dupré, Kincaid, Dupré and Wylie2007, 31–32) nicely adds, “What I want to say about physics is that if most of physics is value free, it is not because physics is science but because most of physics simply doesn’t matter to us.… The statement that electrons have negative charge is … value free in a quite banal sense: It has no bearing on anything we care about.”
Philosophers, moreover, highlight questions of goals, relevance, and significance as important criteria in choosing among alternative scientific descriptions of the same reality. “Truth,” although an important goal for scientific inquiry, is neither a sufficient nor a fully achievable goal. Both Anderson (Reference Anderson1995; Reference Anderson, Gärdenfors, Woleński and Kijania-Placek2002) and Kitcher (Reference Kitcher2001) highlight just how “significance,” determined by non-epistemic values, makes for another complementary criterion for sciences (see also Weber Reference Weber, Shils and Finch1949, fn 14). It is indeed, as in Dupré’s quote, even more obvious in the social sciences. The concern criterion is but one manifestation of significance. It is a specific consideration of why we find a feature significant/relevant.
Gerring and Yesnowitz (Reference Gerring and Yesnowitz2006) express related points in the context of political science. In their account, not only is there a certain mix between the normative and empirical in political science but it is also a necessary mix because it provides political science with relevance. As long as political science is “practically oriented,” it must be tied into “some broader telos”: “Normative theorizing must deal in facts just as empirical work must deal in values; they do not inhabit different worlds” (108). The important question is, therefore, how best to combine facts and (non-epistemic) values in a constructive and objective manner, not how to avoid such a mixture.
The sense of concern can change, however, not only because of changes in values but also when the social and political world changes. How legitimate (and constructive) is it to change the instrument of measurement when the object is changing? Well, in a sense concept formation always reacts to the world and to empirical phenomena. Concepts are created and abandoned according to a changing physical and social world. For example, when the phenomenon of climate change emerged in the real physical world, this created a need to construct new concepts and measurements to describe and address the change. Likewise, it was only when economic growth became an actual phenomenon of economic reality in the nineteenth century that its conception came to be developed (with the need to come up with a measurement becoming pressing in the circumstances of depression and war in the twentieth century).
The same goes for concepts’ structure. If we acknowledge the need to craft new concepts under specific circumstances and concern, there is no reason why we should not also accept the need to fit concepts’ structures to a changing world and concerns; otherwise, we condemn them as less fit and less valid.
Such changes in concepts and measurements, in reaction either to changes in the world or to changing normative convictions, should obviously not happen too often: too frequent changes would harm the stability and standardization of concepts and measurements. This, however, holds for every criterion of concept formation/structuring and is not unique to concern as a criterion; see, for instance, the warning against conceptual instability in Schedler (Reference Schedler, Badie, Berg-Schlosser and Morlino2011, 374). With concepts, the pace of evolution is important. For concept validity, so it seems, there is a need, on the one hand, for stability, high standardization, and the grounding of conventions and, on the other hand, for some openness to revisions and amelioration.
The second related hesitation concerns issues of legitimacy and trust, which are also well known as a problem regarding conceptualization far beyond the concern criterion. When concepts and measurements in political science are affected by scientists’ concerns (their values and normative interpretation of the world), this may weaken the trust of policy makers and the public in their research processes and products. Thus, let us consider a few alternative tactics. One option is withdrawing to the position of choosing not to choose. As in the example given by Coppedge et al. (Reference Coppedge, Gerring, Altman, Bernhard, Fish, Hicken and Teorell2011), scientists sometimes prefer to leave normative judgments (and hence decisions about structure) to others. However, whereas in some specific cases scientists can stay (temporarily) agnostic, when proceeding to the phase of scientific operations of providing descriptions and explanations, choices are necessary.
Sometimes the suggestion is made that, whenever disagreement appears, scientists should let go of one concept and break it down into several. An example would be a concept of political polarization that breaks into two (or more!) separate concepts; for instance, “issue-based polarization” and “affective polarization.” Cappelen (Reference Cappelen, Burgess, Cappelen and Plunkett2020, 142) describes this as a “lexical expansion.” Sometimes this kind of abandonment of the original single concept is required and constructive.
However, under this description, the formation and acceptance of the new concept (polarization2) were also due to the concern criterion. Moreover, in many cases it is not so easy to let go of one background concept because we do care about the “key term” itself—for instance, political polarization or democracy—that has both reputation and effect. It was not in vain, for example, that political scientists did not readily let go of “democracy” as a background concept, as Cappelen (Reference Cappelen2023) advised them to do. After all, and as Cappelen (Reference Cappelen, Burgess, Cappelen and Plunkett2020, 145) explains, such high-reputation terms often play roles as “markers of topic continuity” and serve as “anchor points for amelioration.”Footnote 18 In other words, in some cases, conception validity—as an account of the specific term—matters to political scientists.
Recent discussions highlight another strategy—referring to evaluative decisions about concepts as a joint task for scientists, practitioners, the public and other stakeholders—thereby avoiding trust and legitimacy difficulties (Alexandrova and Fabian Reference Alexandrova and Fabian2022; de Shalit Reference De Shalit2020; Duque, Tal and Barbic Reference Duque, Tal and Barbic2024; Sarkar Reference Sarkar2019; Thoma Reference Thoma2024). Although these are valuable suggestions in many contexts, the benefits and costs of adopting such solutions within political science should be carefully considered. As pointed out by Schroeder (Reference Schroeder2017), these solutions could carry both practical and ethical-normative burdens for scientists.Footnote 19 Scholars have pointed out that science is not always supposed to be a democratic task in the same sense that public policy decisions should be democratic (Douglas Reference Douglas2016; Gerring Reference Gerring2012).Footnote 20
Considering how and in what cases it is justified to integrate external inputs into political science is an important task and one that goes beyond the scope of this article (Wolff [Reference Wolff2020] discusses such considerations in political theory). For justifying the concern criterion, however, it is sufficient to point out that when other external interested parties are participating, they also may use the concern criterion when deliberating about concepts.
In cases that the scientific community must decide, other tactics can yet be used to guard against problems of legitimacy and trust. Alexandrova (Reference Alexandrova2018, 437–38), for instance, suggests that being explicit and transparent may guard against “inattention” and “imposition”; see also a similar point in Gerring and Yesnowitz (Reference Gerring and Yesnowitz2006, 108). Inattention occurs when scientists engage in mixed science and fail to notice the value judgments that they are making, and imposition is when scientists act as if the value judgment is obvious. When scientists acknowledge explicitly the contingency of the evaluative considerations underpinning concepts and are explicit about them, they are thereby more obligated to present their logic and reasons and so open a path to deliberation within the scientific community— an essential task in building legitimacy and trust and in avoiding idiosyncrasy and subjectivism (Schedler Reference Schedler2012). A pluralistic community ensures that the discussion is not monolithic and may further strengthen legitimacy and trust (Douglas Reference Douglas2016, 27–29).
Importantly, whether in processes of concepts’ contestation and deliberation within the scientific community or when other parties’ views are integrated, the particular strengths of the criterion are the same: it has the potential to better focus ongoing discussions about conceptualization and better direct them to agreements—features that may contribute not only to conceptual validity but also to legitimacy. In particular, when hitting the ethical-normative surface in conceptualization, it is assumed that using such a criterion may lead to reaching a wider consensus than would be achieved through discussions of common positive ideals (as pointed out by Popper and Shklar). This is especially true in a worrisome political world in which hazards become intimidating and transparent.
Finally, one might criticize the concern criterion as too conservative, in the sense that its rationale is based on damage prevention instead of ideals promotion. The response to this is twofold. First, the concern criterion is not relevant for every concept in the same way. Obviously, it is not very relevant in cases of concepts, the purpose of which is unrelated to political hazards.Footnote 21 Still, many concepts have such a purpose underpinning them, such as political corruption, populism, and poverty. In these cases especially, scientists are advised to explicitly employ the concern criterion.
To this, we should add the actual state of the social and political world. Although in some historical contexts, structuring would benefit from reasoning about ideals, in other contexts, particularly in worrisome times, the very same concept structuring might benefit (in concept-validity terms) more from addressing concerns. When politics is regressing, making the concepts and hence the measurement instruments more sensitive to the worrisome aspects might result in a better, more constructive science. Thus, one important difference between ideals and concern is that although ideals may stay stable, concern is dynamic and more context dependent.
Concluding Remarks
In the literature on concept validity in political science, criteria for good conceptions usually exclude (ethical) normative considerations. The problem with this lacuna is that conceptualization processes may end at the fundamental problem of conceptualization, namely an extreme indeterminacy of systematized concepts. Although in political theory such a situation is welcomed, in political science it may lead to confusion, inefficiency, and the use of invalid concepts. Many of the concepts in political science are not fully analogous to Goertz’s “copper” or Dupré’s “electrons.” Copper-ness is not a matter of concern or approval, and therefore criteria for good concepts in the natural sciences should not be fully parallel to criteria for good concepts in the social sciences. A way out is acknowledging this difference and then coming up with complementary criteria.
A concern criterion is constructive precisely for this purpose because, on the one hand, it does not avoid normative judgments (thus, it can prevent scientists from “inattention”), but on the other hand, it is directed toward reaching well-reasoned consensus (especially compared to contestation about ideals). It could be that there are additional criteria, but a concern criterion is central because it pertains to this wider common ground and because it fits a worrisome political world, as is ours.
To guard against abuse of such a criterion, it is advised that it be used with caution, with other criteria in mind, and not as a privileged criterion. As with other criteria, it does not provide answers but only the grounds for contestation.
Data replication
Data replication sets are available in Harvard Dataverse at: https://doi.org/10.7910/DVN/6ETXRZ.
Acknowledgments
I would like to thank Ittay Nissan-Rozen, Preston Werner, and Charles Lesch for their constructive comments on this article, and to Gal Arbitman for research assistance. I also thank the journal’s two anonymous reviewers for their helpful feedback. A version of this paper was presented at the PPE Annual Meeting in London in July 2025.
Open access
