Construct proliferation has long been viewed as a problem in psychology. In 1937, Gordon Allport observed an abundance of motivational constructs, calling it “the problem of motivation” (Allport, Reference Allport1937, p. 110). In 1958, Robert Guion hosted a symposium on the many meanings of industrial “morale” (Guion, Reference Guion1958). Today, analyses of the APA’s PsycTest database have found over 38,000 constructs with 43,000+ unique measures—with over 80% only used once or twice (Anvari et al., Reference Anvari, Alsalti, Oehler, Marion, Hussey, Elson and Arslan2025; Elson et al., Reference Elson, Hussey, Alsalti and Arslan2023). Although new constructs often signal the health of a scientific field (Iliescu et al., Reference Iliescu, Greiff, Ziegler, Nye, Geisinger, Sellbom, Samuel and Saklofske2024), they can also fragment research landscapes, prevent standard comparisons, and otherwise make it harder to do good science (Flake et al., Reference Flake, Davidson, Wong and Pek2022; Meehl, Reference Meehl1992; Yarkoni, Reference Yarkoni2022).
Further complicating matters, Bowling and colleagues (Reference Bowling, Sessa, Shaffer and Banks2026) recommend that researchers identifying measures of potentially redundant constructs collect data and then analyze empirical covariation (see also Shaffer et al., Reference Shaffer, DeGeest and Li2016). This practice is resource intensive and becomes increasingly prohibitive in terms of cost and data quality as the number of constructs and items increases—it doesn’t scale! As the number of constructs and measures increases, it becomes increasingly challenging to explore construct proliferation (Larsen et al., Reference Larsen, Nevo and Rich2008; Wulff & Mata, Reference Wulff and Mata2025). In this context, calling for a construct moratorium sounds reasonable.
We believe, however, that advances in natural language processing, particularly with transformer architectures, offer tools uniquely designed to structure large amounts of previously unstructured content (Vaswani et al., Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017). Although the traditional approaches require new data collection for every construct comparison, natural language approaches can evaluate the conceptual similarity of hundreds or thousands of constructs using only the content that already exists: the definitions, scale labels, and item wordings that compose our measures. From a content sampling perspective, each new measure represents an attempt to sample from the universe of possible psychological content (Highhouse, Reference Highhouse2009; Loevinger, Reference Loevinger1957). Instead of stopping construct development, therefore, we should be using natural language to refine the construct space. In other words, the information needed to refine constructs is already there; we just need to use it.
Sampling psychological content with natural language
Although it is unclear whether psychological experiences are internally represented as natural language (Mandelbaum et al., Reference Mandelbaum, Dunham, Feiman, Firestone, Green, Harris, Kibbe, Kurdi, Mylopoulos, Shepherd, Wellwood, Porot and Quilty-Dunn2022), our internal experiences are ultimately understood, shared, and then studied as such (Boyd & Schwartz, Reference Boyd and Schwartz2021). Psychometrics uses natural language at every stage: conceptualizing new phenomena, delineating content areas, and sampling the universe of content with representative items (Bringmann et al., Reference Bringmann, Elmer and Eronen2022; Highhouse, Reference Highhouse2009; Podsakoff et al., Reference Podsakoff, MacKenzie and Podsakoff2016). The language of our concepts and measures therefore occupies a unique position, sitting between the ontological (i.e., constructs as they are) and epistemological (i.e., constructs as empirical models) levels of validity (Borsboom et al., Reference Borsboom, Mellenbergh and Van Heerden2004; Messick, Reference Messick1981)—what we might call the phenomenological level (i.e., constructs as we experience them). Items, at this level, can function as natural language indicators of the conceptual areas they were designed to sample (Arnulf et al., Reference Arnulf, Olsson and Nimon2024; Bagozzi & Edwards, Reference Bagozzi and Edwards1998).
The idea that language maps psychological content is not new (e.g., Allport, Reference Allport1937; Galton, Reference Galton1884; Pennebaker & King, Reference Pennebaker and King1999), and modern applications have broadly accepted this intuition (e.g., Yarkoni, Reference Yarkoni2010).Footnote 1 Beginning with Larsen and colleagues (Reference Larsen, Nevo and Rich2008), researchers have used semantic similarity, with latent semantic analysis (LSA), to predict scale validity, showing that linguistic overlap between survey items predicts observed associations (Arnulf et al., Reference Arnulf, Larsen, Martinsen and Bong2014, Reference Arnulf, Olsson and Nimon2024). This has been directly applied to construct proliferation: detecting jingle and jangle fallacies (Larsen & Bong, Reference Larsen and Bong2016); measuring overlap between engagement and satisfaction (Nimon et al., Reference Nimon, Shuck and Zigarmi2016); and building large-scale tools like the Semantic Scale Network (Rosenbusch et al., Reference Rosenbusch, Wanders and Pit2020).
Around this time, the broader NLP field underwent a paradigm shift with transformer architectures (Vaswani et al., Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017), and psychological applications gradually followed, replacing the LSA previously used with transformer-based embeddings that better captured contextual semantics. At the current frontier, Hommel and colleagues (Reference Hommel, Külpmann and Arslan2025) updated the Semantic Scale Network approach with embeddings and introduced the Synthetic Nomological Net; Guenole and colleagues (Reference Guenole, Samo and Sun2024, Reference Guenole, D’Urso, Samo, Sun and Haslbeck2025) developed an embedding-based pseudofactor analysis to recover latent structure directly from item language; and Wulff and Mata (Reference Wulff and Mata2025) used embeddings to systematically map taxonomic incommensurability across constructs.
Some practical recommendations for using natural language to refine construct space
In practice, there are a few basic ways to use natural language to explore construct space at multiple levels of analysis depending on your goals:
-
1. Situating a new construct. If you have an idea for a new construct but want to check how similar it is to existing constructs (or identify dis/similar constructs to include in a nomological network) before investing in validation, you can write a description or definition of the construct and input it into the Semantic Scale Network (Rosenbusch et al., Reference Rosenbusch, Wanders and Pit2020) or the Synthetic Nomological Net (Hommel et al., Reference Hommel, Külpmann and Arslan2025). This allows you to estimate your novel construct’s similarity to existing constructs and identify other measures to consider when refining your construct space or to include when establishing convergent and discriminant validity (Meehl, Reference Meehl1992).
-
2. Comparing between-construct space. If you are examining potential overlap, redundancy, or taxonomic incommensurability (i.e., jingle and jangle; Larsen & Bong, Reference Larsen and Bong2016) between constructs, then embedding-based semantic similarity offers a scalable alternative to empirical covariation. Following Wulff and Mata (Reference Wulff and Mata2025), you can encode scale labels and item content, compute pairwise semantic similarity, and systematically identify jingle (i.e., high label similarity, low item similarity) or jangle fallacies (i.e., low label similarity, high item similarity) to investigate.
-
3. Examining within-construct space. If you want to look within a construct space, when there is a particularly ambiguous construct or multiple measures claim to capture the same thing, a more granular analysis can identify specific areas of content convergence (or divergence) within a construct space. One approach we propose integrates content overlap (whether measures sample similar conceptual domains) with semantic similarity (whether item language is similar) yielding the 2 × 2 framework shown in Table 1. This can reveal which content areas within a construct space are a source of jingle or jangle and whether the issue lies in content sampling, natural language operationalizations, or both, providing a diagnostic.
Content and Semantic Overlap Framework for Within-Construct Overlap
Table 1 Long description
The table presents a framework for analyzing within-construct overlap by examining the relationship between content area overlap and semantic similarity. It is divided into four quadrants. The top-left quadrant, labeled 'Jungle risk,' indicates situations with high semantic similarity but low content area overlap, described as sampling different content areas with similar item language. The top-right quadrant, labeled 'Convergence,' represents high semantic similarity and high content area overlap, described as sampling similar content areas with similar language. The bottom-left quadrant, labeled 'Distinct,' shows low semantic similarity and low content area overlap, described as sampling unique areas with different language. The bottom-right quadrant, labeled 'Jingle risk,' indicates low semantic similarity but high content area overlap, described as sampling similar content areas with different language. Each quadrant provides a diagnostic tool for identifying potential issues in construct space.
Each of these approaches benefits from large, open-source repositories of psychological content. As starting points, the APA’s PsycTests database (e.g., used by Hommel et al., Reference Hommel, Külpmann and Arslan2025), the International Personality Item Pool (IPIP) content (e.g., used by Wulff & Mata, Reference Wulff and Mata2025), or domain-specific inventories like the judgment and decision-making (JDM) content from the Decision-Making Individual Differences Inventory (DMIDI; Appelt et al., Reference Appelt, Milch, Handgraaf and Weber2011) provide rich pools of psychological content for exploration. To illustrate the accessibility of these methods, as shown in Figure 1, we collected the 200+ JDM measures cataloged in the DMIDI, encoded their available items as embeddings using open-source sentence transformers, and projected them to two-dimensional embedding space with UMAP reduction. The resulting visual shows clusters of linguistically similar measures, organized by JDM categories, and potential areas of redundancy within JDM measurement. We would expect that category (color) groups cluster more closely together, as they should share more similar content. Any points that are overlapping may share redundant content. Similarly, any unexpected colors in a larger cluster may need closer inspection. As we can see, on the right side of the figure, risk attitude measures do seem to cluster together, yet there are several decision-making styles and one personality measure that are closely grouped with risk attitudes. On the left side, we can see a mix of personality and risk attitude categories, with a small cluster of motivation measures toward the top. This is only a brief illustration, but there are a number of unexpected groupings to explore in more detail (see Figure 1).
Semantic Embeddings of Decision-Making Individual Difference Inventory (DMIDI) Measure Categories
Note. Embeddings were encoded with an open-source all-mpnet-base-v2 sentence transformer and projected with a UMAP reduction. Measures were chosen to label by identifying the most central and extreme category members based on UMAP dimension centroid distances.
Figure 1 Long description
A scatter plot visualizes semantic embeddings of decision-making individual difference inventory measure categories. The plot features dozens of data points, each representing a different measure. The x-axis and y-axis are labeled as UMAP Dimension 1 and UMAP Dimension 2, respectively. Data points are color-coded into categories: Cognitive Ability, Motivation, Decision, Personality, Risk Attitude, and Miscellaneous. Notable clusters include Big Five Inventory, Iowa Gambling Task, and Cognitive Reflection Test. The plot shows various measures labeled for clarity, indicating their central and extreme positions based on UMAP dimension centroid distances. Embeddings were encoded with an open-source all-mpnet-base-v2 sentence transformer and projected with a UMAP reduction.
Bowling et al. call for a moratorium until steps are taken to address construct proliferation. We outline several steps based in natural language analysis: situating new constructs within semantic nomological networks, comparing existing constructs with embedding-based semantic similarity (Figure 1), and examining within-construct heterogeneity through an integrated content-semantic overlap framework (Table 1). We also propose a systematic effort to take advantage of the natural language of psychological measurement: a large-scale, open-source inventory of psychological content, similar to the APA’s PsycTests, IPIP, and DMIDI. So instead of reading lips to say “no new constructs,” I-O psychology should, in fact, continue reading people’s lips, using natural language to refine our understanding of psychological constructs.
Open access
