2. Concentration

In this section we subsume the comparatively high level of self-reference and the strongly hierarchical character of economics under the heading of concentration to signify that important authors, contributions, departments, and outlets are (more) strongly concentrated in terms of their spatial distribution and their personal as well as professional relations. We argue that one of the main roots of concentration in economics is the highly stratified tertiary education system comprised of a very small number of high-profile schools in economics, which in turn have a tight grip on the most important outlets and professional associations.^{Footnote 2} The hierarchical structure within the economic discipline has been confirmed empirically in various ways. Medoff, for instance, finds evidence of a “positive institutional Matthew effect” (Medoff Reference Medoff2006, 485) for Harvard University and the University of Chicago leading to a disproportionate amount of peer recognition devoted to contributions by economists affiliated with these institutions. On a more general level, Baghestanian and Popov find a strong association between the reputation of an economists PhD-granting institution and the probability of publishing in a top economic journal^{Footnote 3} which they call the “Alma mater”-effect.

Another finding on “concentration” in economics is due to Hodgson and Rothman who analyze the institutional background and affiliation of the editors and authors of a set of thirty “top” economic journals. They identify a strong dominance of institutions located in the U.S., and, within that subset, a dominance of a small group of elite universities. For instance, in 1995 more than half (54 percent) of the authors of articles published in these “top thirty” journals obtained their PhD degrees at one of twelve prestigious U.S. universities,^{Footnote 4} while at the same time more than one fifth (21 percent) of authors in said journals were also affiliated with one of these universities (Hodgson & Rothman Reference Hodgson and Rothman1999).

A partial replication of the analysis of Hodgson & Rothmann (Reference Hodgson and Rothman1999) with data from 2014 suggests that this level of “institutional oligopoly” has remained fairly constant over time: Just looking at the subsample consisting of three major journals, the Quarterly Journal of Economics (QJE), the American Economic Review (AER)^{Footnote 5} and the Journal of Political Economy (JPE) reveals that a majority of authors (55 percent) publishing in these outlets received their PhD degrees at one of the twelve universities mentioned above. In 2014, more than one third (38 percent) of these authors were affiliated with one of these universities at the time of publication (see also Wu Reference Wu2007).

Similar tendencies of concentration can also be found in large-scale analyses of the economic literature. Laband (Reference Laband2013), for instance, analyzes the representation of journals in the reference sections of the 409 most cited economics papers published between 2001-2005: more than half (55 percent) of the references cited by these top-papers came from a group of only twenty journals. Even more strikingly, the proportion of citations devoted to the three journals mentioned above – QJE, JPE and AER – is about one fifth (20.6 percent) of the total references (cf. Laband Reference Laband2013, F250-F251). In light of these observations, it comes as no surprise that a recent analysis of the development of economic “top-journals,” which has been published in the Journal of Economic Literature, was limited to the study of only five journals, adding Econometrica and the Review of Economic Studies to the set of three journals mentioned above (see Card & DellaVigna Reference Card and DellaVigna2013).

Our empirical application takes a closer look at the “Big 5,” i.e. the five core journals covered by Card & DellaVigna (Reference Card and DellaVigna2013), to gain potential insights on tendencies of concentration within economic discourse similar to those already observed in the context of editor- and authorships in top-journals (Hodgson & Rothman Reference Hodgson and Rothman1999) or the recruitment of officers for academic associations (Fourcade et al. Reference Fourcade, Ollion and Algan2015). Specifically, we measure the concentration of attention within top-journals by assessing the degree of self-referentiality within this group of journals to compare the relative openness of top-journals across fields. Figure 1 plots the share of journal self-citations as well as the share of citation flows originating from within the “Big 5” relative to total citations in these journals for the period 2009-2013. Figure 1 conveys our first finding: more than a quarter of all sources cited by the top five economic journals originate are self-referential – they stem from the very same set of journals.

Figure 1.

The intra-group citation behavior of five core economic journals. (The authors’ own calculation based on data from Clarivate Analytics.).

In addition to this key observation – that one out of four citations made in this journal sample also originates from within this sample – it can also be inferred from Figure 1, that the average share of self-citations within a top-journal plus the share of citations imported from its four “best buddies” is fairly stable across the top five journals with individual values ranging from 23.8 percent (QJE) to 31.7 percent (EMA).

When comparing the citation pattern found across economics’ top journals to the degree of self-reference in top journals located in other fields – e.g. the “top five” journals in psychology, sociology, political science, and physics (fig. 2) – there is the difficulty of selecting appropriate sets of top journals for these fields to advance a sensible comparison across disciplines. To enhance comparability of the respective “top” samples in each discipline we first harmonized our selection criteria across all fields under study. Specifically, we selected every journal that was ranked in the top 20 of Clarivate Analytics’ ^{Footnote 6} annually published Journal Citation Reports (JCR) for ten consecutive years (2006-2015) to account for the relative long-term position of outlets – a condition inspired by the ranking patterns in economics, where the “Big 5” are constantly within the Top 20 of journals as ranked by the JCR. In the case that more than five journals fit those criteria, the five journals with the highest average rank were selected.

Figure 2.

An interdisciplinary comparison on concentration – the intra-group citation behavior of five core journals in (a) psychology (multidisciplinary), (b) sociology, (c) political science, and (d) physics (multidisciplinary). (The authors’ own calculation based on data from Clarivate Analytics.).

In comparison with fig. 1, the results shown in fig. 2 reveal two remarkable findings. Firstly, in the other social science disciplines considered here, the degree of concentration measured as the share of journal self-citations plus the citation flow within top journals is – with average values ranging between 7.9 percent and 12.4 percent of total citations – pending between a third and a half of what we found for economics (27.5 percent). This finding reinforces the diagnosis that economics exhibits a much more pronounced hierarchy, or “pecking-order” (Leijonhufvud Reference Leijonhufvud1973, 328), compared to other social science disciplines. Secondly, this relative high degree of concentration in economics also exceeds the degree of concentration measured within our sample of (multidisciplinary) physics journals (20.3 percent). Hence, although the difference between economics and physics as its conceptual archetype is less pronounced than the one between economics and the other social sciences, there remains a substantial gap even in comparison to the much different and often more narrow discourses in the realm of the natural sciences.

In response to the finding on concentration in physics documented in fig. 2, we acquired data for analyzing a second example in the realm of physics, but, this time, focusing on top journals in very specific research areas to assess the intuition that economics is similar to specialized, instead of general, discourses in physics. In doing so, we found a striking similarity between our novel sample from physics consisting of specifically chosen field journals (Physical Review A-E), see fig. 3, and economics. Although the network structures differ markedly between these two cases – the Physical Reviews are huge journals with an average publication output of more than 3,000 papers per year focusing on very different subfields, the degree of concentration is nearly as high (27.3 percent) as in the economics sample. While the stronger content-based separation of the respective journals into subfields (designated with letters A-E^{Footnote 7}) provides an explanation for a much higher share of self-citations, we also find that the average amount of self-citations within these journals (24.1 percent) is eventually rather close to the average share of attention the “Big 5” in economics devote to themselves (27.5 percent). In sum, the top five journals in economics taken together behave rather similarly to specific field journals within physics in terms of discursive concentration.

Figure 3.

Citation patterns in specialized physics journals – the intra-group citation behavior of five core field journals in physics. (The authors’ own calculation is based on data from Clarivate Analytics.).

The result that the “Big 5” in economics behave similarly to more specialized subfields in physics and exhibit correspondingly high rates of concentration is especially striking in the face of an overall decrease in the concentration of cited sources within the academic literature over the past decades (Larivière et al. Reference Larivière, Gingras and Archambault2009). Beside this anachronism, it is quite remarkable that a multi-faceted discipline such as economics, which deals with mathematical, game-theoretical, sociological, historical, philosophical, and psychological components, behaves in the same concentrated manner as, say, Physical Review B, which focuses on condensed matter and material physics. The fact that the relative citation patterns of top multidisciplinary journals in physics (as documented in the lower right panel of fig. 2) shows much less regularity, gives an indication for the tight conceptual and theoretical commitments established in contemporary economics.

While these findings are interesting in themselves, it is worthwhile to further explore how their actual implications depend on the respective frame of reference: in terms of an evaluative interpretation, for instance, the relevant problem is to explain the comparatively strong agglomeration of high-quality articles in such a small number of outlets. A potential conclusion to be drawn from this perspective is that the editors of these journals do excellent work and manage to concentrate most high-quality research in economics in their outlets. This constellation, in turn, makes it easier to locate said contributions.

In contrast, while a cognitive interpretation could take such reasoning into account, it would eventually be forced to ask a broader set of questions and inspect other variables, like the diversity of authors, readers and institutional backgrounds, the role of reputational concerns, or the impact of “Big 5” publications on tenure decisions and related career concerns (Heckman & Moktan Reference Heckman and Moktan2018). In comparison with physics (fig. 3), such a cognitive perspective would necessarily go beyond asserting that editors in economics are simply superior, but would rather ask which properties make economics discursively similar to subfields in modern physics.

Under such premises, the outcome looks quite different: here the findings above reveal that the economic discourse at the very top is to a large extent self-contained and exhibits a strong hierarchical character, while the discourse in sociology and political science can be characterized as more diverse and, correspondingly, less hierarchical and insular. This example illustrates how the interpretation and associated valuation of citations is guided by theoretical presumptions, which effectively decide how to translate numerical into verbal assessments.

3. Reactivity

The second aspect in this paper relates to the phenomenon of reactivity, specifically in connection with scientific research evaluation. Our basic argument is that the introduction of journal rankings in economics in particular and in academia in general has led to a change in and a refinement of the institutional incentives governing the academic sphere.

Notwithstanding the fact that rankings have a long tradition in economics (see, for instance, Diamond Reference Diamond1989; Fusfeld Reference Fusfeld1956; Graves et al. Reference Graves, Marchand and Thompson1982; Liebowitz & Palmer Reference Liebowitz and Palmer1984; or the rankings hosted by RePEc on repec.org), we argue that the importance of journal rankings received a significant boost within the last two decades (see also Kalaitzidakis et al. Reference Kalaitzidakis, Mamuneas and Stengos2011; Stern Reference Stern2013; Zimmermann Reference Zimmermann2013). The journal rankings originally produced by the Institute for Scientific Information (ISI), whose database is today widely known as Clarivate Analytics’ Web of Science, proved to be pivotal for the ascension of journal rankings in academic assessment and evaluation, which eventually led to visible changes in the disciplinary trajectory of economics at the national level (Lee Reference Lee2007; Bloch Reference Bloch2010; Corsi et al. Reference Corsi, D’Ippoliti and Lucidi2010). As these rankings are imposed onto the scientific field for means of assessment and evaluations of researchers and institutions, they do not come without an impact on the institutional incentives individual researchers face. “There is a powerful feedback between the ranking systems used to assess scientific productivity and the actions of scientists trying to further their careers via these ranking systems” (Braun et al. Reference Braun, Bergstrom, Frey, Osterloh, West, Pendlebury and Rohn2010, 871). In short, the introduction of journal rankings for purposes of evaluation causes reactive behavior invoked by the “authority of numbers” in the sense of Espeland and Sauder, who argue that “[c]asting reactivity as a threat to validity of measures draws attention to the dynamic nature of quantitative authority and its capacity to intervene, sometimes dramatically, with other forms of authority” (Espeland & Sauder Reference Espeland and Sauder2007, 6-7).

The notion of reactivity is a well-known concept in the methodology of the social sciences and refers to instances in which the application of some instrument of investigation triggers a change in the behavior of the investigated subjects or institutions. Such behavioral changes are conceived as a source of bias, which may arise in the course of data-acquisition (especially in circumstances, where data is collected repeatedly). While it is methodologically correct to speak of a bias here in the sense that such a behavioral change is to be considered as an artifact created in the course of observation, it seems important to note that the change in behavior is, eventually, real: although induced by external observation, such changes in behavior can be profound and have far-reaching effects.

In the case of methodologically guided evaluations in general and evaluative scientometrics in particular, reactivity induces actors to anticipate evaluation criteria. This anticipation in turn affects the behavior of the subjects or institutions evaluated according to these criteria (Ferraro et al. Reference Ferraro, Pfeffer and Sutton2005, Ghoshal Reference Ghoshal2005). Such forms of “evaluation bias” have been confirmed empirically, especially for evaluation procedures such as rankings, where Espeland & Sauder’s (Reference Espeland and Sauder2007) seminal study on the reactivity in the context of the US News’ law school ranking stands out (see also Rafols et al. Reference Rafols, Leydesdorff, O’Hare, Nightingale and Stirling2012; Willmott Reference Willmott2011). Another example are international rankings based on indexes such as the World Bank’s Doing Business index or even the prominent Human Development Index used by the United Nations Development Programme (UNDP). Such country rankings may induce reactive behavior of government institutions in favor of the defined ranking criteria.

Therefore, it does not come as a surprise that journal rankings do trigger similar effects when used for large-scale institutional evaluation: Italy, for instance, has recently introduced a quantitative standard for supporting decisions on appointment and promotion within the academic system. While the official purpose of this reform is to serve as an objective way to decrease the amount of nepotism in hiring, one cornerstone is the number of scientific publications as indexed by either Scopus or Web of Science. This has led to various attempts of Italian scholars to have past publications that are not covered by Scopus/WoS included into Scopus/WoS and created a strong incentive to only publish in journals covered by these two indices (Abatemarco & Dell’Anno Reference Abatemarco and Dell’Anno2012). While we observe an overall “climate of perverse incentives” (Edwards & Roy Reference Edwards and Roy2017, 51) and also more blunt forms of reactivity taking the form of simple gaming strategies in academia (Kapeller Reference Kapeller2010; Wilhite & Fong Reference Wilhite and Fong2012; Necker Reference Necker2014), in this paper we want to ask a more nuanced question: how would the introduction of journal rankings and their rise to institutional authority change existing patterns of scientific recognition to impact citation behavior in economics? Taking the classic Mertonian view (Reference Merton1968) that the future recognition depends on past recognition already received (see also Haucap & Muck Reference Haucap and Muck2015), we argue that the introduction of journal rankings as evaluative devices affects the self-reinforcing routines governing the distribution of academic attention by raising the relative importance of journals as compared to specific authors or contributions. As journals gain in visibility in the course of the introduction and utilization of journal rankings, some attention is redistributed from the actual producers and products of scientific activities to their (most important) outlets. And, indeed, Attema et al. claim to have found that U.S. economists “would sacrifice more than half a thumb for an AER publication” (Attema et al. Reference Attema, Brouwer and Van Exel2014, 495).

So far, we have introduced two hypotheses on reactivity, namely that the introduction of journal rankings redistributes prestige in the favor of journals (see also Mcdonald & Kam Reference Mcdonald and Kam2007; Lariviére & Gingras Reference Lariviére and Gingras2010) and that economists intuitively and/or deliberately consider this redistribution of prestige, which eventually leads to a change in recorded citation patterns. Specifically, our theoretical argument implies that we should see the share of citations received by the top fraction of articles within a top journal decrease, while the share of citations going to the least considered fraction of papers is expected to rise as all papers in a given outlet draw on the latter’s prestige (see fig. 4 for a stylized illustration of the expected effect).

Figure 4.

Expected effect of a relative increase in journals’ prestige on observed citation patterns.

We would assume to find effects as envisaged in fig. 4 – an increase in the share of the bottom two thirds of articles with a corresponding decrease in the share of top-cited papers – across all disciplines. However, we also expect these effects to be most pronounced in economics as we assume a higher degree of conformity to institutional incentives in economics as compared to other fields.

In what follows, figs. 5 to 7 show our empirical findings on this question. In acquiring these results we analyzed the citation history of all (top-)journals sampled so far^{Footnote 8} and further extended our sample by acquiring corresponding data for Nature and Science. More specifically, we compare the distribution of citations to papers published between 1981 and 1985 (counting received citations up to 1990) to the distribution of citations to all articles published between 2004 and 2008 (counting citations up to 2013) for all analyzed journals to identify changes in aggregate citation patterns.

Figure 5.

Change of citation patterns in top journals: 1980s vs. today. (The authors’ own graph is based on data from Clarivate Analytics.).

Figure 5 shows the results so obtained by providing individual estimates for all disciplines under study. The resulting pattern closely corresponds with our theoretical expectations – a decrease in the share of citations dedicated to the upper limit of the distribution is complemented by a corresponding increase in lower deciles. The discipline of economics indeed shows the biggest effect with an effective decline in the share of the top decile of papers by roughly nine percent. Only the generalist top journals Science and Nature (as shown in fig. 6) as well as top journals in psychology come close to this pattern, while the same effect for the other social science journals – as well as the physics journals under study – is much less pronounced.

Figure 6.

Citation patterns in the 1980s and today: An aggregated comparison. (The authors’ own graph is based on data from Clarivate Analytics.).

Figure 7.

Publication output in the 1980s and today: An aggregated comparison. (The authors’ own graph based on data from Clarivate Analytics.).

Note: Published document types such as editorials, book reviews, bibliographies or biographic items were excluded in this analysis.*Includes sociology, political science and psychology (multidisciplinary).**Includes physics multidisciplinary, physics nuclear, physics atomic molecular chemical, physics condensed matter, and physics particles fields.***The Web of Science Category “Multidisciplinary Sciences” Includes journals that cover a broad spectrum of different specialties, such as PLoS ONE, PNAS, Nature, Science etc… The overlap with other categories (e.g. social sciences, physics) is marginal in both periods (8.1 percent and 3.1 percent of all papers respectively).

Figures 6 provides a different perspective on the same data by further aggregating results across disciplines: it shows, that the expected pattern is much more pronounced in economics as compared to the average social science or physics papers in our sample. Indeed, the journals coming closest to the pattern found for economics are multidisciplinary journals, which enjoy an high visibility and, hence, command a considerable amount of prestige. Thus, it seems that, in terms of reactivity induced by journal prestige, economics really plays in the league of the world’s most prolific journals, while leaving the mundane world of ordinary research well behind.

Eventually, the question is what to make of this result? Although the alignment between expected citation behavior and actual patterns retrieved is close and the supposed pattern non-trivial, how can we assure that our inter-temporal comparisons are due to the role of journal rankings and not, let’s say, due to other factors, like different research cultures^{Footnote 9} or simply the increasing size of the academic literature in general? While such alternative explanations are indeed appealing and, at the same time, the possibilities to quantitatively assess qualitative differences across fields are limited, we can confront the most obvious argument in this context, namely the growth of the academic literature in general.

Figure 7 compares the overall output of scientific publications between the two periods, which has increased significantly in all disciplines under consideration. However, the similarity between the citation patterns in economics and the multidisciplinary journals shown in fig. 6 stands somewhat in contrast to different growth rates in these disciplines (+187 percent in economics vs. +51 percent in multidisciplinary sciences). Conversely, the two remaining disciplines, which experience growth rates more similar to economics (+88 percent in the social sciences and +103 percent in physics), both exhibit a similar citation pattern (see again fig. 6). This pattern is, in turn, quite different to the one in economics. As this comparison is inconclusive, it can hardly be argued that the specific differences noted in figs. 5 and 6 are solely due to the different growth rates affecting these fields.

While our argument – that the difference in responsiveness to incentives across disciplines contributes to the differences in citation patterns – is surely of a cognitive origin, an evaluative approach to scientometrics would be able to offer several alternative explanations. For instance, one could argue that the average quality of papers in top journals has increased more strongly over time than the quality of top-cited papers in these journals – and that the very same process is going on faster in economics than in other fields. Again, the interpretation of the observed empirical patterns depends on our preferred theoretical framing as the data hardly put us in a position to assess the relative appropriateness of a particular perspective.

4. Diversity

Our final section deals with an alleged lack of diversity in economics. Diversity is a concept that can be approached from a variety of angles, as, for instance, demographic, disciplinary, or theoretical diversity. A lack of diversity in terms of demography is well documented in economics (e.g. Bayer & Rouse Reference Bayer and Rouse2016). Fourcade et al. recently studied the matter of intellectual diversity in economics from a disciplinary angle (Fourcade et al. Reference Fourcade, Ollion and Algan2015). By analyzing both, attitudes towards interdisciplinary research as well as interdisciplinary citation patterns in economics, sociology, and political science, Fourcade et al. show that economists are comparatively less inclined to import findings from other disciplines and also have less trust towards interdisciplinary approaches and research strategies (ibid.). These findings are highly consistent with past results from data-driven research on citation patterns, which document the solitary character of economics in terms of disciplinary location and age of imported references. Relative to other disciplines from the natural and the social sciences, papers in economics refer with a higher probability to their own specialty (see fig. 8) and the cited contributions are, on average, older than papers cited in other disciplinary contexts (Wallace et al. Reference Wallace, Larivière and Gingras2012). The only discipline in our data exhibiting a higher degree of self-referentiality than economics – astrophysics and astronomy – is coined by a very specific and clearly separable research focus (“outer space”), which may explain the high share of inward-looking citations in this field.^{Footnote 10} For economics, on the other hand, these results not only point to the comparatively strong imprint of traditional approaches and findings on current research, but also signify a comparatively strong intellectual autarky in current economic discourse.

Figure 8.

Citation patterns in economics and other disciplines: a large scale sample. Authors’ own graph based on data from Wallace et al. (Reference Wallace, Larivière and Gingras2012).Footnote ¹¹.

In a similar vein, also Fourcade et al. (Reference Fourcade, Ollion and Algan2015) summarized their findings under the heading of “insularity” to signal an absence of theoretical and conceptual diversity. And indeed, economists often conceive interdisciplinary interaction as a competition rather than cooperation, where disciplines are to be judged with regard to their relative “performance.”

“What is interesting to think about are the terms of trade between economics and all these other disciplines. We are clearly a net exporter to political science and sociology” (Wolfers Reference Wolfers2010, 30).

In what follows, we want to add to this picture by supplying a perspective on conceptual diversity in economics. This question is especially peculiar as economics can be regarded as a “contested discipline” (Lee et al. Reference Lee, McConnell and Dean2010, and Lee & Elsner Reference Lee and Elsner2011). The character of scientific discourse in economics differs from that found in other social sciences, because economics is dominated by a single paradigm – neoclassical economic theory – which significantly shapes academic teaching and economic research.^{Footnote 12} A thus resulting exclusion of alternative economic approaches, like institutional microeconomics, evolutionary analysis of economic change, or Keynesian macroeconomics, from the “official” economic discourse in further consequence led to the foundation of – nowadays eminent – heterodox economic journals, like the Cambridge Journal of Economics, the Journal of Post Keynesian Economics or the Journal of Economic Issues in the late 1960s and 1970s (King Reference King2003, 134-136). Notwithstanding these efforts for providing alternative economic approaches with adequate institutional foundations, alternative or “heterodox” schools of thought nowadays only constitute a small fraction inside the economics discipline. While heterodox economists are confronted with a series of exclusionary practices and routines (Dobusch & Kapeller Reference Dobusch and Kapeller2009; King Reference King2013), their works also draw on a broader variety of sources and inputs and, hence, exhibit a distinctive, more interdisciplinary communicative pattern (Glötzl & Aigner Reference Glötzl and Aigner2015). Moreover, Glötzl & Aigner (Reference Glötzl and Aigner2015) provide additional evidence for this contestedness of economics by pointing to a “mainstream core – heterodox periphery structure” (see, also, Dobusch & Kapeller Reference Dobusch and Kapeller2012a) in the citation networks associated with different economics departments at the University of Vienna and the Vienna University of Economics and Business, that differ with respect to their openness for alternative and heterodox approaches in economics.

The resulting lack of diversity within the economic discipline has been studied empirically, in particular by analyzing the relative citation flows between 13 high-ranked mainstream and 13 high-ranked heterodox economic journals in two different periods: before 1989-2008 (Dobusch & Kapeller Reference Dobusch and Kapeller2012b) and after the recent financial and economic crisis, 2009-2013 (Aistleitner et al. Reference Aistleitner, Kapeller and Steinerberger2018). The results indicate that the monistic and closed citation behavior exhibited by (highly-ranked) mainstream economic journals remained stable after the crisis, as the mainstream journals continue to devote only minimal attention to heterodox inputs (as is evidenced by the small share of citations – less than 3 percent – imported from heterodox sources^{Footnote 13} – see fig. 9). On the contrary, the heterodox journals themselves exhibit a more balanced citation pattern with import shares from the mainstream sample of 47.6 percent (pre-crisis) and 24.6 percent (post-crisis), where this decrease in part also reflects the fact that heterodox contributions typically cite older papers from the mainstream literature more strongly than recent ones. In addition, this result lends support to Kuhn’s Reference Kuhn1962 prediction, which stated that a dominant paradigm that faces a significant anomaly (the crisis) will strive for adapting already established theses, models, and methods instead of turning to more fundamental revisions.

Figure 9.

Diversity and the interaction pattern between economic mainstream and heterodox journals. (a) the discourse between a sample of top 13 orthodox and top 13 heterodox journals (1989-2008; taken from Dobusch & Kapeller Reference Dobusch and Kapeller2012b). (b) replication with a corresponding control group (1989-2008; the authors’ own calculation based on data from Clarivate Analytics). (c) replication for the post-crisis period (2009-2013; the authors’ own calculation based on data from Clarivate Analytics). (d) Analysis of a large-scale journal sample (1969-2013; taken from Aigner & Glötzl, in this issue).Footnote ¹⁴

To further assess the robustness of these results, fig. 9 additionally shows results derived from data on a control group and a large-scale sample. The first robustness-check uses a control group to ask whether the comparative neglect of heterodox inputs in mainstream outlets can be explained by the lower ranks of heterodox journals, which supposedly signify a lower overall quality. In composing the control group we used a group of mainstream journals with similar ranking positions than the heterodox sample. The second robustness check analyzes data of a large-scale sample acquired from the Web of Science database to scrutinize the robustness of the observed differences.

By comparing these results to other recent findings on the “insularity” of the economics discipline (Fourcade et al. Reference Fourcade, Ollion and Algan2015), one notices several similarities between an interdisciplinary and an intradisciplinary view on conceptual openness in economics. For once, we find that economics is comparatively closed not only to outside disciplines, but also to theoretical outsiders within the discipline. For another, we observe that these theoretical outsiders share close ties to other branches of the social sciences (e.g. Glötzl & Aigner Reference Glötzl and Aigner2015), and thereby do not resemble the disciplinary isolation practiced by the economic mainstream. A possible, tentative conclusion to be drawn from this is that the interdisciplinary insularity of economics as diagnosed by Fourcade et al. (Reference Fourcade, Ollion and Algan2015) or Gingras and Schinckus (Reference Gingras and Schinckus2012) with respect to econophysics is not primarily due to the idiosyncracies and peculiarities of the subject matter in economics, but rather represents an outgrowth of the dominant economic approach which seems to discount external inputs a priori.

Qualitative examples for illustrating the above line of argument are easily found. One instance is provided by the “Economics of Identity” – a stream of research based on a well-received paper by Akerlof & Kranton (Reference Akerlof and Kranton2000), which gathered more than 800 citations in Web of Science, most of them from the field of economics. In this context, it is interesting to note that the conceptual core of “identity” as introduced in economics – that people have ideas about who they are and try to conform to these ideas in their behavior – overlaps strongly with the meaning of identity as used by social psychologists since, at least, the 1950s. While Akerlof & Kranton (Reference Akerlof and Kranton2000) obviously import this idea into economics and thereby attract much interest within their profession, classic contributions on social identity in other fields, e.g. Tajfel (Reference Tajfel1974) or Ashforth & Mael (Reference Ashforth and Mael1989), remain largely neglected in the economic discourse, although they are widely cited in general. In sum, these two papers have more than 3,000 documented citations in Web of Science, where only 34 of those stem from economics.^{Footnote 15} Hence, Akerlof & Kranton (Reference Akerlof and Kranton2000) have contributed significantly to increasing the prominence of the psychological concept of identity within economics. It, however, remains difficult to say, whether this increase in prominence is to be attributed to an increase in visibility or legitimacy. In the former interpretation, “economists read narrowly” and Akerlof & Kranton (Reference Akerlof and Kranton2000) simply introduce economists to a hitherto unknown concept – personal identity – that is in turn taken up for further investigation. In the latter interpretation, “only economists do it right” and personal identity as a theoretical concept gains credibility and plausibility exactly because it is being taken up by well-known economists in a very respected outlet. While our results point towards the importance of both factors, visibility as well as legitimacy, it is remarkable that the integration of identity in the economics’ discourse does not spill over to the extra-disciplinary origins of said concept.

Against this backdrop, the relative “superiority” of economics derived from inspecting citation-trade balances – as suggested by Wolfers (Reference Wolfers2010) – can be explained not only by a higher quality of research outputs, but also by a higher degree of group-think and inward-looking behavior within the economics profession, which effectively suppresses the import of ‘foreign’ citations. As we have seen, such an interpretation would be well in line with the average economists stance on the value and merit of extra-disciplinary contributions. Yet again, we find that the two ways of looking at and interpreting citation data lead to very different implications.