The Perks and Perils of Interdisciplinary Research

This paper synthesizes findings from two studies the author conducted that examine how engagement in interdisciplinary research (IDR) influences scholars’ careers. Results from these two studies, one large-scale and quantitative and the other small-scale and qualitative, provide a much needed empirical assessment of IDR’s effects on individual careers. In essence, they provide a nice antidote (and some caution) to the rhetoric and enthusiasm surrounding IDR. My co-authors of these studies and I find that engaging in interdisciplinary research increases a scholar’s visibility in terms of citations, but also presents challenges, including reduced productivity, cognitive challenges, lack of support, extra time and commitment, and framing of one’s work. This paper concludes by discussing the policy implications of this research.

2017, is a quantitative study of almost 900 scientists affiliated with research centers in the United States and their 30,000 scholarly articles, and was co-authored with Christine Beckman and Taryn Stanko. We find that interdisciplinary research experiences perks (more visibility as measured by citations) as well as perils (reduced productivity), and we explore the origins of such a 'productivity penalty'. The second study, 11 published in Investigating Interdisciplinary Collaboration: Theory and Practice across Disciplines, edited by Scott Frickel, Barbara Prainsack, and Mathieu Albert, and co-authored with David J. McBee, is a qualitative study of interdisciplinary scholars and the possible roadblock and challenges they experience. Our interview with tenured interdisciplinary scholars in the humanities (e.g., English, History, Philosophy) reveals that interdisciplinary research requires extra cognitive effort and additional time to produce. Moreover, interdisciplinary scholars long for greater levels of support from their institutions and colleagues. Below, I provide a short synopsis of each study.

Study 1
Two strands of organizational theory led us to believe that IDR would incur both costs (in terms of productivity) and benefits (in terms of citations). The first documents penalties associated with category spanning. 12 Rather than being perceived as innovative, offerings spanning multiple domains have an ambiguous identity that is difficult for audiences to understand, and are thus devalued. 13 Moreover, such works may be more difficult to produce: investing in multiple categories limits mastery and dilutes quality, resulting in a 'Jack of all trades' who is master of none. 14,15 Grasping ideas and perspectives from another field is cognitively taxing and time-consuming, working with diverse collaborators from multiple disciplines can produce frustration and conflict, and reviewers may have difficulty digesting and evaluating crossdisciplinary products. 16 All of these challenges may lengthen the time to publication, and thereby depress interdisciplinary scholars' productivity. The second strand is recombinant innovation, [17][18][19] which documents the benefits of joining distinct ideas across domains. This work suggests that bridging disconnected knowledge spaces will result in better ideas that will be rewarded in the marketplace. 20 Research on the impact of category-spanning in science finds that atypical, category-spanning offerings have higher impact. [20][21][22][23][24][25] Interdisciplinary publications, as a form of atypical, domain-spanning publications, likely experience these same benefits, and garner more citations.
To test these two hypotheses, we identified all scientists associated with 52 NSFfunded industry/university-cooperative research centers (IUCRCs) in the United States. Our analysis is limited to the subset of 854 PhD-level scientists with a publication record, because our key variables (IDR, productivity, and visibility) cannot be computed for scientists who have not published. Most (80%) of these scientists are in science or engineering fields, with the other 20% coming from math, computer science or the social sciences. We obtained their publication records (as of 2005, about n = 32,000) from Thomson Reuters' Web of Science (WoS). To these data we added field-, university-and individual-level data from various sources.
We turned to previous scholarship for guidance on how to measure the key concepts of interest to us. To capture each scholar's productivity, we rely on the total number of articles published (in WoS journals) from the beginning of a scholar's career (i.e. the year they first published) until 2005. We measure visibility by collecting the (forward) citations that have accrued to each published article (indexed in WoS) as of 2010. And we borrow Porter and colleagues' measure of integration (equivalent to the Rao-Stirling Index of Diversity) to capture interdisciplinary research. This relies on the WoS Subject Categories to proxy disciplines. While a number of other measures of interdisciplinary research have been proposed and used in previous research, no other measure incorporates the relatedness of the categories that are joined. Spanning cognitively similar fields such as civil and chemical engineering does not boost the IDR score as much as spanning cognitively distant fields such as geography and optics. The IDR score ranges from 0 to 1, with scores closer to 1 indicating greater interdisciplinarity. More details can be found in Ref. 10, especially Table A1. We also control for variables that have been shown to influence engagement with IDR as well as productivity and visibility, including: gender; professional age; status and the individual and institutional level; as well as field level characteristics (like average turnaround time at journals).
Our models, specified at both the paper level (using fixed effects models for panel data) and the individual level (using structural equation modeling), lend support to our hypotheses. As expected, IDR depresses scholarly productivity: an increase of 0.10 in the IDR score reduces productivity by 9.6% over one's career, controlling for professional age and other factors. This effect suggests that interdisciplinary scientists do indeed experience lower productivity. This productivity penalty holds, and even gets stronger, when we weight the article count by number of co-authors (such that a paper with two authors only contributes 0.5 to a scholar's productivity). We also examine IDR's effect on productivity at the person-year level (in which IDR scores are averaged, and publications are summed). We find that in the years when scholars do more interdisciplinary work, they publish fewer articles. However, once published, IDR shines: in support of our second expectation, we find that IDR increases scholarly visibility. The coefficient for IDR suggests that an increase of 0.10 in the IDR score increases a scholar's citations, on average, by 6.2%. This effect also holds when we exclude 'group authors', whose papers tend to be highly cited, and when we consider a narrow window of only 2001-2002 publications, a period beyond the five years useful for predicting long-term citations.
Beyond documenting an important trade-off to IDR engagement, we also find that IDR is indeed a high-risk, high-reward strategy: scientists who publish more IDR are more likely to produce both frequently-cited and rarely-cited works. Here, we modeled the standard deviation in citations rather than mean citations (and also control for the standard deviation rather than the mean of journal impact factor). We find that interdisciplinary scientists, in addition to having a greater total number of citations, also experience more variability in citations across their papers. Scientists The Perks and Perils of Interdisciplinary Research S57 with a record of interdisciplinary scholarship experience more 'hits' and more 'flops' than their mono-disciplinary counterparts. We also document that distancei.e. the cognitive dissimilarity among fieldsmatters. Specifically, it contributes to the visibility boost and determines the productivity penalty. We assessed this by extracting distance out of the IDR measure: we simply calculate the total number of unique subject categories (SCs) referenced by a scientist (across all his or her papers) with no regard for their similarity. When we substitute this measure for the IDR measure, the positive effect on citations holds, suggesting that even spanning related fields improves citations (if only slightly), presumably by broadening one's prospective audience. However, the negative effect on productivity does not hold. In fact, this alternative measure of IDR positively affects productivity, perhaps because drawing on multiple disciplines expands the number of possible journal outlets. Simply drawing upon more SCs doesn't hinder productivity, unless those SCs are cognitively dissimilar. This suggests that it is more difficult to produce and successfully publish scholarship that spans unrelated fields (e.g. chemical engineering and anthropology) than related fields (e.g. chemical engineering and civil engineering). This offers support for the mechanism behind the production penalty we theorize: IDR is cognitively difficult and slow to produce when it blends disparate fields.
We also found that the level of IDR in the field matters. In highly interdisciplinary fields such as the life sciences, IDR's positive impact on visibility is stronger than in less interdisciplinary fields such as electrical engineering. What affects productivity is trends toward IDR: fields trending toward IDR invoke fewer penalties for producing this type of work, perhaps because they provide better training in how to manage the cognitive challenges, and/or are more amenable to IDR in the peer review process. In sum, static differences in field IDR (high (in life science) versus low (in electrical engineering)) shape the reception of IDR work, but the trends in field IDR moderate the production of that work.
Supplemental data sources and analyses lead us to conclude that IDR projects face communicative and collaborate hurdles that in turn drive the productivity penalty we observe. First, we find that working repeatedly with a similar set of collaborators reduces the productivity penalty. Second, we surveyed scientists to ask about the nature of the collaboration on their most recent co-authored paper, and 68 of these also fall in our sample. Some, of course, are more ID than others. We found that projects leading to interdisciplinary articles endure communication difficulties: communication is reportedly less clear (p = 0.065), more difficult (p = 0.083), and of lower quality (p = 0.106). Moreover, interdisciplinary teams have more difficulty generating ideas (p = 0.109). Third, we constructed a measure of multidisciplinarity, and find that merely pooling multiple fields does not affect productivity negatively, only true integration (measured via IDR) does.
However, we find little support for the common assumption that IDR faces a more challenging review process. We collected data on the length of the review process (i.e. turnaround time) for two journals represented in our data from WoS: one publishing articles with an above-average IDR score, and another publishing articles with a S58 Erin Leahey below-average IDR score. For the 711 articles published in these journals (written by 145 of our sampled authors), the median time under review is 85 days. However, there is no significant correlation (r = 0.04) between time under review and the paper's IDR score, and the IDR score does not predict time under review in a paper-level model with controls and fixed effects. Confidence in this finding is buttressed by an analysis of data on these scholars' unpublished working papers that we collected from www. arXiv.org. We compared working papers that eventually got published with those that did not. Here, the eventually published papers are actually more interdisciplinary than the still unpublished papers. When we limit this comparison to authors with one unpublished working paper and one eventually published working paper in www. arXiv.org, we find no difference in IDR. Taken together, these results suggest that IDR papers are not hindered in the review process.

Study 2
To investigate the productivity penalty more in depth, and to explore other possible costs, Dave McBee and I studied recipients of Andrew W. Mellon Foundation's New Directions Fellowships. The goal of the New Directions Fellowship program is to 'assist faculty members in the humanities and humanistic social sciences who … wish to acquire systematic training outside their own disciplines'. 26 Each year since its inception in 2002, about 20 faculty members from the humanities or humanistic social sciences have been awarded year-long fellowships that allowed them to work on interesting cross-disciplinary problems. Recipients had received tenure in their home discipline and were eager to move beyond their disciplinary bounds. Of the 115 fellows to date, we selected all (34) recipients from three disciplines (English/ Literature, History and Philosophy) who received fellowships before 2010. These three traditional disciplines were well represented among the fellows. We excluded fellows from less well-represented disciplines, such as East Asian studies, to allow the possibility of disciplinary comparisons. We omitted more recent fellows (from 2011-2013) to ensure sufficient post-fellowship experience with interdisciplinary research. Eighteen (more than half) of these fellows agreed to participate in a recorded telephone interview. All interviews were conducted between December 2012 and February 2013. The interview schedule focused on faculty members' experiences, reflections, and intellectual identities. Specifically, we asked about: the interdisciplinary nature of their work; how they presented IDR to relevant to communities inside and outside their home disciplines; how IDR affected their productivity; the extent of institutional support; hurdles during peer review; and possible psychological or emotional repercussions of engaging in IDR. We asked directly about frustrations, roadblocks, and other challenges that these scholars may have encountered, and probed for details. Open-ended answers to these questions provide a rare glimpse into the world of scholarly production, risks, and receptionone that is rarely visible from analyses of bibliometric data alone, 27,28 and in this way serves as a nice complement to study 1. We developed a comprehensive coding scheme based on extant literature, but allowed other themes to emerge from the data during our multiple readings of the transcripts. The two authors, plus an undergraduate research assistant coded responses independently and reconciled all differences. Team-based coding is an intense and timeconsuming task that is critical for assuring reliability and validity. 29

Theme 1: IDR is Hard
Almost all (89%) New Directions fellows we interviewed noted the extra cognitive effort required to integrate knowledge across disciplinary boundaries. For example, when we asked Mary if she could claim expertise in her second discipline, she said, 'Well, I wouldn't go that far. I think I am a much more able "channeler" of what's going on in that area, but I wouldn't publish in a theology journal. I don't feel I've become a complete native in that environment.' When we asked David, a philosopher, about the challenges he faced integrating law into philosophy, he mentioned the vast amount of additional reading that was required to get up to speed: 'The amount of material on any given topic is just bewildering and there isn't enough time in the day to read it all…'. John, a historian, described the difficulties he had just 'learning how to learn' in a new field: '[The new area was] something I didn't know anything about, something that I didn't know how to study even if I had the technical ability to study […].' Cognitive challenges may be heightened for scholars in the humanities, where collaboration is uncommon and individual expertise, rather than team-based knowledge-integration, is required. 30 For them, interdisciplinary research means not only working in an unfamiliar domain, but doing so in isolation, as Niels describes. Having been turned away by a potential mentor in his second discipline, he said 'I just sort of found myself a bit in a gap because the material was hard enough that I just really wasn't able to teach myself quickly enough.'

Theme 2: IDR Requires Extra Commitment
Most New Directions fellows we interviewed said that they worked hard to move beyond mere surface-level engagement with other fields They recognized that ideas from other intellectual domains can be misrepresented as they are passed on to unsuspecting disciplinary audiences. 31 As Daniel, a historian noted, All of us have read things where scholars of one field try to incorporate another discipline into their own and failed miserably because they make fundamental errors about how another field works. I think the danger in dabbling in anything interdisciplinary is that if you skim the surface […you could say] something really obvious or banal or insulting about another field before really attempting to engage it seriously.
Specifically, three-quarters (78%) of respondents noted the extra commitment and responsibility that thorough IDR requires. Daniel described his efforts to stay current with rapidly-changing trends in a medical field: Renaud said that he was concerned to make sure 'that I do justice to what I am trying to do.' For Jordan, a philosopher, the standard for thorough interdisciplinary work was publication in the journals of one's secondary discipline: … in my mind, if you are a philosopher and you are interdisciplinary, then you should be able to publish in journals of the discipline that you're engaged with. So if you're doing philosophy of science with respect to genetics, is it possible for you to publish your papers in Nature Genetics? Or is it you only publish in philosophy journals but you talk about genetics. To me, that's not necessarily interdisciplinary.
[…] I have several papers in science and medical journals, in journals that are not philosophy journals.

Indeed, this extra commitment can help maintain interdisciplinary engagement.
When asked about what it was like to take classes as a student again, Elizabeth mentioned that she thought that many professors refrained from interdisciplinary work because it threatened their professional identity as an expert. To really delve into interdisciplinary training, she said, she '… needed to be willing to become a learner again, to not be good at things.' She is not alone; 26% of all respondents shared similar sentiments. When asked how he used the fellowship to further his research agenda, Renaud said, 'There is something terrifying and useful about being a student again, you know grappling with things that are way too hard for you.' When we asked Jessica, an English professor, what advice she would give to others seeking to do similar interdisciplinary research, she said '… you have to have […] a willingness to look stupid and then just test it.' From these excerpts, the added responsibility and commitment of good IDR is palpable. Meeting the added responsibility of IDR and maintaining its integrity, in the words of John, 'required a deeper investment of time to go beyond the surface, to get beyond what other interdisciplinary researchers were willing to do.'

Theme 3: IDR is Time-consuming
Learning a new field takes time, especially when scholars are committed to doing it well, and thoroughly. In this way, we see that time and commitment are somewhat contingent on each other. Mary, a historian, described becoming a theologian as 'a very long formative process'.
Susan, when asked about the challenges she faced integrating her second discipline into her home discipline, said, 'It's time. It's just time'. Most (78%) of our scholars had similar experiences and indicated that their interdisciplinary work required more time to conduct. Reading and learning new material from another discipline is more difficult, as indicated above, but these challenges are amplified when they compete with existing professional obligations. Susan, a historian of science, described this: Some New Directions fellows claimed their interdisciplinary research took longer because they spent more time checking claims from the secondary discipline.
Although only 17% of respondents explicitly mentioned fact-checking, it is an intense challenge. Jessica describes the extra time she took checking scientific claims before presenting her work to audiences in the humanities: I try to be very clear in the claims that I am making and the claims that I am not. I try to read a lot of the critical counter-discourse and address the kinds of concerns that they have had. As I said I want to make sure that I dot every 'i' and 'j', and cross every single 't' that needs to be crossed so that people can't make any objections to the substance of my work, just the interpretation of it.

Theme 4: IDR Takes a Toll on Productivity
Because learning and incorporating ideas and methods from other disciplines takes time, a substantial number of fellows (44%) experienced slower productivity that resulted in fewer publications. This is consistent with our findings from Study 1. John said, … if I think about the number of articles or book chapters or whatever that I could have written, instead of studying [my secondary discipline], it's immense, right? I've cost myself, in terms of pure scholarly productivity, a great amount of research.
Niels commented on the cost of teaching himself a branch of mathematics: Actually, if one looks at my research outputfor the sake of my career promotion and advancement I don't try to say this too loudlybut there's actually a big gap. It looks like I went into a coma or something.
In particular, writing becomes inefficient. Renauld, commenting about the benefits and drawback of adding new methodological approaches from his second discipline to his research repertoire, said 'Interestingly, it has probably slowed down some of my writing. Because the book I'm writing now is probably more ambitious some ways in scope.' Mary had a similar experience and discussed how much longer it takes her to generate an article in her interdisciplinary area. When asked if engaging another discipline hindered her ability to produce high-impact scholarship, she said, 'I wouldn't say difficulty, but I would think about the time frame. [Part of why it takes] longer to go from conception to a finished product has to do with trying to think through two separate disciplinary concerns.' S62 Erin Leahey

Theme 5: Lack of Support
Most of the New Directions fellows we interviewed (61%) were discouraged by a lack of support for their interdisciplinary work. Funding was not a major concern; only 17% of respondents mentioned a lack of research funds. Instead, the fellows were concerned with interpersonal and administrative support. They were discouraged by disciplinary peers who pushed back against their interdisciplinary efforts. Sometimes fellows were compelled to justify their interdisciplinary research in the face of criticism. Cynthia, a historian, describes how members of her field viewed her interdisciplinary direction. She said, People understoodthey didn't understandthey understood what I was writing about, they didn't quite understand why I wanted to take the tack that I did to it. Why I was using philosophy, and why would anyone be interested in philosophy in relation to physics?
Niels, a philosopher eager to integrate mathematics into his research, describes being teased by a department colleague after learning of his fellowship award: '…he jokingly said to me, "What new thing are you going to learn for the Mellon scholarship? Are you going to learn philosophy?"' The premise here is that a philosopher so focused on mathematics no longer 'knows' philosophy and has to re-learn it. Later when we asked Niels whether he felt like he had a home discipline, he described the lack of informal social support he received from his home discipline: 'I definitely feel a bit alienated in mainstream philosophy.'

Theme 6: Framing
Almost all (89%) fellows were aware of epistemological differences between disciplines, and were continually challenged to frame their interdisciplinary scholarship as legitimate and interesting to others. Scholars embedded in disciplinary cultures must be convinced that unfamiliar ideas, subjects, and methodologies are interesting, valid, and appropriate. As Justin put it, the question is '…are you more interesting than different, or different than interesting? […] You're doing something that, in someone's lights, is seen as naïve. You feel like an idiot when you expose yourself to being judged by other standards.' Making valued contributions to multiple disciplinary audiences is not a smooth process; interdisciplinary researchers run the risk of criticism from both sides. When I asked Samuel, a historian, how his interdisciplinary dissertation was received by historians and academics in other fields, he summarized his concerns succinctly: 'There were quite a few years that my nightmare was that the Jewish historians would dismiss it as the work of a Russianist, and the Russianists would dismiss it as the work of a Jewish historian.' New Directions fellows took pains to circumvent perceptions of diluted quality and the resulting devaluation that besets domain-spanning efforts such as IDR. 32 For example, Sarah, a historian, requested friendly reviews from colleagues before sending her papers to a journal. She described her colleagues as being '…quite nervous dealing with stuff that doesn't fit […] and the more you try to sort of incorporate

Conclusion
Results from these two studies, both published (and elaborated in more detail) elsewhere, provide a much needed empirical assessment of IDR's effects on individual careers. In essence, they provide a nice antidote (and some caution) to the rhetoric and enthusiasm surrounding IDR. Study 1 does show that interdisciplinarity is beneficial, at least in terms of providing scholars with some added visibility in the scientific community (as gauged by citations). This in turn creates a societal level benefit in the form of more useful and valuable science. However, this benefit is not a sure thing: we find that IDR not only increases (mean levels) of citations, but also variability (variance) in citations, lending support to the idea that IDR is a high-risk, high-reward endeavor. Moreover, the penalties to IDR are numerous. Study 1 shows that scholars who engage in IDR experience a decrease in productivity, and that this is attributable to coordination and communication challenges of working with diverse teams rather than hurdles in the peer review process. Study 2 corroborates this productivity penalty using a smaller sample of scholars in very different fields (the humanities) and qualitative data and analytic techniques. Study 2 also uncovers numerous other costs to IDR engagement, including cognitive challenges, lack of support, extra time and commitment, and framing. Results from these studies are hopefully useful to a variety of communities. Scholars, for example, might obtain a better sense of what IDR entails, and what hurdles to expectwhether they work in teams like most of the scientists in Study 1 or alone like the New Directions fellows in Study 2. University administrators might also learn that while funding for IDR is useful, it alone is insufficient.
Interdisciplinary scholars need additional time to conduct research (e.g. release from teaching or reduction in service) and more opportunities to foster supportive networks. Yet in many academic institutions, statements of support from the administration are still decoupled from actual practice. This is likely to continue because there are many 'trading zones' that require support and coordination. 33 Finding more ways, especially non-material ways, to support interdisciplinary researchers may mitigate some of the challenges we identified. Some institutions have done this via cluster hiring, which may help reduce the isolation that some IDR scholars in the humanities experienced. Institutions might also be encouraged to revise their promotion and tenure guidelines and help educate P&T (Promotion and Tenure) committee members about the additional challenges associated with IDR so that they can better evaluate interdisciplinary candidates. Professional associations, colleges, and departments can offer workshops and mentoring