Introduction
One central goal of translational science is to develop innovations that positively impact clinical or population-level health and reduce disparities [Reference Smith, Molldrem, Farroni and Tumilty1]. To track progress toward these goals, the Clinical and Translational Science Award (CTSA) program, the National Center for Advancing Translational Sciences (NCATS), and the translational science community have invested heavily in evaluating the field’s impact on individuals, scientific careers, biomedical innovation, and society [Reference Patel, Rainwater and Trochim2,Reference Daudelin, Peterson and Welch3]. Much progress has been made in evaluating the impact of translational science on specific outcomes, such as the success of CTSA training and pilot grants [Reference Sorkness, Scholl, Fair and Umans4–Reference Lee, Pusek and McCormack6], returns on investment for particular innovations [Reference Wnorowski, Yang and Wu7,Reference Allwardt, Ainscough and Viswanathan8], and the impact of major initiatives like the Streamlined, Multi-site, Accelerated Resources for Trials (SMART) IRB platform [Reference Vardeny, Hernandez and Cohen9]. However, measurable assessments of the field’s overall impact on biomedical innovation and society remain elusive, despite a desire for such metrics [Reference Duda, Grainger and Guldberg10,Reference Luke, Sarli and Suiter11].
Using approaches from the field of Science and Technology Studies (STS), including archival and policy analysis methods, we have found that CTSA evaluators have regularly lamented falling short of aspirations to measure the impact of translational science on society, even while promoting the field’s potentially transformational impact [Reference Molldrem, Moses, Smith, Tumilty and Farroni12]. By assessing key outcomes such as publications and grants received by CTSA-funded scientists and collaboration across hubs, NCATS has enacted what we call an “evaluative way,” whereby translational science’s self-assessment strategies have deeply shaped its innovations and raison d’être [Reference Molldrem, Moses, Smith, Tumilty and Farroni12].
Against this backdrop, we argue that CTSA program stakeholders should incorporate STS scholars into the next generation of evaluation efforts to develop evaluation strategies suited to measuring the large-scale impacts of translational science. STS is an interdisciplinary field with a 50-plus year history of analyzing how scientific innovations reshape society – including describing how novel sciences develop, produce knowledge, disseminate innovations, and inform policy [Reference Callon, Lascoumes and Barthe13,Reference Jasanoff14]. For example, STS scholars have studied the emergence of synthetic biology and nanotechnology since their inception, tracking these fields’ development and broader impacts on science and economic processes [Reference Rabinow and Bennett15,Reference Viseu16].
In mobilizing STS expertise to evaluate the CTSA program and translational science, NCATS can invest in examining the comparative historical, social, and policy-oriented drivers that have led major initiatives to succeed or fail. For example, undertaking long-term ethnographic research or historical studies using archival documents can potentially shed light on the larger-scale impacts of CTSA programming and translational initiatives more effectively than narrowly tailored evaluation metrics, therein opening new opportunities for improvement and innovation. This is partly because of the epistemological underpinnings of STS, which focus on the plurality of contingent ways in which knowledge is created, disseminated, and taken up. We now sketch out what it might look like to pursue these goals.
STS and CTSA evaluation
STS scholars should become embedded in the CTSA network and its evaluation enterprise. STS uses varied methods to assess the development and impact of scientific disciplines – generally using embedded approaches such as long-term ethnographic fieldwork, interpretive policy analysis, qualitative methods, and archival research [Reference Jasanoff14,Reference Guston17–Reference Evans19]. Like translational science, STS is not a discipline, but an interdisciplinary field with theoretical and applied aims, including to make science more responsive to social values. STS scholars can mobilize approaches to help ensure that future evaluation efforts are robustly informed by prior developments and are well-suited to meeting the CTSA program’s desire to understand translational science’s impact on science and society [Reference Smith, Molldrem, Farroni and Tumilty1,Reference Molldrem, Moses, Smith, Tumilty and Farroni12,Reference Evans19]. Luckily, STS scholars are already present at most universities across departments and disciplines, constituting readymade potential partners and collaborators for CTSAs.
Translational science is not entirely foreign to STS. However, to date, STS scholars have studied translational science as a research object, without attempting to develop applied tools that could contribute to the practice of the science [Reference Pinel, Wyatt and Faulkner-Gurstein20]. This work has documented the impacts of translational science on the health sciences and society, including on “translational lag” discourses [Reference Aarden, Marelli and Blasimme21], the emergence of new public-private partnership models [Reference Jeske, Hauray, Boullier and Gaudillière22,Reference Robinson23], and the role of evaluation in the emergence of translational science [Reference Molldrem, Moses, Smith, Tumilty and Farroni12]. We suggest that STS scholars not only pursue studies of translational science, but also become embedded within translational enterprises and CTSA hubs. Supporting STS research can help fulfill longstanding goals within translational science to measure the field’s large-scale impacts, because STS provides methods for studying how translational science has sought to transform how biomedical knowledge is produced and disseminated. Therefore, being embedded in CTSAs can allow STS scholars to identify internal factors that shape translational innovation while also foregrounding STS’s broader focus of tracking the social, political, and economic impacts of scientific discovery.
Table 1 shows “enduring areas of evaluative focus” and “example measurements” in CTSA program evaluation history, drawn from a paper by our group [Reference Molldrem, Moses, Smith, Tumilty and Farroni12]. It then presents potential STS methods that could be mobilized to evaluate translational science and its impacts on science and society across a variety of scales.
Table 1. Existing models and potential STS approaches for evaluating the impact of the CTSA program
Models for embedding diverse disciplines exist in the CTSA program in community engagement, biostatistics, research ethics, dissemination and implementation (D&I), team science, and evaluation. STS scholars can partner with these formations while also becoming embedded in translational initiatives as autonomous experts. Our CTSA is one site where STS is becoming embedded, facilitated through our longstanding embedded ethics program, where humanities faculty engage in CTSA activities related to ethics, team science, community engagement, D&I, and other areas. In addition to embedding STS in CTSAs, NCATS should develop mechanisms to directly fund STS scholars to conduct large-scale studies of translational impact. This work can also contribute to the further development of STS literatures.
Conclusion
NCATS has consistently framed measuring translational science’s impact on biomedical innovation and society as a central goal. However, even after efforts such as the Common Metrics Initiative, the field still falls short of this [Reference Duda, Grainger and Guldberg10–Reference Molldrem, Moses, Smith, Tumilty and Farroni12]. STS offers methods well-suited to assessing the impact of translational science on the health sciences and society-at-large that operate at different registers than traditional evaluation [Reference Molldrem, Moses, Smith, Tumilty and Farroni12,Reference Pinel, Wyatt and Faulkner-Gurstein20–Reference Robinson23]. By working collaboratively with existing CTSA evaluation efforts while bringing new intellectual resources to bear on evaluating translational science, STS can advance the field while enriching understandings of how translational ways of knowing are remaking the world.
Author contributions
Stephen Molldrem: Conceptualization, Writing-original draft, Writing-review and editing; Jacob D. Moses: Conceptualization, Writing-original draft, Writing-review and editing; Emma Tumilty: Writing-review and editing; Peyton Swanson: Writing-review and editing; Jeff Farroni: Writing-review and editing; Elise M. R. Smith: Writing-review and editing.
Funding statement
This study was conducted with the support of the Institute for Translational Sciences at the University of Texas Medical Branch, supported in part by a Clinical and Translational Science Award (UL1TR001439) from the National Center for Advancing Translational Sciences at the National Institutes of Health. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Competing interests
The authors have no competing interests to report.
Open access
