Skip to main content Accessibility help
×
×
Home

Feasibility of common bibliometrics in evaluating translational science

  • M. Schneider (a1), C. M. Kane (a2), J. Rainwater (a3), L. Guerrero (a4) (a5), G. Tong (a6), S. R. Desai (a7) and W. Trochim (a2)...

Abstract

Introduction

A pilot study by 6 Clinical and Translational Science Awards (CTSAs) explored how bibliometrics can be used to assess research influence.

Methods

Evaluators from 6 institutions shared data on publications (4202 total) they supported, and conducted a combined analysis with state-of-the-art tools. This paper presents selected results based on the tools from 2 widely used vendors for bibliometrics: Thomson Reuters and Elsevier.

Results

Both vendors located a high percentage of publications within their proprietary databases (>90%) and provided similar but not equivalent bibliometrics for estimating productivity (number of publications) and influence (citation rates, percentage of papers in the top 10% of citations, observed citations relative to expected citations). A recently available bibliometric from the National Institutes of Health Office of Portfolio Analysis, examined after the initial analysis, showed tremendous potential for use in the CTSA context.

Conclusion

Despite challenges in making cross-CTSA comparisons, bibliometrics can enhance our understanding of the value of CTSA-supported clinical and translational research.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Feasibility of common bibliometrics in evaluating translational science
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Feasibility of common bibliometrics in evaluating translational science
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Feasibility of common bibliometrics in evaluating translational science
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Pressmust be obtained for commercial re-use or in order to create a derivative work.

Corresponding author

*Address for correspondence: M. Schneider, Ph.D., 258 Social Ecology I, University of California, Irvine, Irvine, CA 92697, USA. (Email: mls@uci.edu)

References

Hide All
1. National Institutes of Health. Institutional Clinical and Translational Science Award [Internet]. 2008 [cited Mar 18, 2016]. (http://grants.nih.gov/grants/guide/rfa-files/RFA-RM-08-002.html)
2. Dilts, DM, Rosenblum, D, Trochim, WM. A virtual national laboratory for reengineering clinical translational science. Science Translational Medicine 2012; 4: 118cm2.
3. Rubio, DM, et al. Developing common metrics for the Clinical and Translational Science Awards (CTSAs): lessons learned. Clinical and Translational Science 2015; 8: 451459.
4. Trochim, WM, et al. Evaluation guidelines for the Clinical and Translational Science Awards (CTSAs). Clinical and Translational Science 2013; 6: 303309.
5. Scott, CS, et al. Expanding assessments of translational research programs: supplementing metrics with value judgments. Evaluation & the Health Professions 2014; 37: 8397.
6. Pincus, HA, et al. Evaluation and the NIH Clinical and Translational Science Awards: a “top ten” list. Evaluation & the Health Professions 2013; 36: 411431.
7. Madlock-Brown, C, Eichmann, D. Assessing the translational capacity of five CTSA institutions, 2015. Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015; 768769.
8. Thomson Reuters. Using Bibliometrics: A Guide to Evaluating Research Performance with Citation Data. Philadelphia, PA: Thomson Reuters, 2008.
9. Broadus, RN. Early approaches to bibliometrics. Journal of the American Society for Information Science 1987; 38: 127129.
10. Garfield, E. Citation Indexing: Its Theory and Application in Science, Technology, and Humanities. New York: Wiley, 1979.
11. Garfield, E. Commentary—fifty years of citation indexing. International Journal of Epidemiology 2006; 35: 11271128.
12. Rosas, SR, et al. Evaluating research and impact: a bibliometric analysis of research by the NIH/NIAID HIV/AIDS clinical trials networks. PLoS One 2011; 6: e17428.
13. Trochim, WM, et al. The evaluation of large research initiatives—a participatory integrative mixed-methods approach. American Journal of Evaluation 2008; 29: 828.
14. Donovan, C, et al. Evaluation of the impact of National Breast Cancer Foundation-funded research. Med J Aust. 2014; 3;200(4): 214–218.
15. Sengupta, IN. Bibliometrics, informetrics, scientometrics and librametrics—an overview. Libri 1992; 42: 7598.
16. Adams, J. Bibliometrics: the citation game. Nature 2014; 510: 470471.
17. Marx, W, Bornmann, L. On the causes of subject-specific citation rates in Web of Science. Scientometrics 2014; 102: 18231827.
18. Campbell, P. Escape from the impact factor. Ethics in Science and Environmental Politics 2008; 8: 56.
19. Schekman, R. How journals like Nature, Cell and Science are damaging science. The Guardian 2013; 9: 12.
20. Werner, R. The focus on bibliometrics makes papers less useful. Nature 2015; 517: 245.
21. Schekman, R. Reforming research assessment. Elife 2013; 2: e00855.
22. Colledge, L, Verlinde, R. SciVal Metrics Guidebook [Internet]. Amsterdam: Elsevier, 2014. (https://www.elsevier.com/research-intelligence/resource-library/scival-metrics-guidebook).
23. Hutchins, BI, et al. Relative Citation Ratio (RCR): a new metric that uses citation rates to measure influence at the article level. PLOS Biology 2015; 14: e1002541. doi:10.1371/journal.pbio.1002541.
24. Slaughter, A, Frechtling, J. Initial Report on CTSA-Supported Publications: 2007–2010. Westat Report to the National Center for Research Resources . Rockville, MD: Westat, 2011.
25. Rafols, I, et al. How journal rankings can suppress interdisciplinary research: a comparison between innovation studies and business & management. Research Policy 2012; 41: 12621282.
26. Lauer, MS. Thought exercises on accountability and performance measures at the National Heart, Lung, and Blood Institute (NHLBI): an invited commentary for circulation research. Circulation Research 2011; 108: 405409.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Clinical and Translational Science
  • ISSN: -
  • EISSN: 2059-8661
  • URL: /core/journals/journal-of-clinical-and-translational-science
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed