3 results
4549 Reproducible Informatics for Reproducible Translational Research
- Ram Gouripeddi, Katherine Sward, Mollie Cummins, Karen Eilbeck, Bernie LaSalle, Julio C. Facelli
-
- Journal:
- Journal of Clinical and Translational Science / Volume 4 / Issue s1 / June 2020
- Published online by Cambridge University Press:
- 29 July 2020, pp. 66-67
-
- Article
-
- You have access Access
- Open access
- Export citation
-
OBJECTIVES/GOALS: Characterize formal informatics methods and approaches for enabling reproducible translational research. Education of reproducible methods to translational researchers and informaticians. METHODS/STUDY POPULATION: We performed a scoping review [1] of selected informatics literature (e.g. [2,3]) from PubMed and Scopus. In addition we reviewed literature and documentation of translational research informatics projects [4–21] at the University of Utah. RESULTS/ANTICIPATED RESULTS: The example informatics projects we identified in our literature covered a broad spectrum of translational research. These include research recruitment, research data requisition, study design and statistical analysis, biomedical vocabularies and metadata for data integration, data provenance and quality, and uncertainty. Elements impacting reproducibility of research include (1) Research Data: its semantics, quality, metadata and provenance; and (2) Research Processes: study conduct including activities and interventions undertaken, collections of biospecimens and data, and data integration. The informatics methods and approaches we identified as enablers of reproducibility include the use of templates, management of workflows and processes, scalable methods for managing data, metadata and semantics, appropriate software architectures and containerization, convergence methods and uncertainty quantification. In addition these methods need to be open and shareable and should be quantifiable to measure their ability to achieve reproducibility. DISCUSSION/SIGNIFICANCE OF IMPACT: The ability to collect large volumes of data collection has ballooned in nearly every area of science, while the ability to capturing research processes hasn’t kept with this pace. Potential for problematic research practices and irreproducible results are concerns.
Reproducibility is a core essentially of translational research. Translational research informatics provides methods and means for enabling reproducibility and FAIRness [22] in translational research. In addition there is a need for translational informatics itself to be reproducible to make research reproducible so that methods developed for one study or biomedical domain can be applied elsewhere. Such informatics research and development requires a mindset for meta-research [23].
The informatics methods we identified covers the spectrum of reproducibility (computational, empirical and statistical) and across different levels of reproducibility (reviewable, replicable, confirmable, auditable, and open or complete) [24–29]. While there are existing and ongoing efforts in developing informatics methods for translational research reproducibility in Utah and elsewhere, there is a need to further develop formal informatics methods and approaches: the Informatics of Research Reproducibility.
In this presentation, we summarize the studies and literature we identified and discuss our key findings and gaps in informatics methods for research reproducibility. We conclude by discussing how we are covering these topics in a translational research informatics course.
1. Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: advancing the approach and enhancing the consistency. Res Synth Methods. 2014 Dec;5(4):371–85.
2. McIntosh LD, Juehne A, Vitale CRH, Liu X, Alcoser R, Lukas JC, Evanoff B. Repeat: a framework to assess empirical reproducibility in biomedical research. BMC Med Res Methodol [Internet]. 2017 Sep 18 [cited 2018 Nov 30];17. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5604503/
3. Denaxas S, Direk K, Gonzalez-Izquierdo A, Pikoula M, Cakiroglu A, Moore J, Hemingway H, Smeeth L. Methods for enhancing the reproducibility of biomedical research findings using electronic health records. BioData Min. 2017;10:31.
4. Burnett N, Gouripeddi R, Wen J, Mo P, Madsen R, Butcher R, Sward K, Facelli JC. Harmonization of Sensor Metadata and Measurements to Support Exposomic Research. In: 2016 International Society of Exposure Science [Internet]. Research Triangle Park, NC, USA; 2017 [cited 2017 Jun 17]. Available from: http://www.intlexposurescience.org/ISES2017
5. Butcher R, Gouripeddi RK, Madsen R, Mo P, LaSalle B. CCTS Biomedical Informatics Core Research Data Service. In Salt Lake City; 2016.
6. Cummins M, Gouripeddi R, Facelli J. A low-cost, low-barrier clinical trials registry to support effective recruitment. In Salt Lake City, Utah, USA; 2016 [cited 2018 Nov 30]. Available from: //campusguides.lib.utah.edu/UtahRR16/abstracts
7. Gouripeddi R, Warner P, Madsen R, Mo P, Burnett N, Wen J, Lund A, Butcher R, Cummins MR, Facelli J, Sward K. An Infrastructure for Reproducibile Exposomic Research. In: Research Reproducibility 2016 [Internet]. Salt Lake City, Utah, USA; 2016 [cited 2018 Nov 30]. Available from: //campusguides.lib.utah.edu/UtahRR16/abstracts
8. Eilbeck K, Lewis SE, Mungall CJ, Yandell M, Stein L, Durbin R, Ashburner M. The Sequence Ontology: a tool for the unification of genome annotations. Genome Biol. 2005;6:R44.
9. Gouripeddi R, Cummins M, Madsen R, LaSalle B, Redd AM, Presson AP, Ye X, Facelli JC, Green T, Harper S. Streamlining study design and statistical analysis for quality improvement and research reproducibility. J Clin Transl Sci. 2017 Sep;1(S1):18–9.
10. Gouripeddi R, Eilbeck K, Cummins M, Sward K, LaSalle B, Peterson K, Madsen R, Warner P, Dere W, Facelli JC. A Conceptual Architecture for Reproducible On-demand Data Integration for Complex Diseases. In: Research Reproducibility 2016 (UtahRR16) [Internet]. Salt Lake City, Utah, USA; 2016 [cited 2017 Apr 25]. Available from: https://zenodo.org/record/168067
11. Gouripeddi R, Lane E, Madsen R, Butcher R, LaSalle B, Sward K, Fritz J, Facelli JC, Cummins M, Shao J, Singleton R. Towards a scalable informatics platform for enhancing accrual into clinical research studies. J Clin Transl Sci. 2017 Sep;1(S1):20–20.
12. Gouripeddi R, Deka R, Reese T, Butcher R, Martin B, Talbert J, LaSalle B, Facelli J, Brixner D. Reproducibility of Electronic Health Record Research Data Requests. In Washington, DC, USA; 2018 [cited 2018 Apr 21]. Available from: https://zenodo.org/record/1226602#.WtvvyZch270
13. Gouripeddi R, Mo P, Madsen R, Warner P, Butcher R, Wen J, Shao J, Burnett N, Rajan NS, LaSalle B, Facelli JC. A Framework for Metadata Management and Automated Discovery for Heterogeneous Data Integration. In: 2016 BD2K All Hands Meeting [Internet]. Bethesda, MD; November 29-30 [cited 2017 Apr 25]. Available from: https://zenodo.org/record/167885
14. Groat D, Gouripeddi R, Lin YK, Dere W, Murray M, Madsen R, Gestaland P, Facelli J. Identification of High-Level Formalisms that Support Translational Research Reproducibility. In: Research Reproducibility 2018 [Internet]. Salt Lake City, Utah, USA; 2018 [cited 2018 Oct 30]. Available from: //campusguides.lib.utah.edu/UtahRR18/abstracts
15. Huser V, Kahn MG, Brown JS, Gouripeddi R. Methods for examining data quality in healthcare integrated data repositories. Pac Symp Biocomput Pac Symp Biocomput. 2018;23:628–33.
16. Lund A, Gouripeddi R, Burnett N, Tran L-T, Mo P, Madsen R, Cummins M, Sward K, Facelli J. Enabling Reproducible Computational Modeling: The Utah PRISMS Ecosystem. In Salt Lake City, Utah, USA; 2018 [cited 2018 Oct 30]. Available from: //campusguides.lib.utah.edu/UtahRR18/abstracts
17. Pflieger LT, Mason CC, Facelli JC. Uncertainty quantification in breast cancer risk prediction models using self-reported family health history. J Clin Transl Sci. 2017 Feb;1(1):53–9.
18. Shao J, Gouripeddi R, Facelli J. Improving Clinical Trial Research Reproducibility using Reproducible Informatics Methods. In Salt Lake City, Utah, USA; 2018 [cited 2018 Oct 30]. Available from: //campusguides.lib.utah.edu/UtahRR18/abstracts
19. Shao J, Gouripeddi R, Facelli JC. Semantic characterization of clinical trial descriptions from ClincalTrials.gov and patient notes from MIMIC-III. J Clin Transl Sci. 2017 Sep;1(S1):12–12.
20. Tiase V, Gouripeddi R, Burnett N, Butcher R, Mo P, Cummins M, Sward K. Advancing Study Metadata Models to Support an Exposomic Informatics Infrastructure. In Ottawa, Canada; 2018 [cited 2018 Oct 30]. Available from: = http://www.eiseverywhere.com/ehome/294696/638649/?&t=8c531cecd4bb0a5efc6a0045f5bec0c3
21. Wen J, Gouripeddi R, Facelli JC. Metadata Discovery of Heterogeneous Biomedical Datasets Using Token-Based Features. In: IT Convergence and Security 2017 [Internet]. Springer, Singapore; 2017 [cited 2017 Sep 6]. p. 60–7. (Lecture Notes in Electrical Engineering). Available from: https://link.springer.com/chapter/10.1007/978-981-10-6451-7_8
22. Wilkinson MD, Dumontier M, Aalbersberg IjJ, Appleton G, Axton M, Baak A, Blomberg N, Boiten J-W, da Silva Santos LB, Bourne PE, Bouwman J, Brookes AJ, Clark T, Crosas M, Dillo I, Dumon O, Edmunds S, Evelo CT, Finkers R, Gonzalez-Beltran A, Gray AJG, Groth P, Goble C, Grethe JS, Heringa J, ’t Hoen PAC, Hooft R, Kuhn T, Kok R, Kok J, Lusher SJ, Martone ME, Mons A, Packer AL, Persson B, Rocca-Serra P, Roos M, van Schaik R, Sansone S-A, Schultes E, Sengstag T, Slater T, Strawn G, Swertz MA, Thompson M, van der Lei J, van Mulligen E, Velterop J, Waagmeester A, Wittenburg P, Wolstencroft K, Zhao J, Mons B. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016 Mar 15;3:160018.
23. Ioannidis JPA. Meta-research: Why research on research matters. PLOS Biol. 2018 Mar 13;16(3):e2005468.
24. Stodden V, Borwein J, Bailey DH. Setting the default to reproducible. Comput Sci Res SIAM News. 2013;46(5):4–6.
25. Stodden V, McNutt M, Bailey DH, Deelman E, Gil Y, Hanson B, Heroux MA, Ioannidis JPA, Taufer M. Enhancing reproducibility for computational methods. Science. 2016 Dec 9;354(6317):1240–1.
26. Stodden V, McNutt M, Bailey DH, Deelman E, Gil Y, Hanson B, Heroux MA, Ioannidis JPA, Taufer M. Enhancing reproducibility for computational methods. Science. 2016 Dec 9;354(6317):1240–1.
27. Stodden V. Reproducible Research for Scientific Computing: Tools and Strategies for Changing the Culture. Comput Sci Eng. 2012 Jul 1;14(4):13–7.
28. Baker M. Muddled meanings hamper efforts to fix reproducibility crisis. Nat News Available from: http://www.nature.com/news/muddled-meanings-hamper-efforts-to-fix-reproducibility-crisis-1.20076
29. Barba LA. Terminologies for Reproducible Research. ArXiv180203311 Cs 2018 Feb 9; Available from: http://arxiv.org/abs/1802.03311
3339 Development of a Competency-based Informatics Course for Translational Researchers
- Ram Gouripeddi, Danielle Groat, Samir E. Abdelrahman, Tom Cheatham, Mollie Cummins, Karen Eilbeck, Bernie LaSalle, Katherine Sward, Julio C. Facelli
-
- Journal:
- Journal of Clinical and Translational Science / Volume 3 / Issue s1 / March 2019
- Published online by Cambridge University Press:
- 26 March 2019, pp. 66-67
-
- Article
-
- You have access Access
- Open access
- Export citation
-
OBJECTIVES/SPECIFIC AIMS: Translational researchers often require the use of informatics methods in their work. Lack of an understanding of key informatics principles and methods limits the abilities of translational researchers to successfully implement Findable, Accessible, Interoperable, Reusable (FAIR) principles in grant proposal submissions and performed studies. In this study we describe our work in addressing this limitation in the workforce by developing a competency-based, modular course in informatics to meet the needs of diverse translational researchers. METHODS/STUDY POPULATION: We established a Translational Research Informatics Education Collaborative (TRIEC) consisting of faculty at the University of Utah (UU) with different primary expertise in informatics methods, and working in different tiers of the translational spectrum. The TRIEC, in collaboration with the Foundation of Workforce Development of the Utah Center for Clinical and Translational Science (CCTS), gathered informatics needs of early investigators by consolidating requests for informatics services, assistance provided in grant writing, and consultations. We then reviewed existing courses and literature for informatics courses that focused on clinical and translational researchers [3–9]. Using the structure and content of the identified courses, we developed an initial draft of a syllabus for a Translational Research Informatics (TRI) course which included key informatics topics to be covered and learning activities, and iteratively refined it through discussions. The course was approved by the UU Department of Biomedical Informatics, UU Graduate School and the CCTS. RESULTS/ANTICIPATED RESULTS: The TRI course introduces informatics PhD students, clinicians, and public health practitioners who have a demonstrated interest in research, to fundamental principles and tools of informatics. At the completion of the course, students will be able to describe and identify informatics tools and methods relevant to translational research and demonstrate inter-professional collaboration in the development of a research proposal addressing a relevant translational science question that utilizes the state-of-the-art in informatics. TRI covers a diverse set of informatics content presented as modules: genomics and bioinformatics, electronic health records, exposomics, microbiomics, molecular methods, data integration and fusion, metadata management, semantics, software architectures, mobile computing, sensors, recruitment, community engagement, secure computing environments, data mining, machine learning, deep learning, artificial intelligence and data science, open source informatics tools and platforms, research reproducibility, and uncertainty quantification. The teaching methods for TRI include (1) modular didactic learning consisting of presentations and readings and face-to-face discussions of the content, (2) student presentations of informatics literature relevant to their final project, and (3) a final project consisting of the development, critique and chalk talk and formal presentations of informatics methods and/or aims of an National Institutes of Health style K or R grant proposal. For (3), the student presents their translational research proposal concept at the beginning of the course, and works with members of the TRIEC with corresponding expertise. The final course grade is a combination of the final project, paper presentations and class participation. We offered TRI to a first cohort of students in the Fall semester of 2018. DISCUSSION/SIGNIFICANCE OF IMPACT: Translational research informatics is a sub-domain of biomedical informatics that applies and develops informatics theory and methods for translational research. TRI covers a diverse set of informatics topics that are applicable across the translational spectrum. It covers both didactic material and hands-on experience in using the material in grant proposals and research studies. TRI’s course content, teaching methodology and learning activities enable students to initially learn factual informatics knowledge and skills for translational research correspond to the ‘Remember, Understand, and Apply’ levels of the Bloom’s taxonomy [10]. The final project provides opportunity for applying these informatics concepts corresponding to the ‘Analyze, Evaluate, and Create’ levels of the Bloom’s taxonomy [10]. This inter-professional, competency-based, modular course will develop an informatics-enabled workforce trained in using state-of-the-art informatics solutions, increasing the effectiveness of translational science and precision medicine, and promoting FAIR principles in research data management and processes. Future work includes opening the course to all Clinical and Translational Science Award hubs and publishing the course material as a reference book. While student evaluations for the first cohort will be available end of the semester, true evaluation of TRI will be the number of trainees taking the course and successful grant proposal submissions. References: 1. Wilkinson MD, Dumontier M, et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data. 2016 Mar 15. 2. National Center for Advancing Translational Sciences. Translational Science Spectrum. National Center for Advancing Translational Sciences. 2015 [cited 2018 Nov 15]. Available from: https://ncats.nih.gov/translation/spectrum 3. Hu H, Mural RJ, Liebman MN. Biomedical Informatics in Translational Research. 1 edition. Boston: Artech House; 2008. 264 p. 4. Payne PRO, Embi PJ, Niland J. Foundational biomedical informatics research in the clinical and translational science era: a call to action. J Am Med Inform Assoc JAMIA. 2010;17(6):615–6. 5. Payne PRO, Embi PJ, editors. Translational Informatics: Realizing the Promise of Knowledge-Driven Healthcare. Softcover reprint of the original 1st ed. 2015 edition. Springer; 2016. 196 p. 6. Richesson R, Andrews J, editors. Clinical Research Informatics. 2nd ed. Springer International Publishing; 2019. (Health Informatics). 7. Robertson D, MD GHW, editors. Clinical and Translational Science: Principles of Human Research. 2 edition. Amsterdam: Academic Press; 2017. 808 p. 8. Shen B, Tang H, Jiang X, editors. Translational Biomedical Informatics: A Precision Medicine Perspective. Softcover reprint of the original 1st ed. 2016 edition. S.l.: Springer; 2018. 340 p. 9. Valenta AL, Meagher EA, Tachinardi U, Starren J. Core informatics competencies for clinical and translational scientists: what do our customers and collaborators need to know? J Am Med Inform Assoc. 2016 Jul 1;23(4):835–9. 10. Anderson LW, Krathwohl DR, Airasian PW, Cruikshank KA, Mayer RE, Pintrich PR, Raths J, Wittrock MC. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives, Abridged Edition. 1 edition. New York: Pearson; 2000.
3399 Systematically Integrating Microbiomes and Exposomes for Translational Research
- Ram Gouripeddi, Andrew Miller, Karen Eilbeck, Katherine Sward, Julio C. Facelli
-
- Journal:
- Journal of Clinical and Translational Science / Volume 3 / Issue s1 / March 2019
- Published online by Cambridge University Press:
- 26 March 2019, pp. 29-30
-
- Article
-
- You have access Access
- Open access
- Export citation
-
OBJECTIVES/SPECIFIC AIMS: Characterize microbiome metadata describing specimens collected, genomic pipelines and microbiome results, and incorporate them into a data integration platform for enabling harmonization, integration and assimilation of microbial genomics with exposures as spatiotemporal events. METHODS/STUDY POPULATION: We followed similar methods utilized in previous efforts in charactering and developing metadata models for describing microbiome metadata. Due to the heterogeneity in microbiome and exposome data, we aligned them along a conceptual representation of different data used in translational research; microbiomes being biospecimen-derived, and exposomes being a combination of sensor measurements, surveys and computationally modelled data. We performed a review of literature describing microbiome data, metadata, and semantics [4–15], along with existing datasets [16] and developed an initial metadata model. We reviewed the model with microbiome domain experts for its accuracy and completeness, and with translational researchers for its utility in different studies, and iteratively refined it. We then incorporated the logical model into OpenFurther’s metadata repository MDR [17,18] for harmonization of different microbiome datasets, as well as integration and assimilation of microbiome-exposome events utilizing the UPIE. RESULTS/ANTICIPATED RESULTS: Our model for describing the microbiome currently includes three domains (1) the specimen collected for analysis, (2) the microbial genomics pipelines, and (3) details of the microbiome genomics. For (1), we utilized biospecimen data model that harmonizes the data structures of caTissue, OpenSpecimen and other commonly available specimen management platform. (3) includes details about the organisms, isolate, host specifics, sequencing methodology, genomic sequences and annotations, microbiome phenotype, genomic data and storage, genomic copies and associated times stamps. We then incorporated this logical model into the MDR as assets and associations that UPIE utilizes to harmonize different microbiome datasets, followed by integration and assimilation of microbiome-exposome events. Details of (2) are ongoing. DISCUSSION/SIGNIFICANCE OF IMPACT: The role of the microbiome and co-influences from environmental exposures in etio-pathology of various pulmonary conditions isn’t well understood [19–24]. This metadata model for the microbiome provides a systematic approach for integrating microbial genomics with sensor-based environmental and physiological data, and clinical data that are present in varying spatial and temporal granularities and require complex methods for integration, assimilation and analysis. Incorporation of this microbiome model will advance the performance of sensor-based exposure studies of the (UPIE) to support novel research paradigms that will improve our understanding of the role of microbiome in promoting and preventing airway inflammation by performing a range of hypothesis-driven microbiome-exposome pediatric asthma studies across the translational spectrum.
![](/core/cambridge-core/public/images/lazy-loader.gif)