Introduction
In 2016, Europe counted 1.7 million deaths under 75 years of age, of which 1.2 million could have been avoided with the translation of scientific knowledge into medical practice, adequate prevention, and public health interventions (Hegyi et al. Reference Hegyi, Petersen, Holgate, Erőss, Garami, Szakács, Dobszai, Balaskó, Kemény, Peng, Monteiro, Varró, Lamont, Laurence, Gray, Pickles, FitzGerald, Griffiths, Jassem, Rusakov and Szentesi2020; Eurostat 2023). To improve this ratio, physicians need to understand the scientific language and translate scientific knowledge into everyday practice (Hegyi et al. Reference Hegyi, Erőss, Izbéki, Párniczky and Szentesi2021). To move forward from the current situation, we first need to understand where the translation is blocked and what the difficulties are, and then identify opportunities for improvement.
Advances in medical science have always modified patient care. In the 19th century, physicians treated patients on the basis of individual ideas, and fiction without scientific experience. For example, in the past, tuberculosis was treated with remedies such as bleeding and purging, which probably contributed to high mortality (Iseman Reference Iseman2002). In the 20th century, scientists started to understand how the human body, organs, and cells work. Basic scientists developed lots of new methodologies and generated large amounts of new data and knowledge; however, clinical scientific activities were very immature. During this period, physicians primarily developed therapeutic protocols that could be logically deduced from basic research findings, regardless of whether they had ever been proven in patients. For example, in the treatment of acute pancreatitis, patients were regularly administered gastric acid suppression to reduce the stimuli of the pancreas, antibiotics to prevent infection, and still a nil-per-os (nil-by-mouth) diet was common to leave the pancreas at rest (Bradley Reference Bradley1989). Importantly, these approaches were later discarded based on the results of clinical trials (Demcsák Reference Demcsák, Soós, Kincses, Capunge, Minkov, Kovacheva-Slavova, Nakov, Wu, Huang, Xia, Deng, Hollenbach, Schneider, Hirth, Ioannidis, Vincze, Bajor, Sarlós, Czakó, Illés and Hegyi2020; Párniczky Reference Párniczky, Lantos, Tóth, Szakács, Gódi, Hágendorn, Illés, Koncz, Márta, Mikó, Mosztbacher, Németh, Pécsi, Szabó, Szücs, Varjú, Szentesi, Darvasi, Erőss and Izbéki2019). From the late 1900s onwards, there was a growing need for clinical research that could directly improve patient care. The quality of clinical research has developed and numerous new methodologies have emerged (e.g., meta-analysis – combining the findings of different studies on the same question, randomized clinical trials, etc.). A wealth of new knowledge has been generated and a large number of evidence-based medicine (EBM) guidelines have been published. This century has seen the number of clinical science articles increase exponentially and this trend has had a visible impact on patient care.
In the 21st century, high-quality patient care can only be achieved through the use of EBM guidelines. Today, two significant parameters can indicate an improvement in quality. The first is the quality indicator, namely the impact factor (IF), which is steadily increasing for clinical scientific journals. The other is quality control, which implies adherence to registration, pre-study protocols, and reporting guidelines. Importantly, IF is by far not the most ideal indicator of quality, as it is field-specific and can be easily influenced by editorial policies. In order to disseminate accurate and objective evaluation criteria for research quality, a scientific group has developed a set of recommendations, referred to as the San Francisco Declaration on Research Assessment (DORA) in 2013 (DORA 2023). In addition, the strengthening of the IT sector is making the use of more sophisticated, well-structured databases increasingly available. However, there is considerable evidence to suggest that the translation of these scientific data into everyday practice is very limited (Demcsák Reference Demcsák, Soós, Kincses, Capunge, Minkov, Kovacheva-Slavova, Nakov, Wu, Huang, Xia, Deng, Hollenbach, Schneider, Hirth, Ioannidis, Vincze, Bajor, Sarlós, Czakó, Illés and Hegyi2020; Párniczky Reference Párniczky, Lantos, Tóth, Szakács, Gódi, Hágendorn, Illés, Koncz, Márta, Mikó, Mosztbacher, Németh, Pécsi, Szabó, Szücs, Varjú, Szentesi, Darvasi, Erőss and Izbéki2019; Zádori Reference Zádori, Párniczky, Szentesi and Hegyi2020). A clear pathway for implementing the latest discoveries, especially for senior doctors with conservative attitudes, remains unclear.
While medical schools primarily teach the basics of medicine to undergraduate students, doctoral schools teach scientific methodologies and provide opportunities for PhD students to pursue scientific activities. The question arises whether university doctoral schools certainly facilitate the training of science-savvy physicians. Therefore, in this study, we aimed to investigate (1) the quality and quantity of basic and clinical research by analysing ten doctoral schools, (2) to identify strengths, weaknesses, opportunities, and threats (SWOT), and finally (3) to set up specific, measurable, achievable, relevant and time-oriented (SMART) goals for improvement.
In this study, the results support our hypothesis that there is a huge lack of education and implementation of new clinical methodologies in PhD doctoral schools, which probably slows down translation of knowledge to the benefit of patients.
Methods
Data Sources
Available data from ten Hungarian doctoral schools at two medical universities were analysed. Data were taken from the official website of the Hungarian Doctoral Council (2023). The Medical Faculties of the University of Szeged (USZ) and the University of Pécs (UP) run ten doctoral schools (Table 1).
Summary of basic characteristics of supervisors and core members and the number of their PhD graduates in each doctoral school each year and in 5 years.
Age is expressed in mean±SD, UP: University of Pécs; USZ: University of Szeged.
The following data were collected: core members (distinguished supervisors on the basis of their outstanding scientific record, usually with an associate professor or professor position) and general supervisors of the doctoral school, the number of PhD students, core members, and supervisors in the periods of 2008–2012 and 2013–2017, and details of all PhD theses between 2013 and 2017. The institution and type of research were divided into theoretical (basic), clinical, and mixed categories.
The term ‘theoretical institution’ refers to departments teaching basic subjects and conducting basic research that does not require patient care (e.g., physiology, biochemistry). Basic research investigates how different processes work and helps to understand particular phenomena (i.e., discovery research). The term ‘clinical institution’ refers to departments where patient care and clinical research are carried out, where humans are involved in the investigations, and can be either observational studies or clinical trials (e.g., internal medicine, surgery). The term mixed institution refers to departments where these medical approaches are combined (e.g., genetics, microbiology).
Measurement of Research Quality
Although there is no suitable model for measuring the quality of articles, it is widely accepted to use scientometric parameters calculated from citation numbers of published articles. We used two independent systems: (1) the impact factor (IF), calculated by the Web of Science Group Journal Citation Reports (JCR), and (2) the SCImago Journal and Country Rank (SJR), which is freely available, to access ranking and individual analyses of journals grouped by research area or country (SJR 2023).
Measurement of Research Quantity
In Hungary, PhD students are required to have first- and co-author publications before completing their studies. Therefore, we collected the publications in PhD theses and analysed their methodology.
General Data Protection Regulation (GDPR)
The General Data Protection Regulation (GDPR) was followed during data collection. All data are available on the official website of the Hungarian Doctoral Council (see https://doktori.hu). The ages of core members and supervisors were collected from doctoral schools.
Statistical Analysis
Descriptive statistical methods were used to demonstrate prevalence data. The chi-square goodness-of-fit test was applied to test whether two categories of a discrete variable were equiprobable. To test the homogeneity of two discrete distributions, a Fisher-exact test was used. As a post-hoc analysis, in case of significant overall difference, Fisher-exact tests were performed separately for each category. To address the problem of multiple comparisons, the Holm-adjusted p-values were reported, as appropriate (Holm Reference Holm1979). Statistical analyses were performed using the statistical program R (version 4.1.2.). For all statistical analyses, a p-value of less than 0.05 was considered significant.
Results
One-third of Core Members and Supervisors had no PhD Students over a 10-year-Period
Over a 5-year-period, a large proportion of core members and supervisors had no graduated PhD students (53% from 2008 to 2012, 37% from 2013 to 2017, Figure 1). Moreover, 28% of the mentors had no graduated student in the investigated 10-year period, indicating low activity of the listed supervisors. Another characteristic was that there was a definite male predominance (72%) and a relatively high average age (52.47±10.34 years) among mentors. Details can be found in Table 1.
Distribution of graduated PhD students per supervisor.
A Total of 54% of Physicians Pursue their PhD Studies in Basic Science in Theoretical Institutions
More PhD degrees were awarded to students affiliated with theoretical than clinical institutions (56% n = 267 vs. 44% n = 200). A total of 87% of PhD students affiliated with a theoretical institution did their research in the basic science category and only 38% of the PhD students affiliated with a clinical institution did their PhD in the clinical science category. In both theoretical and clinical institutions, students were significantly more likely (<0.001) to do basic research (χ 2 = 177.37, df = 1 χ 2 = 6.29, df = 1, p = 0.012, respectively), suggesting that neither the infrastructure nor knowledge and willingness of mentors were attractive to students (Figure 2). Details can be found in Table A1 in the Supplementary Material.
Distribution of PhD graduates by type of institution and subject. Percentages mean the distribution of PhD students by types of scientific fields and mentoring institutions. In theoretical institutions, 87% (n = 231) and in clinical institutions, 55% (n = 109) of the students had a basic science topic
Both Quantity and Quality of Clinical Science Papers were Lower than those for Basic Science Ones
A total of 1398 articles were reviewed, 56 of which were not listed in the SCImago journal ranking; therefore, these articles were not included in this part of the evaluation. Of the 1342 reviewed articles, 990 (74%) were publications on basic science and only 264 (20%) were on clinical science, indicating a very low level of clinical science activity at postgraduate level. Overall, most publications (59%) were ranked as Q1 category (Figure 3, Table A2 in the Supplementary Material). In basic science, there were significantly more Q1 publications than others (p<0.001). In clinical science, however, there was no significant difference between Q1 publications and other ranking categories (p = 0.389).
Distribution of publications by the SCImago Journal Ranking (SJR) and impact factors in basic and clinical science
The majority of the papers (n = 456, 32%) were published in journals with an IF of 2–3. A total of 15% of basic publications were published in a journal with 4+ IF, whereas this ratio was only 8% in clinical science (p = 0.006). As clinical journals tend to have a higher IF than basic journals, these data suggest a lack of high-quality publications in clinical science (Figure 4). Further details on the distribution of SJR and IF of articles can be found in Table A3 in the Supplementary Material.
Distribution of the methodologies between 2013 and 2017
Modern Clinical Methodologies are Rarely Used in Clinical Scientific Papers
To understand the above-mentioned phenomenon, we examined the different methodologies used in the clinical science dissertations between 2013 and 2017.
The vast majority of research methodologies were from the oldest ones, such as retrospective data analyses (90/272, 30%) and cross-sectional studies (129/272; 43%). However, the newest clinical methodologies such as randomized clinical trial or meta-analysis were used extremely rarely (4/272; 1% and 2/272; 1%, respectively). The rate of sample size calculation in clinical research was zero and, in addition, only 5% (n = 4) of prospective clinical trials were registered as prospective studies. Unfortunately, 90% of studies were single-centre, and only 5% of the publications were the result of multi-centre investigations (Figure 4).
Blended Education as a Future Approach
In 2018, a hybrid PhD programme was launched with a completely new approach and structure in Hungary. In 2019/2020, 20, in 2020/2021, 17, in 2021/2022, 91, and in 2022/2023, 83 doctoral students applied, indicating the interest of the students and the evolution of the programme. This is a five-level hybrid educational approach that combines traditional, face-to-face instructions with online learning materials, including face-to-face seminar lectures and online project meetings and e-learning. The educational module provides students with the opportunity to engage in patient care and academic activity simultaneously. Within the framework of training, they acquire clinical research methodologies using the ‘learning by doing’ approach through independent scientific projects. They are provided with the opportunity to join workgroups, participate in meta-analyses, studies related to different registers, and in clinical work, and they will gain an understanding of clinical research design. The programme helps students to become critical consumers of medical research papers, to collect primary data on health issues by interviewing and observing patients, and to conduct biomedical research with high efficiency. The blended learning module is detailed in the latest Yearbook (Kocsis et al. Reference Kocsis, Tichyné, Attila, Dóra, Czapári, Kerekes, Nagy and Váncsa2023).
Discussion
The aim of this study was to understand how doctoral schools help to translate scientific knowledge into community benefits. Here, we discuss our results by demonstrating the strengths, weaknesses, opportunities, and threats (SWOT) of the system and we will suggest some ways of how to set up specific, measurable, achievable, relevant, and time-oriented (SMART) goals (Bailey Reference Bailey2017; Teoli Reference Teoli, Sanvictores and An2023).
SWOT Analysis
Strengths: One of the strengths of the Hungarian doctoral school system is the wide range of research possibilities available both in basic and clinical science categories. Doctoral schools employ an adequate number of core members and supervisors. To complete their studies, students are required to publish quality articles with a relatively high IF, which is a rarity in Europe. The publication of basic science articles reaches a very good international standard. The number of students and supervisors, and the quality of publications are assessed in a database, which allows transparency and easy follow-up.
Weakness: There is a significant difference in the gender balance of core members and supervisors with a high male predominance (72%). Moreover, our results showed that the average age of supervisors was 53. We investigated the background of this phenomenon and found that the regulations slowed down the entry of young researchers into doctoral schools due to a difficult application and admission system. However, in contrast to the difficult entry system, the quality and activity of supervisors are not investigated any longer. Once someone entered, he/she is unlikely to be excluded from the supervisory board. Another weakness is the quality and quantity of publications. Clinical research is behind the international high-quality standard. Quality clinical research elements and parameters are fairly well adopted; as can be seen from the results, there were no studies that involved sample size estimation, most clinical trials were not registered and the majority of studies involved only a single centre, including fewer than 100 patients. Some doctoral schools do not accept the latest clinical methodologies, such as meta-analysis, regardless of whether they generate new knowledge in the field.
Opportunities: In 2016, the University of Pécs established the Centre for Translational Medicine (CTM) to teach science methodologies and to build bridges between clinical and basic researchers. The CTM has created a unique interdisciplinary unit to support methodology education, the creation of structural clinical databases, and the analysis of large clinical data. Within a relatively short period, a structured methodology education system for PhD students and their supervisors has generated a large increase in scientific activity. Publication activity has increased tenfold in 5 years, with CTM results published in Nature Medicine, one of the most prestigious medical journals (Hegyi et al. Reference Hegyi, Erőss, Izbéki, Párniczky and Szentesi2021). The CTM at UP has demonstrated that paradigm shifts in doctoral schools can generate a huge change in activity.
Threats: Universities are usually very cautious about making big changes to their systems. Therefore, there is a risk of ‘no change’ in doctoral schools; which would mean that physicians will not have the chance to learn how to analyse scientific papers critically; therefore, the translation of scientific knowledge into healthcare would stay very slow. In addition, the quality of doctoral schools would remain at the same level, without sufficient innovations in doctoral training. Of course, the same phenomenon is most probably not unique to Hungarian doctoral schools; therefore, uniform global systems should be developed. The Academia Europaea has developed a position paper to propose pathways at different levels to facilitate the translation of scientific knowledge back to patient benefits (Hegyi et al. Reference Hegyi, Petersen, Holgate, Erőss, Garami, Szakács, Dobszai, Balaskó, Kemény, Peng, Monteiro, Varró, Lamont, Laurence, Gray, Pickles, FitzGerald, Griffiths, Jassem, Rusakov and Szentesi2020). The international education system of the League of European Research Universities (LERU) also focuses on high-quality teaching in research: (1) supporting mobility of doctoral candidates within Europe to broaden knowledge and strengthen research connections between European partners; (2) defining expectations and training plans by establishing a written agreement to assess progress and satisfaction of the student and supervisory team; (3) continuously review and, if necessary, upgrade training programmes to maintain continuous challenges (Abdi Reference Abdi, Pizzolato, Nemery and Dierickx2021; LERU 2016)
As a summary of quantity and quality analysis of the doctoral schools, it is clear that significant improvements and modern innovations are needed. These changes can be structured by setting up SMART goals.
SMART Goals
Specific goals that are achievable and realistic, with a defined timeframe for achieving the desired results, should be defined. Some of these specific goals can be easily measured by an increase in scientific output; the quantity of quality parameters applied in studies, the frequency of using modern methods, and the increasing number of IF and PhD dissertations, are all measurable parameters (Hegyi et al. Reference Hegyi, Petersen, Holgate, Erőss, Garami, Szakács, Dobszai, Balaskó, Kemény, Peng, Monteiro, Varró, Lamont, Laurence, Gray, Pickles, FitzGerald, Griffiths, Jassem, Rusakov and Szentesi2020).
Monitoring core members and supervisors would be a more efficient way of creating more quality institutions, rather than quantity-based institutions. Becoming an official member of a doctoral school as a core member or supervisor should not be age but activity dependent, and a supervisory role should be terminated if the scientist is continuously inactive. There is also a need to establish interdisciplinary units where international quality methods are taught, involving biostatisticians and IT specialists. In many medical centres where good quality clinical research is achieved, better patient care is provided. For this reason, in many places, it is no longer called clinical science but health science (Streed Reference Streed2015).
The Hungarian healthcare system could make huge progress by establishing these units, as Hungarian healthcare is far below international standards. Great innovations, developments, and scientific knowledge are needed to improve healthcare. It should be added that even within Europe, scientific work is not equally supported by governments, leading to huge differences in the financial allocation for R&D in Europe and worldwide (Petersen Reference Petersen2022).
In medical universities and doctoral schools, consideration should also be given to teaching new methods and quality parameters and an interdisciplinary group should be set up to promote the development of clinical methods. The above objectives could presumably be achieved within 5 to 10 years if these educational units become widespread and operational throughout Hungary. A very promising trend is that Semmelweis University has already set up a CTM where 91 PhD students started their training in modern clinical methodologies in 2021.
Strengths and Limitations
This is the most comprehensive summary of the achievements of the ten doctoral schools and the PhD education system in Hungary. However, our analysis has an important limitation. Table 1 shows the number of PhD graduates of core members and supervisors in each doctoral school, but these tables are not representative of the competitiveness of the doctoral school, as PhD graduates are associated with the supervisors, not with the doctoral schools themselves.
Conclusion
In terms of recent clinical research methodology, doctoral schools are seriously lagging behind in both teaching and scholarly activity. Innovation and new teaching platforms are essential to improve the proportion of science-oriented physicians. Hybrid education could be an appropriate solution to improve the overall scientific work and vision of healthcare workers and monitoring supervisors could be an effective way to achieve higher quality postgraduate training.
Conflicts of Interest
The authors declare no conflict of interest.
Funding
Funding: none to declare.
Author Contributions
Author contributions according to ICMJE recommendation: JH: conceptualization, project administration, writing the original draft; TK: statistical analysis, visualization; RN: data curation, methodological guidance, writing the original draft – review, and editing; PH conceptualization, methodological guidance, supervision, writing the original draft. All authors have (1) contributed to the concept of the study; (2) revised the manuscript; (3) read and approved the final version of the manuscript.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S1062798723000601
About the Authors
Judit Hegyi is Assistant Professor at Semmelweis University and University of Szeged, Hungary. Her research focuses on the health-economic aspects of health care.
Rita Nagy is Assistant Professor at Semmelweis University, Hungary. Her main research fields are cystic fibrosis and acute pancreatitis.
Tamás Kói is Assistant Professor at the Department of Stochastics at Budapest University of Technology and Economics and biostatistician at Semmelweis University, Hungary. His main research focuses are information theory and applied and theoretical statistics.
Péter Hegyi is Professor at Semmelweis University and University of Pécs, Hungary. His research mainly focuses on pancreatitis as clinical research and he leads PhD training at the Centre for Translational Centre at Semmelweis University in Budapest.
Open access
