Introduction
Telemedicine, defined as the use of information and communication technologies to deliver healthcare services at a distance, increased during the COVID-19 pandemic to offer access to healthcare without direct contact. In the years 2020–2022, video calls and home monitoring of patients with chronic disease increased internationally (Reference Omboni, Padwal and Alessa1). In Ontario, Canada, remote visits increased from 2 percent of ambulatory visits in the second quarter of 2019 to 71 percent in the second quarter of 2020 (Reference Bhatia, Chu and Pang2). In the United States, data from the Centers for Medicare and Medicaid Services showed an increase across nearly all medical specialties in weekly remote care – for example, video and audio-only visits, care chat, secure e-mail, and telemonitoring transfer of remote vital data – from 13,000 before the pandemic to 1.7 million in April 2020 (Reference Koonin, Hoots and Tsang3). In China, 95 percent of 148 surveyed physicians from 57 hospitals in 16 provinces adopted telemedicine systems during the pandemic (Reference Liu, Liu, Zheng and Bi4); In the United Kingdom, remote consulting increased in primary care from 30 to 89 percent in the months immediately before and after the start of lockdown and use of short message service from general practitioner to patients more than tripled from 23 messages per 1000 patients in July 2019 to 76 messages per 1000 patients in July 2020 (Reference Murphy, Scott and Salisbury5).
The pandemic has brought attention to a substantial healthcare force deficit caused by demographic challenges, clinicians’ increased risk of exposure to infections, increased patient demand for care, and clinician burnout and practice exit (Reference Shanafelt, Jonathan and Trockel6;Reference Giacomo, Saverio and Rosaria7). The UN High-level Commission on Health Employment and Economic Growth (8) and the WHO’s National Workforce Accounts have estimated a global health workforce shortage of 15.4 million health workers in 2020 (Reference Boniol, Kunjumen and Nair9) distributed unequally, affecting African and Eastern Mediterranean countries worse than the countries in the Western Pacific Region.
Experts have argued that telemedicine may favorably affect the workload of health professionals and reduce clinician time spent on delivering care. For example, a letter to the editor in the Journal of Clinical Virology argues that telemedicine can be used to maintain healthcare providers’ well-being by reducing unnecessary patient visits, promoting self-quarantine, and reducing emergency department overuse thus reducing the workload of physicians and minimizing exposure risk for healthcare workers (Reference Moazzami, Razavi-Khorasani and Dooghaie Moghadam10). Some evidence supports these speculations. For example, a systematic review including fourteen studies of the effectiveness of tele-triage found that using telehealth to triage patients could reduce the number of unnecessary emergency room visits by 1.2–22.2 percent and reduce clinician workload (Reference Farzandipour, Nabovati and Sharif11). Were telemedicine to reduce the time needed to treat, it could represent an important solution to the problems stemming from post-pandemic workforce constraints and increased healthcare demand.
We recently proposed that policymakers should estimate the time needed to treat (TNT) to clarify the effect of translating recommended care into practice on clinicians’ time and on the resulting opportunity costs for other patients and problems (Reference Johansson, Guyatt and Montori12).
Several frameworks and guidelines for health technology assessment (HTA) and economic evaluation recommend the assessment of the effect of telemedicine on system performance, such as productivity and efficiency, but do not specify the need to assess clinician’s time (Reference Unsworth, Dillon and Collinson13;Reference Gomes, Murray and Raftery14). One HTA-based model, the model for assessment of telemedicine (MAST) (Reference Kidholm, Clemensen, Caffery and Smith15), specifies that resources used when delivering the telemedicine application (e.g., clinician time) should be included in the estimated costs. MAST also includes assessment of the organizational aspects of telemedicine defined as “what kind of resources have to be mobilized and organized when implementing a new technology, and what kind of changes or consequences the use can further produce in the organization” (Reference Kidholm, Clemensen, Caffery and Smith15). However, a review of empirical health technology assessments using the MAST framework (Reference Kidholm, Ekeland and Jensen16) has found only one study assessing the effect of telemedicine on workload, and time spent by physicians and nurses (Reference Dario, Delise and Gubian17). This inattention to the effect of telemedicine on clinicians’ time may be due to a lack of evidence about this effect in clinical trials of telemedicine interventions. The objective of this article is to describe the degree to which clinical trials of telemedicine interventions have assessed their impact on clinicians’ time as an outcome. We then discuss how including TNT in HTA of telemedicine could affect the design of telemedicine services and studies. It is our hope that our study will emphasize the need for assessment of the effects of telemedicine on TNT in clinical studies and HTA, and thereby improve the possibilities to identify technologies that reduce the problems related to staff shortage in the healthcare systems.
Methods
Data sources
To identify how and to what extent studies of telemedicine interventions have included clinician time as a primary or secondary outcome, we carried out a review of studies of telemedicine on www.clinicaltrials.gov (18). The database is maintained by the United States National Library of Medicine and the National Institutes of Health and is the largest database of clinical studies in the world. The database includes both publicly and privately funded clinical studies, and the structured description of the studies includes detailed information on the intervention, study design, and primary and secondary outcome measures. Thus, by reviewing this database information about all outcome measures included in each of the studies could be collected.
Two authors (K.K. and L.K.J.) searched the database using the search term “telemedicine” and limited results to studies registered between January 1, 2012 and December 1, 2022 and tagged as completed or terminated studies with results.
Study selection
Eligible studies were randomized trials or observational studies with a comparison group of telemedicine interventions, that is the delivery of healthcare services through the use of information and communication technologies at a distance. Thus, the inclusion criteria are only regarding the study design and the type of clinical intervention. Two observers (K.K. and L. K.J.) working independently and in duplicate, evaluated the eligibility of each registration resolving disagreement by consensus after a joint review of the registration.
Data collection and analysis
Two observers (K.K. and L. K.J.) extracted from each trial registration record the trial primary outcomes and whether the secondary outcomes included direct or indirect measures of clinician time, and trial results for these outcomes. To be inclusive, we considered outcomes that assessed the direct effect of telemedicine on clinician time, that is, the time (minutes per patient and/or minutes for the whole population of patients) the clinician spent providing the telemedicine service and also outcomes assessing the indirect effect on the clinician time in the healthcare system in general, for instance considering the effect of telemedicine on the number of outpatient visits, hospital admissions or bed days. Related publications of the trial (indexed automatically by the registry) were reviewed whenever clinicians’ time was a trial outcome but the record did not include results.
Results
We identified 113 clinicaltrials.gov records in the search. Of these, thirteen records were excluded because the studies did not match our inclusion criteria, for example, they were testing how to implement telemedicine and not the effect of a telemedicine service. Based on the full-text report, an additional twenty-two records were excluded for various reasons (Figure 1), while seventy-eight records met the inclusion criteria and were included in the review (Table 1). Of the seventy-eight records, seventy-five were randomized controlled trials and three were observational studies with a control group.
PRISMA flow chart of included studies.
Records identified in clinicaltrials.gov
Outcome measures
Of the seventy-eight studies, twenty studies (26 percent) considered clinicians time (nine as primary outcomes, eleven as secondary outcomes, two as both primary and secondary), and four studies measured clinician time directly (one as a primary outcome and three as secondary outcome). The primary outcome measures included the number of admission, readmissions, and length of hospital stay, thus only indirect measures on clinicians’ time. The majority of the studies (66 studies) had clinical measures of morbidity as the primary outcome, and a few were only or in addition to the clinical outcomes focusing on patient satisfaction (8 studies) or adherence (8 studies). Eleven studies included secondary outcome measures related to clinician time, for example, healthcare utilization (2 studies), number of visits to emergency department (1 study), nursing home visits (2 studies), hospitalizations (3 studies), time spent on management of insulin or time spent on provider visits (3 studies).
Effects on clinicians’ time
Four studies assessed direct measures of clinicians’ time (Table 2), but no statistically significant difference between telemedicine versus usual care groups was found. Of the sixteen studies assessing the indirect effects of telemedicine on clinicians’ time, thirteen found no significant effects. Two studies (https://ClinicalTrials.gov/show/NCT02585232 and https://clinicaltrials.gov/study/NCT02528370) found statistically significant fewer hospitalizations favoring the telemedicine arm, and one trial found that patients in the telemedicine arm attended statistically significant more therapy sessions (https://clinicaltrials.gov/study/NCT00498706).
Records assessing direct measures of time needed to treat
Discussion
Health systems globally struggle with a shortage of clinician time, caused by a lack of healthcare workers and an increased demand for healthcare in the population. Telemedicine (and other digital solutions) may contribute to solving this problem by making care more time-efficient. However, we found that randomized controlled trials and observational studies of telemedicine from the last 10 years generally did not assess whether telemedicine saved clinicians’ time compared to usual care – or not. When assessed, clinician time was usually only considered indirectly. The few studies that measured clinician time directly showed no statistically significant difference, while only three out of sixteen studies measuring indirect assessments showed that telemedicine resulted in either a statistically significant reduction in the use of clinician time (two trials) or a statistically significant increased number of therapy sessions per patient (one trial).
It should be noted that not all telemedicine services are expected to have an impact on clinicians’ use of time. Therefore, not all randomized controlled trials or observational studies should include this as a primary or secondary outcome. Some digital interventions may simply replace a face-to-face meeting with a digital one without affecting the duration of the contact. Hypothetically, it may also be the case that telemedicine increases clinician time needed if loads of data need to be interpreted for healthy patients. As an example, a recent study of the Apple Watch has described the potential problem of false positive screening results leading to overutilization of healthcare resources (Reference Wyatt, Poole, Mullan, Kopecky and Heaton19).
In addition, telemedicine may also lead to increased access to care i.e., a larger proportion of the population eligible for the intervention or a lower no-show rate (Reference Gorodeski, Moennich and Riaz20). Thereby telemedicine may require more clinician time than usual care. It is however important to keep in mind that telemedicine can have several objectives: to improve quality of care, to increase access to care, to save time for patients, and/or to save time for clinicians. If telemedicine is found to improve the quality of care, an increased demand for clinician time compared to usual care may very well be warranted.
Strengths and weaknesses
Our review is strengthened by the systematic search for randomized trials or observational studies with a comparison group registered in the last decade, a choice made to potentially include the most reliable and pertinent evaluations. We may have missed registrations as we only searched using “telemedicine” as the search term. Combinations with other search terms such as “telehealth,” “digital health,” “home monitoring,” and so forth would have resulted in more studies and a more comprehensive review.
The results found in this review need to be validated by more traditional systematic reviews of published studies of the effect of telemedicine on clinicians’ time, for example, by reviewing studies in the MEDLINE, EMBASE, Cochrane database of systematic reviews and so forth Such a review could also find information of changes in the primary and secondary outcomes that is not included in the protocol in clinicaltrials.gov.
Implications
Current guidance for HTA and health economic evaluation of telemedicine recommends assessing the impact of telemedicine interventions on resources and workflow (Reference Unsworth, Dillon and Collinson13;Reference Dario, Delise and Gubian17). Assessing “clinician time” in studies and HTA analyses of telemedicine has several advantages. It could help avoid implementation of telemedicine interventions that would increase the demand on clinician time in an unreasonable way, and it could increase the implementation of effective interventions by enhancing acceptability among stressed clinicians.
Assessment of clinicians’ time could be done by direct or indirect measures of time. Examples of direct measures could be number of minutes spent by clinicians on implementation of the telemedicine service and number of minutes per patient spent by clinicians when providing the telemedicine service. Importantly, such measures also have to take into account if the telemedicine intervention expands the population eligible for the intervention compared to usual care. Examples of indirect measures could be number of admissions per patient, length of stay per admission, number of hospital outpatient visits per patient, emergency department visits, phone calls, video consultations per patient, number of visits to general practice per patient, and number of home healthcare visits per patient.
Both randomized controlled trials (RCTs) and observational studies will be valuable in future investigations into the impact of telemedicine on clinicians’ time utilization. While the former design offers the advantage of high internal validity, the latter may better capture effects on clinician time in real-world settings, enhancing external validity. However, observational before-after studies may be susceptible to confounding bias, such as that caused by other changes in staff or healthcare organization over time, and measurements may be less standardized compared to a randomized controlled trial. A potential solution to address these challenges is to design future studies on the effects of clinician time as cluster randomized trials. In this approach, for example, entire clinical departments and their staff are randomized to either telemedicine or usual care (Reference Liu and Wyatt21). This design aims to mitigate confounding factors and enhance the robustness of the findings.
Information about these direct and indirect measures could be collected as patient-level data in clinical studies and HTA by observation of healthcare professionals providing the telemedicine service, by means of digital registration systems (Reference Hsu, Lau and Huang22), by interviews or surveys with healthcare personnel (Reference Kleinpell, Barden, Rincon, McCarthy and Zapatochny Rufo23), or by use of data from electronic health records (EHR) (Reference Rasmussen, Clemensen and Zachariassen24).
If the recommendation by Johansson et al. (Reference Johansson, Guyatt and Montori12) is followed and future studies of telemedicine increase the inclusion of effects on clinician time as primary or secondary outcomes, this may also have implications for the design and content of the telemedicine interventions. Even though it is usual practice to let the primary outcome be defined by the objective and design of the study (Reference Smith, Morrow and Ross25), it is possible that the relation could also be the other way around – that defining clinicians’ time as a primary or secondary outcome may have an impact on the expectations of the effects of the intervention and thereby on the design and the content of the intervention itself. In the process of developing a telemedicine intervention you need to make many choices, because telemedicine often consists of several components. You need to find the right digital communication technology, the right patient group, the right clinicians to use the technology, the right level of integration with other IT systems and so forth. Telemedicine interventions have therefore been described as complex interventions (Reference Wade, Barnett, Martin-Khan and Russell26). Thus, a telemedicine intervention can be developed and designed in many ways depending on the objective. Placing priority on measuring clinicians’ time as an outcome of interest should affect how investigators design and conduct effective studies. Furthermore, consensus about the need to assess the effect of interventions on TNT could contribute to reduce the demands on clinician time encoded in the design of the interventions themselves. For instance, such a consensus could drive the development of digital health solutions that rely on automatically measured or patient-reported measures and on algorithms to determine which patients need clinician time.
Conclusion
In conclusion, the review of seventy-eight protocols from studies of telemedicine found that four studies included direct measures of clinician time, and sixteen studies included indirect measures, for example, readmissions and length of hospital stay. Assessment of effects on clinician time in clinical studies and in HTA of telemedicine may increase the potential of telemedicine to help solve one of the major challenges for health systems today – a shortage of clinician time caused by a lack of healthcare workers occurring in tandem with an increased demand of healthcare in the population.
Acknowledgements
We greatly appreciate the support of the Region of Southern Denmark, who has support the research behind this review.
Funding statement
The authors received no financial support for the research, authorship and/or publication of this article.
Competing interest
The authors declare none.
Key points
-
• This scoping review found that clinician time is not commonly measured in randomized trials or observational studies of telemedicine interventions.
-
• Attention to clinician time in evaluations of telemedicine may bring attention to the organization of workflows and increase the value of telemedicine.
Open access
