Characteristics of included studies

The literature search resulted in a total of 265 articles. After deduplication, a set of 188 articles underwent title and abstract screening where 138 articles were excluded as deemed irrelevant. The remaining 50 articles were included in a full-text review, where a further 16 articles were excluded. A final total of 34 full-text articles that met all criteria were included for review (Figure 1).

Figure 1.

PRISMA 2020 flow diagram.

Most studies were from North America (USA, n = 15; Canada, n = 4), followed by Europe (The United Kingdom, n = 7; The Netherlands, n = 2; France, n = 1; Spain, n = 1; Switzerland, n = 1) and the remaining from Africa (Ghana, n = 1), Asia (China, n = 1), and Oceania (Australia, n = 1). Among these, there were 19 cohort studies, 5 cross-sectional studies, 5 quasi-experimental studies, 2 randomized controlled trials, 1 quality improvement study, 1 descriptive observational study, and 1 mixed-methods randomized controlled trial and cohort study. Twenty-nine unique data sources were identified in these studies; five studies were conducted with data from the Clinical Practice Research Datalink (CPRD) ^{Reference Gulliford, Sun, Anjuman, Yelland and Murray-Thomas19–Reference Sun and Gulliford23} and two obtained data from the same two private family medicine clinics ^{Reference Grigoryan, Zoorob, Shah, Wang, Arya and Trautner24,Reference Grigoryan, Zoorob and Germanos25}

EMR data for supporting AMS

Six categories of EMR data used for supporting AMS were identified from the studies included in the review. These were, (i) assessing antimicrobial prescribing quality, ^{Reference Sun and Gulliford23,Reference Grigoryan, Zoorob and Germanos25–Reference Vanstone, Patel and Berry40} (ii) measuring the effectiveness of an intervention, ^{Reference Grigoryan, Zoorob and Germanos25,Reference Foreman, Westerhof, Benzer, Eid, Egwuatu and Dumkow28,Reference Frost, Lou, Keith, Byars and Jenkins29,Reference McCormick, Cardwell, Wheelock, Wong and Vander Weide34,Reference Ray, Martin and Wolfson37,Reference Robinson, Barsoumian, Aden and Giancola38,Reference Vanstone, Patel and Berry40–Reference May, Sickler, Robbins, Tang, Chugh and Tran44} (iii) analyzing antimicrobial prescribing trends, ^{Reference Rockenschaub, Hayward and Shallcross22–Reference Grigoryan, Zoorob, Shah, Wang, Arya and Trautner24,Reference Brown, Wong, Kandiah, Moore and Quairoli26,Reference Chandra Deb, McGrath and Schlosser27,Reference Hawes, Turner, Buising and Mazza31,Reference Owusu, Thekkur and Ashubwe-Jalemba36,Reference Ray, Martin and Wolfson37,Reference Singer, Fanella and Kosowan39,Reference Adekanmbi, Jones, Farewell and Francis45–Reference Wong, Morkem, Salman, Barber and Leis52} , (iv) assessing patient and provider characteristics in prescribing ^{Reference Rockenschaub, Jhass and Freemantle21–Reference Grigoryan, Zoorob, Shah, Wang, Arya and Trautner24,Reference Brown, Wong, Kandiah, Moore and Quairoli26,Reference Chandra Deb, McGrath and Schlosser27,Reference Hawes, Turner, Buising and Mazza31,Reference Ivanovska, Hek, Mantel-Teeuwisse, Leufkens and Van Dijk32,Reference Owusu, Thekkur and Ashubwe-Jalemba36,Reference Robinson, Barsoumian, Aden and Giancola38,Reference Singer, Fanella and Kosowan39,Reference Kitano, Langford and Brown47,Reference Martinez-Gonzalez, Di Gangi, Pichierri, Neuner-Jehle, Senn and Plate49,Reference Wang, Li and Chen51,Reference Wong, Morkem, Salman, Barber and Leis52} (v) evaluating novel tools or measures ^{Reference Lautenbach, Hamilton and Grundmeier33,Reference Vernacchio, Herigon, Hatoun, Patane and Correa53} , and (vi) measuring specific conditions and outcomes. ^{Reference Gulliford, Sun, Anjuman, Yelland and Murray-Thomas19,Reference Sun and Gulliford23,Reference Moskow, Cook, Champion-Lippmann, Amofah and Garcia35,Reference Loadsman, Verheij and Van Der Velden48,Reference Soudais, Lacroix-Hugues, Meunier, Gillibert, Darmon and Schuers50,Reference Wong, Morkem, Salman, Barber and Leis52} The specific conditions and outcomes measured were: serious infection rates due to lower antibiotic prescribing, impetigo incidence, treatment and recurrence, prevalence and documentation quality of beta-lactam allergies, changes in antibiotic prescribing for different patient demographics and indications over time, male urinary tract infection prevalence, and pre- and post-pandemic respiratory tract infection (RTI) presentations. These are described and summarized in Table 1.

Table 1.

Use categories matched to study outcomes of interest and key findings

Among all the use categories, the most common were for analyzing prescribing trends and examining patient and provider characteristics related to antimicrobial prescribing. Large databases such as CPRD, ⁵⁴ NIVEL Primary Care Database, ⁵⁵ Secure Anonymised Information Linkage, ⁵⁶ Julius General Practitioners Network, ^{Reference Smeets, Kortekaas and Rutten57} and POLAR ⁵⁸ were used for this purpose. Individual practice and smaller-scale EMR data were used mainly for studies measuring the effectiveness of interventions, assessing antimicrobial prescribing quality, or evaluating specific conditions or outcomes.

Strengths and facilitators

Analysis revealed several descriptions of strengths of EMR data for AMS across each of the assessed publications. These were grouped into three overarching categories, which were (i) the availability of comprehensive data where patient, encounter, and practitioner-level data (e.g. comorbidities, signs and symptoms, encounter reason, age, sex, race, allergies, diagnosis, sociodemographic details) and prescription details were captured sufficiently to enable assessments that facilitated assessment of prescribing quality and trends, measurements of intervention effectiveness and outcomes, and evaluation of patient or provider characteristics and novel tools; ^{Reference Gulliford, Sun, Anjuman, Yelland and Murray-Thomas19,Reference Gulliford, Prevost and Charlton20,Reference Rockenschaub, Hayward and Shallcross22,Reference Grigoryan, Zoorob, Shah, Wang, Arya and Trautner24,Reference Grigoryan, Zoorob and Germanos25,Reference Lautenbach, Hamilton and Grundmeier33,Reference Moskow, Cook, Champion-Lippmann, Amofah and Garcia35,Reference Owusu, Thekkur and Ashubwe-Jalemba36,Reference Robinson, Barsoumian, Aden and Giancola38,Reference Gerber, Prasad and Fiks42,Reference Soudais, Lacroix-Hugues, Meunier, Gillibert, Darmon and Schuers50,Reference Vernacchio, Herigon, Hatoun, Patane and Correa53} (ii) coded and standardized data which allowed effective identification of patients or conditions of interest, ^{Reference Grigoryan, Zoorob, Shah, Wang, Arya and Trautner24,Reference Grigoryan, Zoorob and Germanos25,Reference Chandra Deb, McGrath and Schlosser27,Reference Giancola, Higginbotham, Sutter, Spencer, Aden and Barsoumian30,Reference Lautenbach, Hamilton and Grundmeier33–Reference Ray, Martin and Wolfson37,Reference Adekanmbi, Jones, Farewell and Francis45,Reference Gulliford, Sun and Charlton46} and (iii) large centralized databases with nationwide and longitudinal data which allows findings to be more broadly representative and likely more generalizable. ^{Reference Gulliford, Sun, Anjuman, Yelland and Murray-Thomas19,Reference Rockenschaub, Jhass and Freemantle21–Reference Sun and Gulliford23,Reference Ivanovska, Hek, Mantel-Teeuwisse, Leufkens and Van Dijk32,Reference Adekanmbi, Jones, Farewell and Francis45,Reference Loadsman, Verheij and Van Der Velden48,Reference Martinez-Gonzalez, Di Gangi, Pichierri, Neuner-Jehle, Senn and Plate49,Reference Wang, Li and Chen51}

Facilitators for effective EMR data use for AMS were: (i) availability of electronic prescriptions linked with EMR data enabling comparisons of dispensing rates and outcomes, and economic evaluations to be performed, ^{Reference Singer, Fanella and Kosowan39,Reference Adekanmbi, Jones, Farewell and Francis45} (ii) automatic coding tools to improve the quality of data extracted, (iii) mandatory documentation of fields driven by financial incentives which contributes to improved data completeness, ^{Reference Vernacchio, Herigon, Hatoun, Patane and Correa53} (iv) good EMR workflows for data entry which ensured cleaner and more complete data, ^{Reference Owusu, Thekkur and Ashubwe-Jalemba36} (v) interoperability and data linkage between EHR systems and other databases, ^{Reference Rockenschaub, Jhass and Freemantle21,Reference Frost, Lou, Keith, Byars and Jenkins29,Reference Gerber, Prasad and Fiks42,Reference Llor, Moragas and Cots43,Reference Gulliford, Sun and Charlton46,Reference Kitano, Langford and Brown47} and (vi) established processes data access and collection. ^{Reference Sun and Gulliford23–Reference Grigoryan, Zoorob and Germanos25,Reference Frost, Lou, Keith, Byars and Jenkins29,Reference Hawes, Turner, Buising and Mazza31–Reference Lautenbach, Hamilton and Grundmeier33,Reference Adekanmbi, Jones, Farewell and Francis45,Reference Wong, Morkem, Salman, Barber and Leis52}

Barriers and limitations

Descriptions of barriers were only included in a few studies. Barriers described were: (i) inconsistent EMR design across different systems leading to interoperability challenges, where standardization of data was required before use. Authors of one study concluded that “prescribing patterns can be influenced by system design” ^{Reference Foreman, Westerhof, Benzer, Eid, Egwuatu and Dumkow28} and in another “each EMR system has different architecture, even within one EMR system there may be province-specific differences in the EMR structure where information is stored” ^{Reference Wong, Morkem, Salman, Barber and Leis52} and (ii) cited “technical challenges in data extraction processes” leading to the exclusion of data from some regions. ^{Reference Wong, Morkem, Salman, Barber and Leis52}

Descriptions of potential data quality issues were identified in 23 out of 34 studies. Data quality descriptions could not be found in four studies; the absence of this does not suggest that there were no data quality problems. Thirteen studies described elements of data completeness as limitations, such as (i) encounters without corresponding diagnostic codes. ^{Reference Kitano, Langford and Brown47} (ii) unlinked microbiology results ^{Reference Owusu, Thekkur and Ashubwe-Jalemba36} and severity measures, ^{Reference Rockenschaub, Jhass and Freemantle21} or key clinical observations ^{Reference Robinson, Barsoumian, Aden and Giancola38} leading to less-comprehensive or unusable data for stewardship activities, (iii) absence of detailed records of the number of consultations causing problems in ascertaining whether an intervention was executed as intended, ^{Reference Blair, Young and Clement41} and (iv) missing data fields in historical records leading to potential inconsistencies in longitudinal data. ^{Reference Adekanmbi, Jones, Farewell and Francis45} Several studies highlighted the importance of mandatory documentation. One described poor allergy detail documentation where more than 36% (n = 13,679) of patients with a documented beta-lactam allergy failed to have any further description of their allergy, “making it difficult to know if the documented allergy is a true allergy and life-threatening (eg anaphylaxis); a known or anticipated, but undesirable, side effect (eg nausea); or a symptom of illness”. ^{Reference Moskow, Cook, Champion-Lippmann, Amofah and Garcia35} Indication was also described to be poorly documented in several studies; ^{Reference Rockenschaub, Hayward and Shallcross22,Reference Sun and Gulliford23,Reference Hawes, Turner, Buising and Mazza31} and were often captured in free-text in different places depending on the structure of the clinical software package used, ^{Reference Hawes, Turner, Buising and Mazza31} leading to potential “underestimation of associations” where “the direction of bias cannot always be anticipated”. ^{Reference Gulliford, Sun and Charlton46} One study noted this to be particularly problematic for “antibiotic prescriptions based on telephone calls”. ^{Reference Robinson, Barsoumian, Aden and Giancola38} .

Issues surrounding the plausibility of data (the believability or truthfulness of data values ^{Reference Kahn, Callahan and Barnard59} ) were described in fifteen studies. Several studies described unreliable, incomplete, or inaccurate documentation ^{Reference Rockenschaub, Hayward and Shallcross22,Reference Foreman, Westerhof, Benzer, Eid, Egwuatu and Dumkow28,Reference Robinson, Barsoumian, Aden and Giancola38,Reference Gulliford, Sun and Charlton46,Reference Loadsman, Verheij and Van Der Velden48,Reference Wang, Li and Chen51} as potential issues, some reflected this through descriptions of manual review requirements to ensure accuracy. ^{Reference Foreman, Westerhof, Benzer, Eid, Egwuatu and Dumkow28} Other issues were (i) EMR system design limitations potentially discouraging accurate reporting of antimicrobial prescriptions or diagnoses leading to incomplete or biased data, ^{Reference Giancola, Higginbotham, Sutter, Spencer, Aden and Barsoumian30,Reference Singer, Fanella and Kosowan39,Reference Gerber, Prasad and Fiks42} (ii) inability to capture data regarding symptoms and comorbidities causing indication coding errors, ^{Reference Giancola, Higginbotham, Sutter, Spencer, Aden and Barsoumian30} (iii) broken links between consultation and antibiotic data necessitating certain assumptions to be applied before use e.g., if an antibiotic prescription did not link directly to a consultation, the previous or subsequent consultation in that year was used to determine the patient’s age at time of consultation, ^{Reference Hawes, Turner, Buising and Mazza31} (iv) differences in coding between practices contributing to data inconsistencies, ^{Reference May, Sickler, Robbins, Tang, Chugh and Tran44} and (v) duplication in data entry through unstructured entries. ^{Reference Moskow, Cook, Champion-Lippmann, Amofah and Garcia35} More general descriptions of potential limitations relevant to data quality were also described in some studies such as: “limitations of the EHR” ^{Reference Foreman, Westerhof, Benzer, Eid, Egwuatu and Dumkow28} , “possible missing data from external sources”, ^{Reference Gulliford, Prevost and Charlton20} issues “inherent to the quality of the databases through standardization and data structuring”, ^{Reference Soudais, Lacroix-Hugues, Meunier, Gillibert, Darmon and Schuers50} and “limitations in the completeness of the GP records” ^{Reference Ivanovska, Hek, Mantel-Teeuwisse, Leufkens and Van Dijk32} .

A summary of strengths, facilitators, barriers, and limitations for each study is presented in Table 2.

Table 2.

Summary of identified strengths, facilitators, barriers, and limitations by study