Introduction
Antimicrobial resistance is among the greatest public health threats globally and nationally within the United States, posing significant threats to human, animal, and environmental health. 1,2 Specifically, the Center for Disease Control and Prevention estimated that there were more than three million drug-resistant infections in the United States in 2019. 1 Antimicrobial-resistant bacterial infections were associated with an estimated 4.95 million deaths worldwide in 2019. 3
Antimicrobials, particularly antibiotics, are commonly prescribed in outpatient healthcare settings in the United States. There were 250 million oral antibiotic prescriptions in outpatient settings in the United States in 2018, equivalent to 763 prescriptions per 1000 US persons. 4 Estimates suggest that 30%–56% of outpatient oral antibiotic prescriptions may be inappropriate. Reference Fleming-Dutra, Hersh and Shapiro5–Reference Young, Strey and Lee8 Antimicrobial stewardship programs are important and necessary in implementing interventions that target the prescribers. Providing education about appropriate use of antibiotics to patients themselves complements good antibiotic stewardship practice. Although studies suggest there is a discrepancy between provider perceptions and patient expectations of antibiotic prescriptions, medical providers continue to report that they perceive pressure to prescribe antibiotics from patients or parents of patients. Reference Szymczak, Feemster, Zaoutis and Gerber9–Reference Mangione-Smith, McGlynn, Elliott, Krogstad and Brook12 Providers may fear that not prescribing antibiotics could drive patients away or garner negative reviews of their practice. Reference Kohut, Keller and Linder11 This is especially worrisome for providers in urgent care settings, where patients may not have the same level of loyalty and trust in their urgent care provider compared to a longstanding provider with whom a patient has an established relationship. Reference Kohut, Keller and Linder11 In this context, educating patients on the risks and dangers of taking unnecessary antibiotics may be particularly important. In addition to direct impacts on improving patient knowledge around antibiotics, providers may also be more likely to adhere to guidelines if they perceive that patient education materials are being effectively implemented in their clinical setting, thereby reducing unnecessary antibiotic prescribing.
Educating urgent care patients on appropriate use and risks of antibiotics remains challenging. Displaying educational materials, such as posters or handouts, is one technique that has been used to educate patients on medical issues. However, even within the larger scope of antibiotic-related patient education, this technique has varied in its effectiveness, with more memorable formats (eg GIFs, memes) being linked with increased effectiveness. Reference McNulty, Nichols, Boyle, Woodhead and Davey13–Reference Ritchie, Rakhmanova, Out-O’Reilly, Reay, Thomas and Sajtos17 Further, there have been a limited number of studies on patient education stewardship interventions specific to urgent care settings, with most being restricted to single urgent care networks. Reference Laude, Kramer and Lewis18–Reference Grabinski, Leybov and Battistich20
This study examined the effect of multiple educational materials on urgent care patients’ knowledge of antibiotic resistance and appropriate antibiotic use, as well as their expectation level for when an antibiotic prescription should be given. This study also compared outcomes of participants who recalled seeing any educational materials to those who did not recall seeing any educational material. Our study aims to elucidate the impact of educational materials on knowledge about appropriate use of antibiotics and antibiotic resistance, and on patients’ expectations for antibiotic prescriptions in a geographically diverse set of urgent care centers.
Methods
Setting and participants
Urgent care centers (UCCs) (n = 29) from four UCC networks located in Colorado, Florida, Georgia, and New Jersey were recruited to participate in a cluster randomized trial. All of the UCCs served both adult and pediatric patients. UCCs were recruited through emails via an Urgent Care Association listserv from July through October 2018, and study questionnaires were administered from October 2018 through February 2019. Eligible participants were either adult patients or guardians of pediatric patients (ie, any adult that may have accompanied a child to a UCC visit). The study was approved by the George Washington University (GWU) Institutional Review Board (#051866).
Intervention
The educational materials used in this study were developed by a professional marketing and public relations firm (CommunicateHealth, Inc, Rockville, MD) based on quantitative and qualitative research conducted by the research firm Strategies 360, Inc (Seattle, WA) and refined in focus groups of urgent care patients. Specifically, Strategies 360 conducted a four-day online bulletin board focus group (QualBoard) among 36 recent U.S. UCC patients in March 2017, including parents of children (n = 15) and individuals with low health literacy (n = 13). Further quantitative evaluations of educational materials were performed by Strategies 360 and GfK (Nuremberg, Germany) among 610 recent U.S. UCC patients using GfK’s KnowledgePanelTM in April 2017.
The three educational materials included in this study included a priming poster, an informational handout, and a commitment poster (Supplemental Figure S1). The participating clinic was instructed to display the priming poster and the handout in a prominent location on the waiting room and place this poster in a prominent location in the exam room. The effect of the priming poster is based on automatic attitude activation. Reference Fazio21 The educational handout was designed to educate patients that antibiotics are not effective for viral infections and to preemptively answer patients’ questions. The commitment poster, Reference Meeker, Knight and Friedberg22 signed by the practice’s providers, leverages behavioral psychology to enhance accountability for appropriate antibiotic prescribing by displaying a pledge from the providers to responsibly prescribe antibiotics.
Randomization
The UCCs were randomly assigned to study arms using stratified block randomization. Clinics were grouped by UCC network during the block assignments to ensure different arms were represented in each network. There was one control arm (ie, no materials displayed) and five intervention arms assigned using a 3 (commitment poster, priming poster, no poster) by 2 (handout and no handout) study design, resulting in three intervention arms with one material and two arms with two materials. This design was intended to assess the additive or multiplicative effect of two interventions compared with one intervention.
Data collection
Patients completed a voluntary and anonymous study questionnaire at the end of their UCC visit. Due to resource limitations, the number of eligible patients who declined participation in the study was not tracked. The questionnaire included cued recall questions with three false images and three educational images from the study materials, participant knowledge regarding antibiotics, and their expectations of receiving antibiotics. Participant antibiotic knowledge included four questions that assessed information from the study handout (Supplemental Materials).
Patient antibiotic expectations were assessed on the question “What do you expect an antibiotic for?” regarding nine conditions: assessed cough, cold, virus, strep throat, flu, urinary tract infections, bronchitis, skin infection, and sinus infection. The expectation score represented a behavioral indicator and was not intended as a measure of knowledge. Expectation for any condition – both appropriate and inappropriate – received one point, resulting in a score ranging from 0 to 9, with 0 representing least likely to expect antibiotics and 9 being most likely to expect antibiotics. In supplemental analyses, we assessed patient antibiotic expectations for antibiotic-inappropriate conditions (ie, cough, cold, virus, flu, and bronchitis), with a score from 0 to 5.
Data analysis
Responses from participant surveys were manually entered by the GWU team. Both intention-to-treat (ITT) and as-treated analyses were performed. The ITT analysis evaluated whether educational material assignment at UCCs impacted participants’ knowledge or expectation scores. The as-treated analysis focused on whether correctly recalling the material significantly improved participants’ antibiotic expectation or knowledge scores compared to participants who either (i) did not correctly recall seeing the material assigned to their arm or (ii) were from a center assigned to the control arm. For the as-treated analysis, patients assigned to an arm with two assigned materials that recalled seeing both materials were excluded due to sample size limitations; however, if they only saw one material, they were reassigned into the relevant arm for that material.
Descriptive statistics for categorical variables were analyzed using χ2 tests, Wilcoxon Rank Sum tests, and Kruskal–Wallis tests as appropriate. Continuous dependent variables (scores) were analyzed using analysis of variance (ANOVA) tests. Linear mixed effect models, clustered by individual clinics, were employed and controlled for the participant’s gender, age group, ethnicity, race, and education level. Spearman correlations were used to quantify associations between knowledge and expectation scores. To detect a 7.5% difference on the knowledge and expectation scores, we estimated needing a sample size of 155 participants per arm, with five clusters per intervention, an intraclass correlation coefficient of 0.2, a two-sided 5% significance level, and 80% power. Statistical analysis was conducted using SAS 9.4 (Cary, NC) and R (R version 4.3.2). The CONSORT checklist was utilized when writing this report. Reference Schulz, Altman and Moher23
Results
Study population
Completed questionnaires were received from 27 of 29 recruited UCCs, resulting in 2919 questionnaires and four to five centers per arm (Figure 1). The number of respondents differed between study arms. There were 604 in the priming poster arm, 432 in the priming poster with handout arm, 249 in the commitment poster arm, 118 respondents in the commitment poster with handout arm, 761 in the handout arm, and 755 in the control arm (Figure 1). Study participants’ socio-demographic characteristics also varied significantly by study arm (Table 1). Across all arms, most participants were White (range: 69.5%–86.3%) and female (range: 62.4%–70.6%). Control arm participants tended to be older, with 52.6% being over 50 years of age, while 33.1% in the commitment poster with handout arm were over 50 years of age.
Flowchart of randomization and enrollment of urgent care centers.
Distribution of study arms by demographic and potential confounding variables
Patient recall of educational materials
Of the participants assigned to an intervention arm, 1,257 of 2,164 (58.1%) participants reported not seeing any materials during their visit (Table 1). Only 588 (27.2%) participants correctly recalled seeing materials that were assigned to their clinic. Of the 588 participants who correctly recalled seeing materials assigned to their clinic, 125 (5.8%) also reported seeing materials that were not assigned to their clinic. Therefore, only 463 (21.4%) participants correctly identified the materials assigned to their clinic without also reporting incorrect materials. Some participants in the control arm (9.9%) reported seeing a material in their center despite there not being one. Poor observance of materials occurred at nearly all centers. Even among the top quartile of UCCs with the highest recall rates of the materials, less than half of participants (40.9%) were able to correctly recall seeing the educational materials. In the UCCs with the lowest recall rates (bottom quartile), only 8.5% of participants correctly recalled seeing the materials. Knowledge scores were higher among participants who correctly recalled seeing materials, compared with participants who did not see correct materials (P = .029); there were no differences in expectation scores based on recall of materials (P = .17) (Table 2).
Knowledge and expectation score means a by demographic and potential confounding variables
a Fully adjusted model accounts for clinic, gender, age group, ethnicity, race, and education level.
Impact of educational materials on antibiotic knowledge and expectations
In the ITT analysis, although there were differences between study arms in the unadjusted model, there were no significant associations between study arms when accounting for clustering by clinic. In the unadjusted model, knowledge scores were significantly different between groups (P = .004), with the lowest scores in the control arm (mean = 2.11, SD = 1.13), and higher knowledge scores in the priming poster (mean = 2.32, SD = 1.08, P < .001) and handout-only arms (mean = 2.29, SD = 1.05, P = .001) (Table S3). Expectation scores were not significantly different between arms (P = .71) (Table S3).
When controlling for clustering by clinic using a mixed-effects model, there were no significant associations in knowledge scores or expectation scores by group (Figure 2).
Intention-to-treat comparison of knowledge and expectation scores among each study arm (material group) vs control (assigned no materials), clustered by clinic and adjusted for gender, age group, ethnicity, race, and education level. 95% confidence intervals for each score indicated by dashed lines.
As-treated analysis of educational materials and antibiotic knowledge and expectations
After excluding all participants who saw more than one intervention from the two interaction arms, 2,866 participants remained eligible for the as-treated analysis. In both the unadjusted and adjusted regression models, both the commitment poster and handout were associated with higher knowledge scores. In the unadjusted analysis, compared with participants who did not see the materials (mean score = 2.16), higher knowledge scores were noted for participants who saw the commitment poster (10.6% higher, mean score = 2.39, P = .039), handout (7.9% higher, mean score = 2.33, P = .033), and priming poster (7.4% higher, mean score = 2.32, P = .024) (Table S4).
When controlling for clustering by clinic and adjusted for covariates, both the handout and commitment poster remained associated with higher knowledge scores (8.8% higher, P = .02 and 13.2% higher, P = 0.01, respectively), but not the priming poster. The educational interventions were not associated with differences in patient expectation scores in either the unadjusted or adjusted model (Figure 3). In supplemental analyses, when limiting to diagnoses typically inappropriate for antibiotic treatment, inappropriate expectation scores tended to be lower among the handout and commitment poster arms, but differences were not statistically significant (7% lower, P = .16 and 11% lower, P = .13, respectively) (Table S4). Knowledge scores were significantly inversely associated with expectation scores (ρ = –.13, P < .001) and inappropriate expectation scores (ρ = –.42, P < 0.001).
As-treated analysis of knowledge and expectation scores for participants who recalled seeing each intervention compared to control (did not recall seeing materials), clustered by clinic and adjusted for gender, age group, ethnicity, race, and education level. 95% confidence intervals for each score indicated by dashed lines.
Discussion
This multi-state, multi-site block randomized study of over 2,000 patients at U.S. UCCs evaluated the effectiveness of multiple educational materials on improving patients’ antibiotic knowledge and expectations. While educational materials can improve patient knowledge regarding antibiotic prescribing, most urgent care patients do not notice passively displayed educational materials, highlighting the challenge of implementing and studying health promotional interventions in the urgent care setting.
Participants at nearly all UCCs showed poor recall of the material(s) that was/were displayed in their care setting, suggesting that posters and handouts may be ineffective interventions across a variety of urgent care settings. Poor recall may explain the limited impact of education material(s) on participant antibiotic knowledge or antibiotic expectation scores in the ITT analysis. As a result, we also could not evaluate additive or multiplicative interaction of the materials in the two-material arms. After controlling for covariates, the as-treated analysis showed that participants who correctly recalled seeing the material had higher antibiotic knowledge scores compared to those who did not recall seeing the material, while there were no differences in patient antibiotic expectation scores.
Prior studies on patient education for antimicrobial stewardship have had mixed results, potentially also due to patients not noticing the materials. Reference McNulty, Nichols, Boyle, Woodhead and Davey13–Reference Ritchie, Rakhmanova, Out-O’Reilly, Reay, Thomas and Sajtos17 A single center study in a family practice in the northeastern United States evaluating the effect of posters, GIFs, and memes displayed in waiting rooms reported a 12.6% decrease in antibiotic prescribing, with anecdotal notes that GIFs and memes had drawn patient attention and may have bolstered effectiveness of patient education interventions. Reference McNicholas and Hooper14 Factors such as competing health messaging materials in the waiting room, participants who are coping with acute illness, and commonness of phone usage in the waiting room may help to explain why participants were significantly more likely to notice educational materials in the exam room than in the waiting room. While stewardship interventions do not typically assess recall of patient-directed materials, our study quantifies how the effectiveness of patient educational materials may largely be determined by whether or not patients notice the materials. Our findings highlight the low rates of recall across a large, diverse set of urgent care centers, suggesting that simple efforts to increase awareness of materials could significantly improve effectiveness of patient-directed stewardship interventions.
In contrast to improved antibiotic knowledge, there was no effect on the patient antibiotic expectation scores, which may be a result of residual confounding. Control arm participants reported higher educational levels, which is generally associated with lower antibiotic expectations. Reference Ritchie, Rakhmanova, Out-O’Reilly, Reay, Thomas and Sajtos17 The antibiotic expectation questions included both antibiotic-appropriate and inappropriate conditions and better knowledge of antibiotic appropriateness may have resulted in both lower responses for inappropriate conditions and higher responses for appropriate conditions, leading to a similar total score. For example, the handout arm had lower antibiotic expectations for colds (P = .08) but higher expectations for strep throat (P < .001), as compared to the control arm. Supplemental analyses restricting to antibiotic-inappropriate diagnoses showed decreased expectations in the intervention arms, but the differences were not statistically significant. Consistent with prior studies, antibiotic knowledge and expectation were inversely correlated. Reference Thorpe, Sirota, Orbell and Juanchich24
Our study had limitations. Participants with better recall of information may be inherently different from participants who see but are unable to recall interventions. Similarly, the study may have response bias where participants who responded may be more likely to participate depending on their antibiotic related knowledge or expectations. There may be misclassification in the as-treated analysis including participants who did not see the materials but reported having seen them, and vice versa. Additional misclassification or data validity issues may result from data entry errors or through the anonymous self-reporting approach; such non-differential misclassification is likely to bias findings towards the null. Sensitivity analyses that only included participants who correctly recalled seeing the materials without reporting seeing any other materials, compared with participants who did not see any materials, were consistent with the main as-treated analysis. Based on simulation studies with cluster-randomized trials, Reference Leyrat, Morgan, Leurent and Kahan25 our ITT analysis may be underpowered to detect statistically significant differences for small effect sizes. Enrolling UCC sites from multiple states provided many benefits, including increased sample size and improved generalizability across US regions; however, it limited our ability to assess compliance with display of materials at each site, and we were not able to collect the number of patients seen at each clinic to evaluate potential differential participation. The UCCs, including those in the control arm, may have had existing antibiotic stewardship interventions in place. Although we did not capture this information directly, no other materials were noted in setup pictures provided by UCCs or noted by UCCs directly. While the results of the as-treated analysis indicate statistically significant differences in scores for some of the interventions, this does not necessarily imply that there is clinical significance. Lastly, while commitment posters as reported by Meeker et al. are shown to improve antibiotic prescribing, Reference Meeker, Knight and Friedberg22 our study focused on patient-level knowledge and expectations and did not assess impact on providers or rates of inappropriate prescribing.
Overall, the current study assessed how educational materials displayed in UCCs impacted patient antibiotic-related knowledge and expectations, which showed that few participants could recall the displayed materials and that there was no significant passive benefit. Patient educational interventions that require passive observation in UCCs show limited utility if they are not seen. However, participants who recalled the material displayed in their clinic had improved antibiotic-related knowledge compared to participants who did not see the materials, indicating the importance of materials that can capture patients’ attention. Future research should assess factors that promote uptake of educational materials in outpatient settings, in addition to evaluating overall effectiveness of the interventions.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ash.2024.475
Data availability statement
The data are available from the corresponding author on reasonable request.
Acknowledgements
We gratefully acknowledge the patients in this study. We thank the providers and staff at the participating urgent care centers. We also thank the Urgent Care Association and the College of Urgent Care Medicine for their valuable support throughout the project.
Author contribution
RFH, MMT, CML conceptualized and designed the study. CML contributed to funding acquisition. MJC and DEP conducted formal analysis. LR assisted with project administration. MJC and DEP drafted the original manuscript. All authors contributed to review and editing of the final manuscript.
Financial support
This project was supported by a grant from Merck Sharp & Dohme Corp., a subsidiary of Merck & Co. The funder had no role in the design and conduct of the study.
Competing interests
All authors report no conflicts of interest relevant to this article.
Open access
