Introduction
Early into the COVID-19 pandemic, concerns arose about the risks of exposure in healthcare facilities. Once the United States Food and Drug Administration gave Emergency Use Authorization to the two mRNA vaccines in December 2020, 1 healthcare personnel were prioritized for vaccination due to their potentially higher risk of exposure and transmission. Reference Dooling, McClung, Chamberland, Marin, Wallace, Bell, Lee, Talbot, Romero and Oliver2 Early vaccination of U.S. healthcare personnel provided opportunities for large sample, real-world evaluations of vaccine effectiveness Reference Pilishvili, Gierke, Fleming-Dutra, Farrar, Mohr, Talan, Krishnadasan, Harland, Smithline, Hou and Lee3 and investigations of preventive behaviors and risk factors for healthcare-associated SARS-CoV-2 infections. Such information can aid in the prevention of future healthcare-associated infections.
As a participating site in a COVID-19 mRNA vaccine effectiveness study, in 2020–2021 we recruited University of Utah (UU) healthcare personnel to participate in a test-negative case-control study and surveyed participants about potential COVID-19 exposures and protective behaviors at work and in the community. The validity of such observational studies to evaluate COVID-19 vaccine effectiveness depends in part upon successful recruitment efforts to ensure adequate study power and a representative sample of participants that reflect the healthcare personnel population.
This paper addresses three objectives. First, we assess the representativeness of recruited participants relative to the source population of interest. Second, we evaluate the effect of each recruitment contact attempt in our protocol on study enrollment. Finally, we evaluate the prevalence of self-reported protective behaviors and potential exposures among healthcare personnel and determine which factors are associated with testing positive for SARS-CoV-2.
Methods
Multisite vaccine effectiveness study
The Centers for Disease Control and Prevention (CDC) led a 33-site study to evaluate the early effectiveness of a complete series of SARS-CoV-2 vaccines in preventing laboratory-confirmed, symptomatic COVID-19. Reference Pilishvili, Gierke, Fleming-Dutra, Farrar, Mohr, Talan, Krishnadasan, Harland, Smithline, Hou and Lee3,Reference Pilishvili, Fleming-Dutra and Farrar4 The study used a test-negative case-control design. Cases were healthcare personnel with at least one symptom and a positive SARS-CoV-2 test. Week-matched controls were selected among personnel with a negative test result for SARS-CoV-2. Additional details of the study methods have previously been reported. Reference Pilishvili, Gierke, Fleming-Dutra, Farrar, Mohr, Talan, Krishnadasan, Harland, Smithline, Hou and Lee3,Reference Pilishvili, Fleming-Dutra and Farrar4
Utah study
The UU IRB approved the study at the UU Health. Healthcare personnel included all employees in clinical and patient-facing positions as well as support staff and other personnel within the healthcare setting. Employees were eligible if they sought polymerase chain reaction testing for SARS-CoV-2 at the UU Health from December 29, 2020 through July 31, 2021 and were symptomatic at time of testing. Exclusion criteria included a prior positive SARS-CoV-2 test, asymptomatic at time of testing, or participating in another UU COVID-19-related study.
Each week of the study, all SARS-CoV-2-positive symptomatic employees (cases) and three matched randomly selected symptomatic employees testing negative for SARS-CoV-2 (controls) were invited to participate. Vaccination status was ascertained using occupational health records, vaccine cards, or medical records. Vaccination status was classified in accordance with the multisite study as follows: fully vaccinated was defined as 7 or more days post second dose, partially vaccinated was defined as one dose or 6 or fewer days post second dose, and not vaccinated was defined as no vaccine doses.
Recruitment and study procedures
In accordance with best practices, Reference Dillman, Smyth and Christian5 we developed a standardized recruitment protocol that included multiple reminders for nonrespondents and multiple modes of contact. If at any point during recruitment an employee completed the survey, they did not receive any additional messages. The following standard protocol was used for all potential participants with the exception noted below. Contact 1 was an email with a link to the online survey. Contact 2, three days later, was a text message reminder referring recipients to the first email message for information and the survey link. Contact 3, on Day 10, was a reminder email sent to nonrespondents. Nonrespondents were contacted by phone on Day 14 (Contact 4). We left a voicemail about the project for those who were not reached directly. If during the phone call the employee reported no longer having the email message on hand, we sent another email with a link to the survey that same day (Contact 5, second email reminder). All others who did not receive a reminder email on the same day as the phone call received Contact 5 at a later date if they still had not responded. A final reminder email was then sent to remaining nonrespondents (Contact 6). If at any point in this process, if an employee completed the consent form and began completing the survey, they were not subject to remaining reminder messages directed at nonrespondents. Alternatively, if they started the survey but did not complete it within two days of initiation, they received an automated reminder message asking them to complete their survey, but no further messages.
The recruitment email messages contained a link to a REDCap web survey. After confirming eligibility, individuals were asked to provide informed consent and then complete a questionnaire developed by the CDC. The questionnaire asked about COVID-19 testing, medical care related to current illness, comorbidities, demographics, vaccination, infection control practices, and potential COVID-19 exposures at work or outside of work. Potential exposures at work included close contact with an individual (either patient or non-patient) with confirmed or suspected COVID-19, regardless of personal protective equipment use. The survey also inquired about non-work potential exposures including close contact with sick individuals and participation in social activities.
Data analysis
We calculated descriptive statistics for demographic characteristics of study participants. Individuals who participated in the study more than once were only included in analyses once, using data from their first enrollment. We compared demographics of participants to demographics of the study’s source population using chi-square tests. The source population data, provided by the University Electronic Data Warehouse, consisted of aggregate summary statistics representing the entire pool of employees who were tested during the study period, regardless of symptom presentation. These aggregate data were used because the study was not approved to retain demographic information on sampled nonparticipants.
Demographic variables included sex, ethnicity, race, age, and staff role. Staff role was categorized as providers (physicians, physician assistants, nurse practitioners), nursing (including assistants), allied healthcare workers (e.g., pharmacists, dieticians, social workers, physical therapists), administrative staff, support staff (e.g., customer service, facilities managers), other faculty, and other (including research and IT).
To evaluate the effect of the study protocol on response outcomes, we calculated the cumulative survey response proportion at each stage of the contact protocol, prior to each subsequent contact attempt, for cases, controls, and combined.
We assessed differences between cases and controls in protective behaviors and potential exposures using chi-square tests. We used conditional logistic regression to assess predictors of testing positive for SARS-CoV-2. A univariable model included vaccination status, and a multivariable model was conducted including variables exhibiting significant differences between cases and controls: vaccination, any potential exposure at work (composite variable representing contact via patient, coworker, or others), close contact to ill individuals outside work, and always wearing a mask outside of work (compared to less frequent wearing of masks). This multivariable model also adjusted for age, sex, race, Hispanic ethnicity, staff role (administrative vs. other), and presence of any underlying health condition known to increase severity of COVID-19. Reference Pilishvili, Gierke, Fleming-Dutra, Farrar, Mohr, Talan, Krishnadasan, Harland, Smithline, Hou and Lee3 Conditional logistic regression was utilized to account for the weekly frequency-matched study design (3:1 controls to cases selected weekly).
Results
Demographic representativeness of study participants
Excluding 52 duplicate responses, 1456 employees were contacted, 608 initially consented, and 598 started the survey. A total of 503 (34.5%) employees enrolled and completed the survey (see Supplemental Figure: study flow). Participants were 76.7% female, 11.9% Hispanic or Latino, and 90.5% White (Table 1). Participants were younger with only 3.0% 60 years of age or older. Nurses (including registered nurses and nursing/medical assistants) comprised 31.6% of participants. We observed statistically significant differences between study participants and the source population in terms of sex, ethnicity, race, age, and staff role (Table 1). Because the university data warehouse race variable includes more race categories than were used in the study, race was indicated as “other” for 10.2% of the source population, whereas none of the study participants had race listed as “other” (P < 0.001). Participants were younger than the source population, and nursing staff comprised 31.6% of participants compared to 23.3% of the source population. There were no statistically significant differences in the demographics of enrolled cases compared to enrolled controls (see Supplemental Table).
Table 1. Demographics of study participants compared to source population for a study of COVID-19 vaccine effectiveness among healthcare personnel
a Source population data represents characteristics of eligible employees who were tested for Covid-19 during the study period.
Effect of contact attempts on recruitment outcomes
We found that 23.9% of potential participants who received the standard contact protocol completed the survey after the initial invitation email, with no need for further contact attempts (Table 2). These early responders comprised over half of all total participants. A text message sent on day 3 produced 25 additional responses for a cumulative response rate of 26.0%. Each subsequent contact resulted in additional responses, with the final contact only resulting in three additional participants. Initially, participation was slightly higher among cases than controls, but the final response proportions were similar (33.6% of cases and 33.3% of controls). Of note, only 29% of our phone call attempts resulted in a live telephone conversation with the employee.
Table 2. Cumulative survey response by contact attempt, for cases, controls, and combined a
a Table includes only individuals who were subject to the contact protocol displayed in the table (the standard protocol). This table excludes individuals who began answering the survey but did not return to complete it for > 2 days, as they were not subject to the standard protocol after they had initiated the survey. Instead, if someone began the survey but did not complete it, after two days they were sent a reminder email to finish the survey. If they did not return to complete the survey, they received no further messages. Therefore, these individuals are excluded from counts in this table as this contact protocol was no longer applied to them.
b Once someone completed the survey, they did not receive any more recruitment messages.
c All nonrespondents received a phone call on Day 14. In this table we distinguish between those who received the contact 5 email after Day 14 (5a in table) and the subset of nonrespondents who received the phone call and the contact 5 email (contacts 4b and 5b) on the same day. These scenarios are documented separately in the table to reflect the fact that some individuals received two contacts in a single day, which could have influenced their likelihood of response differently than having received them on separate days.
d Final response count does not equate to 503 because table excludes individuals who did not receive the standard protocol.
Association of potential exposures and protective behaviors with testing positive for SARS-CoV-2
Just over 28% of participants were fully vaccinated. Full vaccination was less common among cases (20.9%) than controls (30.8%, P = 0.01; Table 3). Among controls, 13.6% reported a potential COVID-19 exposure from a coworker, compared to only 4.7% of cases (P = 0.01). There were no significant differences between cases and controls in having close contact with a COVID-19 patient, mask-wearing frequency at work, or days per week employees worked in person on site. The percentage of cases who had close contact with someone ill outside of work (45.7%) was significantly higher than that among controls (20.1%; P < 0.001). Cases were slightly less likely to report always wearing a mask when outside of work (83.7% compared to 90.4%; P = 0.01). There were few significant differences between cases and controls in out-of-work behaviors with potential for exposure.
Table 3. Potential COVID-19 exposures at work and outside of work, for all participants and by case-control status
Full vaccination was associated with significantly lower odds of testing positive for SARS-CoV-2 (OR: 0.20; 95% CI: 0.08, 0.46; Table 4). This equates to vaccine effectiveness of 80% (1.0 – 0.20 = 0.80). The odds of testing positive for SARS-CoV-2 among partially vaccinated individuals was 0.62 (95% CI: 0.37, 1.03; vaccine effectiveness: 38%). In the multivariable model, those with contact with someone ill outside of work had 3.74 higher odds of testing positive for SARS-CoV-2 than those who did not (OR = 3.74; 95% CI: 2.29, 6.11). Conversely, potential exposure at work was inversely associated with testing positive for SARS-CoV-2 (OR: 0.51, 95% CI: 0.29, 0.88) as was being fully vaccinated for COVID-19 (OR: 0.21; 95% CI: 0.08, 0.52). Frequency of mask-wearing outside of work was not significantly associated with a positive test, nor were any participant characteristics or underlying health conditions.
Table 4. Conditional logistic regression predicting positive test for SARS-CoV-2 among healthcare personnel a
Note. OR, Odds ratio; CI, Confidence interval; Ref, referent category.
a Conditional logistic regression was used to account for study design using weekly frequency-matched cases and controls.
b Model 2 adjusts for employee age, sex, race, Hispanic ethnicity, staff role, and presence of any underlying health condition.
Discussion
Healthcare facilities still grapple with COVID-19 infections among employees. This study evaluated factors associated with SARS-CoV-2 infection among healthcare personnel. Results can inform efforts to prevent future outbreaks among healthcare personnel. By assessing the effectiveness of our recruitment protocol and the representativeness of participants, this study adds insight into successes and challenges healthcare organizations face recruiting healthcare personnel in challenging circumstances such as a global pandemic.
When initiating this study, we recognized that healthcare personnel had been substantially impacted by the pandemic, including burnout and mental health effects, Reference Danet Danet6,Reference Batra, Singh, Sharma, Batra and Schvaneveldt7 experiencing COVID-19-related bullying and harassment, Reference Dye, Alcantara, Siddiqi, Barbosu, Sharma, Panko and Pressman8 and changes to work hours, duties, and job security. Reference Keihanian, Sharma, Sandhu, Sussman, Tabibian and Girotra9 The proliferation of surveys during the COVID-19 pandemic has increased survey fatigue and reduced participation among healthcare personnel. Reference de Koning, Egiz, Kotecha, Ciuculete, Ooi, Bankole, Erhabor, Higginbotham, Khan, Dalle and Sichimba10 Thus, we sought to assess how well our recruitment protocol fared.
Our final response proportion, 34.5%, exceeds that of some COVID-19-related studies Reference Oberleitner, Lucia, Navin, Ozdych, M. Afonso, Kennedy, Keil, Wu and Mathew11 but is similar to others. Reference Byhoff, Paulus, Guardado, Zubiago and Wurcel12,Reference Momplaisir, Kuter, Ghadimi, Browne, Nkwihoreze, Feemster, Frank, Faig, Shen, Offit and Green-McKenzie13 Over half of the individuals who participated did so after one contact attempt, with no reminder messages needed. However, each reminder message in our protocol added additional participants. Without subsequent contacts, our final response would have been approximately 10 percentage points lower, and we would have had fewer controls in the study. We conclude that each contact was worthwhile except for the sixth contact, which only yielded three additional responses. It is often difficult to recruit controls for case-control studies. Reference Moorman, Newman, Millikan, Tse and Sandler14 This study demonstrates that multiple reminders increase recruitment of controls.
The added value of our reminder messages is consistent with prior survey research, Reference Edwards, Roberts, Clarke, Diguiseppi, Wentz, Kwan, Cooper, Felix and Pratap15 including among healthcare professionals. Reference Cho, Johnson and VanGeest16,Reference Meyer, Benjamens, Moumni, Lange and Pol17 Multiple modes of contact may add novelty to a different stimulus. Reference de Leeuw18 Email messages were the most effective in increasing the number of participants, likely because the emails were the only contacts with a direct link to the online survey. Study team members making phone calls reported that their conversations with prospective participants appeared helpful in encouraging participation. However, less than one-third of call attempts resulted in reaching the person live, which is consistent with contact rates reported for national public opinion research. Reference Marken19
The demographics of enrolled participants differed from the source population The use of a different measure of race (with fewer categories) in the study compared to what is used by the university resulted in substantially more reports of “other” races in the source population than in the study. The overrepresentation of younger individuals in our study was surprising, as young age is often associated with nonparticipation and underrepresentation in research. Reference Beebe, McAlpine, Ziegenfuss, Jenkins, Haas and Davern20,Reference Schneider, Clark, Rakowski and Lapane21 Nursing staff were overrepresented in the study sample relative to other staff roles. We found no significant differences in the demographics of cases compared to controls enrolled in the study, assuring that any potential bias in demographic representativeness was not unevenly affecting cases compared to controls.
The negative association between potential COVID-19 exposure at work and testing positive for SARS-CoV-2 was unexpected. One possibility is that potential workplace exposures were not as risky as outside exposures due to the ample supply, and protocols for use, of personal protective equipment in the workplace. It is also possible that the negative relationship between potential work exposure to COVID-19 and testing positive is indicative of testing bias in the study. Test-negative designs can be susceptible to bias if the exposure of interest differentially influences the propensity to test in cases versus controls. Reference Lewnard, Tedijanto, Cowling and Lipsitch22 Thus, the bias could arise if test-seeking was influenced both by work-related contact and by the likelihood that COVID-19 is the cause of symptoms. Such a situation would occur if a work-related exposure generally led to testing, regardless of clinical presentation, whereas having a clinical picture more specific for COVID-19 increased testing among individuals who did not have a work-related exposure. Consistent with this hypothesis, the apparent protective effect of work-related potential exposure was diminished (and no longer statistically significant) in a model that included fatigue and alteration in smell or taste, two symptoms that were associated with test positivity. The policy at UU Health was that employees with close contact with a coworker testing positive for SARS-CoV-2 were contacted by Employee Health and encouraged to seek testing. A bias in the opposite direction could arise if vaccinated individuals were less likely to seek testing than non-vaccinated individuals. However, we did not find any evidence that adjusting for presence of symptoms modified our estimate of vaccine effectiveness.
Research has found a variety of factors are associated with higher SARS-CoV-2 seropositivity among healthcare workers, including patient-facing positions, front-line COVID-19 healthcare positions, and shortages or lack of use of personal protective equipment. Reference Galanis, Vraka, Fragkou, Bilali and Kaitelidou23 We found that a potential exposure to someone ill outside of work, not potential exposure at work, was the primary predictor of testing positive for SARS-CoV-2. Others have reported the significance of community exposures, not workplace exposures, for predicting infection in healthcare personnel. Reference Jacob, Baker, Fridkin, Lopman, Steinberg, Christenson, King, Leekha, O’Hara and Rock24,Reference Kobayashi, Trannel, Heinemann, Marra, Etienne, Abosi, Holley, Dains, Jenn, Meacham, Schuessler, Wendt, Ten Eyck, Hanna, Salinas, Hartley, Ford, Wellington, Brust and Diekema25 These findings could also suggest that infection control measures in healthcare settings are effective at preventing transmission. Reference Hori, Fukuchi, Sanui, Moriya and Sugawara26
This study is subject to limitations. Our analysis is reliant on participants’ self-reports of their protective behaviors and possible exposures. We were unable to retain demographic data on sampled personnel who declined to participate in the study for purposes of assessing the demographic representativeness of the study participants. Thus, we were limited to using aggregate data obtained from the Electronic Data Warehouse after the study conclusion. This prevented us from assessing representativeness of the sample by week of recruitment. Our study was conducted only in English, which may have been a barrier to participation among some personnel.
This study provides multiple contributions. Our assessment of the effectiveness of recruitment attempts on enrolling healthcare personnel in a research study during a pandemic should guide future researchers in efforts to recruit healthcare professionals. Our results show the value of a multiple contact, multimode approach for increasing study participation. Second, we demonstrated that demographic characteristics are associated with study participation, but that there was no significant difference in demographic characteristics between cases and controls. Our analyses add to the literature showing that among healthcare personnel, potential exposures outside of the healthcare setting are more strongly related to testing positive for SARS-CoV-2 than potential exposures within the healthcare setting. This finding demonstrates the difficulty healthcare facilities face in preventing outbreaks among employees, as it is not possible to control community exposures. Hospitals must emphasize to personnel the ongoing risk of SARS-CoV-2 infection in the community and encourage continued testing.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/ash.2024.44
Author contribution
Brian Orleans and Matt Doane of the University of Utah assisted in data collection for this study.
Financial support
This study was funded by the Centers for Disease Control and Prevention through grant 200-2016-91799. MMM is supported in part by the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UM1TR004409.
Competing interests
All authors report no conflicts of interest relevant to this article.
Open access
