Introduction
Web-based syndromic surveillance has been used in the monitoring of influenza-like illnesses since 2008 in the European Union [Reference Paolotti1]. When the COVID-19 pandemic began, both existing and newly developed surveillance systems were implemented to monitor and prevent further spread of the virus, with the aim of protecting public health. In Denmark, COVID-19 surveillance included surveillance of individuals for the number of tests conducted, confirmed cases, hospital and intensive care unit (ICU) admissions of SARS-CoV-2 patients and COVID-19-related deaths. The national surveillance efforts also included serological studies to establish the prevalence and patterns over time of antibody seropositivity [Reference Espenhain2, Reference Krogsgaard3], analysis of wastewater samples [Reference Krogsgaard4], monitoring vaccinations given at the individual level and assessing symptoms of long COVID [Reference Witteveen-Freidl5].
In addition to the surveillance systems mentioned earlier, at the beginning of the pandemic, an existing symptom surveillance system set up for influenza [Reference Kjelsø6] was used to monitor COVID-19-like symptoms in the population. However, it soon became clear that this system, built for 10000 participants, lacked the capacity to meet the surveillance demands of COVID-19, where many more wished to contribute. This was highlighted by a Facebook group formed within the first weeks of the pandemic period with several hundred thousand participants who recorded their own symptoms – albeit not in a way so that public health information could be readily extracted. Consequently, Statens Serum Institut (SSI) in Denmark developed a new symptom monitoring system, COVIDmeter, which was officially launched on 8 April 2020, inviting participants to respond to a weekly questionnaire about potential illness symptoms experienced in the preceding week.
Guidelines from the European Centre for Disease Prevention and Control (ECDC) [7] and World Health Organization (WHO) [8] recommended that the COVID-19 epidemic be monitored through self-reporting by citizens, enabling authorities to detect changes in the epidemic at an early stage. Examples of self-reporting systems in Europe was the ZOE COVID Study in the United Kingdom [Reference Menni9] and Symptometer in Norway [10]. Some systems, like Influmeter [11], were already in place, while others were developed during the pandemic. We find it important to critically assess the performance and value of these systems as they would potentially find use in future epidemic situations.
The aims of this study were (a) to describe the development, participation rates and application of COVIDmeter during the COVID-19 pandemic, (b) to assess the signal detection capabilities of COVIDmeter compared to other key surveillance indicators, including one based on ECDC’s case definition, and (c) to examine potential changes in reported symptoms across different circulating variants.
Methods
Purpose and development of COVIDmeter
At its core, COVIDmeter was designed as a national surveillance tool to monitor the proportion of Danish residents who had experienced COVID-like illness in the past week. The system relied on an open cohort of volunteers from the general population (anyone was able to join) who completed the online questionnaire each week.
Independent of PCR testing, COVIDmeter was developed to estimate the proportion of people currently infected in Denmark, based on the number of participants reporting symptoms consistent with COVID-19 (COVID-like illness) each week, and provide insights into both national and regional levels. Additionally, data were collected on healthcare system contact, exposure to individuals who tested positive for COVID-19 and changes in vaccination status. To estimate the proportion of individuals likely infected with COVID-19, a case definition was developed to classify COVID-like illness cases.
The platform was developed in a collaboration between three governmental agencies and a private company: the Danish Ministry of Health, Agency for Digital Government, SSI and the private IT company, Netcompany.
Recruitment and retention
To recruit participants for COVIDmeter, the new electronic platform was introduced at Government press conferences, where citizens were encouraged to sign up to the community cohort. Additionally, individuals previously enrolled in Influmeter were invited to transition to COVIDmeter. At certain times, advertisements promoting COVIDmeter were displayed on the front page of sundhed.dk (Denmark’s health portal for individuals), encouraging citizens above the age of 15 years to register for the system.
Participants who registered, subsequently received a weekly email to remind them to complete the questionnaire, including a direct link to the survey. If the questionnaire was not completed within 24 h, a follow-up reminder email was sent. The survey link redirected participants to sundhed.dk, where they logged in using NemID/MitID (Denmark’s digital identification system, compulsory for all national residents) to access and complete the questionnaire. Thereby participants were uniquely identified in the system each time a response were reported by them. To further enhance participant retention, newsletters were sent to all active users in December 2021 and July 2022.
As part of the scale-down of the Danish COVID-19 response, COVIDmeter was officially closed down on 27 March 2023. Upon closure, a concluding newsletter was sent to all active participants, at the same time thanking for their participation.
Description of COVIDmeter and related variables
COVIDmeter collected information on general symptoms (primarily respiratory symptoms) via a secure and anonymous questionnaire system. To be eligible for participation, volunteers had to be at least 15 years of age. Upon registration, participants provided basic demographic information, including their month and year of birth, sex and the postal code where they stayed most frequently. Additionally, participants were able to opt in to receive email notifications reminding them to complete the online survey each week.
The survey consisted of four core questions, with additional subsections of the questionnaire triggered when answering ‘yes’ to any of them.
The four core questions were:
-
1. Have you felt ill in the past week?
-
2. Have you been in close contact with someone who has tested positive for COVID-19 since you last completed the questionnaire?
-
3. Have you been tested for COVID-19 since you last completed the questionnaire?
-
4. Have you been vaccinated against COVID-19 or influenza since you last completed the questionnaire?
Answering yes to question 1, resulted in a detailed list of symptoms folding out. Throughout the pandemic, the questionnaire was adjusted several times to reflect changing surveillance needs (see Supplementary for the full questionnaire in Danish).
Responses were pseudo-anonymous, meaning that SSI did not have access to participants’ personal identities. Instead each participant was assigned a unique identifier, allowing their weekly responses to be linked over time by the SSI. In December 2020, participants were encouraged to complete an additional one-off questionnaire collecting basic demographic and health-related information, including highest level of education attained, living alone, hay fever/allergies, regular medication for diabetes, chronic lung disease, heart disease, kidney disease, weakened immune system, smoking status, height and weight. Participants who joined after December 2020 were required to complete this questionnaire upon registration.
Data sources
For this analysis, we used data collected from COVIDmeter questionnaire responses from 6 April 2020 to 27 March 2023. We excluded all responses from persons under 18 years of age, as we received very few. The data on the Danish population regarding sex, age and geographical region of residence (5 such exist in Denmark) were obtained from Statistics Denmark [12]. Several other surveillance indicators were used to monitor the COVID-19 epidemic in Denmark. These included:
-
– Incidence rate, measured as the number of confirmed cases per 100000 inhabitants
-
– New hospital admissions among individuals who tested positive for SARS-CoV-2
-
– COVID-19-related deaths
All such surveillance data were obtained from SSI registries on 15 January 2024 [13].
Data to develop the case definition for COVID-like illness
For the development of the case definition in 2020, data from two different data collections were used. The first consisted of data collected on symptoms in the initial phase of the pandemic from Danish citizens with either laboratory-confirmed or laboratory-excluded SARS-CoV-2 infections. A total of 1065 individuals with general symptoms tested as part of sentinel surveillance were included. This dataset consisted of persons testing both positive and negative for SARS-CoV-2. The second data source consisted of COVID-19 cases which were interviewed by phone concerning symptoms – this was done by the SSI in the initial phase of the pandemic. A total of 276 patients were included in this dataset [14].
Case definitions
COVID-like illness
The case definition used to calculate COVID-like illness in COVIDmeter was based on data from the two surveillance systems described earlier. A logistic regression model was developed to assess the associations between different symptom combinations and PCR-confirmed COVID-19 infection and, using fivefold cross-validation, to develop a prediction model calibrated to maximize the area under the receiver operating curve.
To be classified as a case with COVID-like illness, an individual needed to experience both general and respiratory symptoms, and have a model-derived predicted probability score of at least 0.29, a threshold identified to maximize the Youden index. General symptoms were (at least one of the following): fever, chills, headache, muscle or body aches, chest pain, fatigue, loss of appetite or stomach aches. Respiratory symptoms were (at least one of the following): runny or blocked nose, cough, sore throat, short of breath or sneezing.
Predicted probability calculation
The probability of being a case with COVID-like illness was calculated using the following logistic regression-derived prediction model:
LP = −1.8529 + (2.0566*‘change in taste’ AND ‘change in smell’) + (1.8660*‘fever’) + (−1.1702*‘sore throat’) + (−0.7457*‘headache’) + (0.9500*‘diarrhea’ + (−‘0.7727*stomach aches’).
odds = exp.(LP).
prob. = odds/(1 + odds).
ECDC case definition
During the COVID-19 pandemic, ECDC defined a suspected case as an individual who had experienced at least one of the following four symptoms in the past week: cough, fever, change in taste or smell, being short of breath.
Data analysis
This analysis includes all COVIDmeter participants with complete data on birth month and year (above 18 years), postal code and sex who completed at least one questionnaire during each of four distinct variant periods. The four periods corresponded to those in which distinct SARS-CoV-2 variants were almost exclusively present in Denmark [Reference Michlmayr15].
-
– Index-(Wuhan)-dominant period (1 February–31 December 2020)
-
– Alpha (B.1.1.7) dominant period (20 February–15 June 2021)
-
– Delta (B.1.617.2) dominant period (4 July–20 November 2021)
-
– Omicron (B.1.1.529) dominant period (21 December 2021–31 January 2022)
We compared demographic characteristics (sex, region and age group) of COVIDmeter participants to the general population above 18 years. Age groups were categorized into 10-year intervals, except for the youngest (18–29 years) and oldest (80+ years) age groups. Regional classification is based on the postal code where participants reported spending most of their time.
We described the proportion of self-reported symptoms among COVIDmeter participants who tested positive for SARS-CoV-2 during the specified periods. Additionally, among all respondents who completed the weekly questionnaire at least three times, we calculated the proportion of COVIDmeter participants classified as a COVID-like illness based on the case definition described earlier and ECDC’s case definition, including 95% confidence intervals. Furthermore, the cumulative per cent for the two case definitions were also calculated. To assess the association between the two case definitions over time, we calculated the Pearson’s correlation coefficient. To test the robustness of the findings, we also performed non-parametric rank-based correlation analyses using Spearman’s ρ and Kendall’s τ.
Participants were classified as COVID-like illness cases from the date they completed the questionnaire and met the criteria of either of the case definitions. If participants met one of the case definitions for two consecutive weeks, they were only counted as cases in the first week.
We compared the COVIDmeter case definition with:
-
– ECDC’s case definition
-
– Incidence per 100000 inhabitants per week
-
– Number of new hospital admissions of SARS-CoV-2-positive individuals (rolling 7-day average calculated with respect to the mid-point)
-
– COVID-19-related deaths per week
Incidence per 100000 inhabitants per week, number of new hospital admissions of SARS-CoV-2-positive individuals and COVID-19-related deaths per week were normalized to a scale from 0 to 3. These comparisons covered the entire period from 6 April 2020 to 27 March 2023.
Results
During the operational period of COVIDmeter, a total of 142938 individuals registered as participants in the community cohort, of whom 140102 (98%) completed at least one weekly questionnaire. In the initial weeks, participation was particularly high, with over 60000 weekly responses recorded. During 2020 (March 6–December 31), a total of 2230585 completed individual questionnaires were received, averaging 53000 responses per week. In 2021, we received 2061660 completed questionnaires, corresponding to an average of 40000 responses per week. During 2022, a total of 1272606 completed questionnaires were received, with an average of 25000 responses per week. In 2023 (January 1–March 27), 239789 completed questionnaires were received, averaging 18000 responses per week. In total, 5804640 completed questionnaires were received during the study period (Figure 1). Most participants had their initial survey reply at the beginning of the operational period. Figure 1 also shows when newsletters and the final farewell message were sent to active participants.
COVIDmeter: weekly number of completed questionnaires and initial survey reply – 6 April 2020 to 27 March 2023.
Table 1 presents the demographic distribution of COVIDmeter participants who completed at least one questionnaire during each of the four distinct periods, compared to the general Danish population aged 18 years and older in 2022. COVIDmeter participants were not fully representative of the general Danish population. Across all periods, women and individuals aged 40–69 years were overrepresented compared to national demographics.
Demographic characteristics of the participants during the index-, Alpha-, Delta- and Omicron-dominant periods and the Danish population ≥18 years in 2022
a At the beginning of the period.
The number of participants varied across the four periods, ranging from approximately 130000 during the period of the original SARS-CoV-2 index strain to around 44000 during the Omicron period. Of those who tested positive for SARS-CoV-2 in the preceding week, the most commonly reported symptoms overall were headache, fatigue, muscle or body aches, cough and fever (Figure 2). There was some evidence of a change in symptoms, depending on the circulating variants, with cold-like symptoms. For example, cough, sneezing, sore throat, and blocked or runny nose were more frequently reported during the omicron period than during the periods dominated by the earlier variants. Changes in the sense of taste and smell were particularly common during the periods dominated by the ancestral strain as well as during the Delta variant period, and least commonly reported during the Omicron period. Fewer participants reported experiencing symptoms during the Alpha period than during any of the other three periods.
Self-reported symptoms among participants who tested positive for SARS-CoV-2 in each of the four periods dominated by a single viral variant.
The COVIDmeter and ECDC case definitions followed a similar trend; however, greater fluctuations were observed when applying the ECDC case definition compared to COVIDmeter’s. Throughout the pandemic, a higher proportion of participants met the ECDC case definition than the COVIDmeter case definition. The cumulative per cent for each of the case definitions also highlights the differences between the two case definitions. After 3 years, the cumulative per cent of COVID-like illness were 112% using the COVIDmeter but 722% using the ECDC case definition (Figure 3).
COVID-like illness calculated using the COVIDmeter and ECDC case definition among the COVIDmeter community cohort. Cumulative per cent shown as dashed lines.
There was a strong positive correlation between the two case definitions (Pearson’s r = 0.89, p < 0.001), indicating that the trends closely followed each other. The result was robust to rank-based methods (Spearman’s ρ = 0.86; Kendall’s τ = 0.68).
Figure 4 compares the proportion of COVID-like illness in COVIDmeter with incidence, admission and COVID-19-related deaths. COVID-like illness appears to increase before the incidence rate rises, particularly in December 2020, August 2021 and May, September and December 2022. A similar pattern is observed for hospital admissions of SARS-CoV-2-positive individuals, where COVID-like illness appears to rise before hospitalizations, specifically in December 2020, August 2021 and June, September and December 2022. However, the high increase in COVID-19-related deaths observed around the turn of 2020/2021 is not reflected in COVID-like illness. In contrast, subsequent peaks in mortality between November 2021 and April 2022, as well as in August, October and December 2022, are visible in COVID-like illness. In these periods, the rise in COVID-like illness precedes the increase in deaths, suggesting early detection of trends in morbidity.
COVID-like illness compared with PCR test-of-individuals incidence per 100000 inhabitants (upper panel), hospital admissions (middle panel) and death related to COVID-19 (lower panel).
Discussion
This study describes COVIDmeter, a national participatory surveillance platform developed to monitor COVID-like illness among Danish citizens during the pandemic for public health planning purposes. Our findings suggest that COVIDmeter provided early indications of epidemic trends, with the proportion of COVID-like illness rising before key traditional surveillance indicators such as incidence and hospitalizations. These results highlight the potential value of symptom-based surveillance as an early warning system for infectious diseases.
Most people reported experiencing one or more symptoms after testing positive for SARS-CoV-2 with cold-like symptoms more commonly reported during the Omicron period, while changes in the sense of taste and smell were less commonly reported during this period. Similar findings were published in a UK study which found that experiences of ‘sore throat’ were more commonly reported, and that loss or altered sense of smell were less commonly reported, by people infected during a period of Omicron dominance than by people infected during a period of Delta dominance [Reference Menni16]. Interestingly, we also found that more people reported experiencing no symptoms after testing positive during the Alpha period than after testing positive during any other period, contradicting established evidence of the Alpha variant generally being more virulent than the preceding ancestral strain and the subsequent Omicron variant [Reference Varea-Jiménez17–Reference Wahlström19]. Likely, our finding is an artefact of changing community testing strategies during the pandemic as large-scale SARS-CoV-2 PCR testing of the general population took place during the Alpha period irrespective of illness or suspected infection.
The case definition developed for estimating COVID-like illness trends closely followed the ECDC case definition over the course of the pandemic. Nonetheless, a consistently higher proportion of participants fulfilled the ECDC case definition, a pattern also reflected in the cumulative estimate of COVID-like illness. Notably, when compared to seroprevalence estimates from Denmark in 2020 and 2021 [Reference Espenhain2, Reference Krogsgaard3], our case definition demonstrated closer alignment with these estimates than the ECDC case definition.
Our data demonstrate that participatory surveillance can serve as an early detection tool for tracking infection trends and changes in transmission dynamics, particularly in the early stages of a pandemic. This aligns with findings from previous research [Reference Menni9, Reference Brainard20–Reference Sudre23], which highlights the role of self-reported symptom data in pandemic surveillance. Even in Denmark, where we have strong register data in real time [Reference Gram24], participatory surveillance has been valuable, especially in the initial phase of the pandemic and in periods with low test activity. In Denmark, data from COVIDmeter were used weekly to evaluate the trends; knowing that the COVID-like illness proportion was not reflecting the proportion of infection as such in the whole population.
Participatory surveillance offers several advantages over traditional surveillance systems. It is cost-effective, rapid and provides health authorities with data on individuals who might not otherwise seek medical care. Particularly when tests are limited or unavailable, such systems may prove highly valuable as surveillance tools. During the pandemic, changes in public health measures, restrictions and testing strategies occurred [Reference Munch25], which might have impacted more traditional surveillance systems, leading to fluctuations in data availability. In contrast, the COVIDmeter community cohort remained more stable and consistent over time, providing a continuous dataflow. To support participant retention, newsletters with general updates and reminders were sent two times during the study period. However, these had limited effect, and no other retention strategies were tested. Future similar platforms may benefit from exploring alternative approaches, such as personalized feedback or individual dashboards, to increase sustained engagement.
COVIDmeter as a system also had several limitations. The platform was developed over a very short time period and technical issues occasionally disrupted the system, leading to temporary service interruptions in 2020. Furthermore, COVIDmeter data were self-reported (by design), which introduced several challenges, including missing or incorrect responses. Further participation was voluntary, and required digital signature log-in, leading to selection of a non-representative sample of the Danish population as measured by basic demographic parameters. Women were overrepresented as were individuals aged 40–69 years, while younger adults and elderly population segments were underrepresented. This may have influenced the COVID-like illness proportion estimates, which could be even more prominent among parts of the population less likely to test or interact with the healthcare system. Future implementations of similar systems might benefit from giving more consideration from the outset to reweighting the responses in order to better match the demographics of the general population, for example, with respect to age, sex and ethnicity. The open voluntary participation differs from other participatory surveillance systems, such as Symptometer in Norway, where participants were randomly selected from the national population registry [26]. This sampling approach may help to alleviate under- or overrepresentation of specific population groups, especially when selection is guided by existing knowledge about participation rates in different demographic groups. Using the national population registry could have enabled more detailed analyses. In particular, it would have allowed for household-level data collection, not only to study transmission dynamics within families, but also linking to national health registers. However, this alternative study design has other limitations relating to difficulty maintaining anonymity, complexity in setup and ensuring compliance with data protection regulations.
Another challenge with community cohorts based on voluntary self-reported data is that individuals may be more likely to sign up after experiencing symptoms. To mitigate first-time reporting bias, only participants who completed at least three surveys were included in the analyses. Furthermore, citizens were more active in reporting when they had symptoms, than when they did not have symptoms, which may have led to an overestimation of the proportion of infected individuals. Although this is likely to effect point estimates, we assume that this bias is similar across time, thus not affected observed trends disproportionately.
The case definition used to estimate the proportion of COVID-like illness was based on laboratory-confirmed and laboratory-excluded SARS-CoV-2 infections developed at the beginning of the pandemic and was not adjusted thereafter. As shown here, symptoms among individuals who tested positive for COVID-19 changed over time, suggesting that the case definition for COVID-like illness could have been optimized during the pandemic. Further misclassification may have occurred, when other respiratory viruses resumed circulation, as differentiating between various respiratory infections can be challenging. This have also been shown in another Danish surveillance system [Reference Larsen27]. However, sentinel surveillance from Denmark shows that from spring 2020 to the season 2021/2022, a limited amount of other respiratory disease were circulating [28, 29]. The increase in the estimated proportion could be attributed to other respiratory diseases resurging after relaxation of non-pharmaceutical interventions, thus in effect being a measure of upper respiratory airway disease. Finally, asymptomatic cases were not captured, meaning that COVIDmeter only reflected trends among symptomatic individuals, unlike PCR-based or serological surveillance.
Our findings highlight the potential role of digital self-reporting platforms in complementing traditional surveillance approaches. COVIDmeter successfully engaged over 140000 participants, demonstrating high public interest in contributing to epidemic monitoring. With increasing reliance on syndromic surveillance in public health, similar platforms could be valuable for real-time detection of emerging infectious diseases, particularly when laboratory testing capacity is limited. During the COVID-19 pandemic in Denmark, data from the platform were used in the weekly situation report to inform the epidemic decision making and the general public about the respiratory illness trend. In Denmark, syndromic symptoms surveillance is currently ongoing in the form of Influmeter, while other approaches to syndromic symptom surveillance are being considered, including random sampling of different individuals each week to reduce bias and have a system ready for the next pandemic [26].
In conclusion, COVIDmeter provided valuable insights into COVID-19-like illness trends, demonstrating early detection potential relative to traditional surveillance indicators. While limitations such as selection bias and self-reporting errors must be considered, the system underscores the feasibility of self-reported symptom surveillance as a rapid, scalable and cost-effective tool for infectious disease monitoring. In a future disease X pandemic, we would recommend setting up such systems for rapid surveillance of epidemic trends in the population base, unaffected of testing for pathogens.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0950268825100678.
Data availability statement
COVIDmeter data may be made available from the corresponding authors upon reasonable request or application via Rigsarkivet (Danish National Archives).
Acknowledgements
The authors thank all participants who, week after week during the pandemic, participated in the community cohort. We also thank Margrethe Harbo and other dedicated employees at the Agency for Digital Government (Digitaliseringsstyrelsen), The Danish Health Data Authority (Sundhedsdatastyrelsen) and the Netcompany who helped develop and maintain the system and to handle its many technical, legal and public governance aspects.
Author contribution
Design of COVIDmeter: S.E., F.T.M., I.G.H. Development, data collection and concurrent analysis: S.E., P.K.M., C.H.H. Data curation: P.K.M, C.H.H., S.K.N., A.T.I. Formal analysis: P.K.M, C.H.H., S.K.N., A.T.I., S.E. Visualization: P.K.M., C.H.H. Writing – review and editing: P.K.M, C.H.H., F.T.M., S.K.N., A.T.I., I.G.H, S.E. Writing – original draft: P.K.M., C.H.H. Funding acquisition: F.T.M., S.E. Supervision: S.E.
Competing interests
All authors declare no competing interests.
Funding statement
Building, maintaining and running COVIDmeter were done through direct support from the Danish Government during the COVID-19 pandemic.
Open access
