Introduction
There has been some question about whether the Sturgis Motorcycle Rally in Sturgis, SD in August 2020 would be a COVID-19 superspreader event. A superspreader event occurs when a gathering causes a greater than average number of people to become infected. Essentially, it is an outlier contributor to infection rates. Reference Cave1 Media Reference Saint Louis, Moyer, Dasgupta and Hall2 and experts Reference Bromage3 have speculated about COVID-19 superspreader events, but the expert opinion was that these risks would be determined by the prevalence of mask wearing and social distancing in Sturgis particularly. Local media reported that the 10-day attendance total was estimated at 460 000, Reference Bonnet4 thus, risks of a superspreader event could be magnified by such a large gathering.
At the time of the rally, non-essential travel was discouraged by the CDC, which prevented in-person measurement. In response, we used a simple and rapid remote measuring technique for gathering public health information by using publicly available video. The City of Sturgis provides a way to gather this information through 3 web video streams of popular city locations (sturgismotorcyclerally.com). These live views of city intersections and other locations help build interest in the event, and provide entertainment and distant engagement for those not attending through concurrent commenting via the YouTube channel’s live comment feature. These video streams can also support remote measurements of public health information (such as mask wearing), and provide data for predicting imminent public health disasters like a superspreader event and subsequent nationwide disease spread originating from the event.
Both private and public live videos have been used to analyze behavior (e.g., crowd management to reduce stampede risks). Reference Johansson, Batty, Hayashi, Al Bar, Marcozzi and Memish5 We found an additional use for the streams as a tool for gathering public health data such as adherence to mask-wearing recommendations and other public mandates.
To begin to get a measurement of mask wearing at the Sturgis Motorcycle Rally, we observed the video stream, and recorded a series of sample videos while concurrently measuring mask wearing using a human observer with minimal training. This method for collecting public health data through widely available live streams may provide a valuable new tool that reduces latency between public health behavior and governmental decision making.
Method
Materials
We observed and recorded the live streams of the public streets in Sturgis, SD available through https://www.sturgis-sd.gov/webcams-in-and-around-sturgis. These live streams are webcams that pan and zoom over the major public streets in Sturgis, SD and are publicly available on YouTube. We looked at: “Live View from Hotel Sturgis on Main St. and Harley-Davidson Way in Sturgis SD.” This is a view from a camera approximately 15 feet (5m) off the ground overlooking the intersection of Main St. and Harley-Davidson Way. An example screenshot is shown in Figure 1. The screenshot has been down-sampled from the original video quality. Markers indicate which pedestrians can be assessed using this method (0 masked and 18 non-masked). The camera pans and zooms on a fixed schedule during the day; at night, it appears to be sometimes manually controlled. The streams are in color, and are available at different resolutions, we used the highest resolution available (HD, 1080p). Behavior on public streets where there is not an expectation of privacy does not require IRB approval to study, and we confirmed this through contact with a PSU IRB representative.
A screenshot from one of the publicly available video streams for the event. Source: sturgis 10aug20.mov.
Streams were recorded from live streams using QuickTime (V10). The analysis here is primarily focused on a single 25-minute file (sturgis 10aug20.mov), and additional videos were used to assess coding reliability. We also collected an additional ∼40 hours of video (180 GB) recorded during the rally. The videos from this process have been added to Databrary.org (project name: Sturgis SD Main St. 2020 Motorcycle Rally data), where other researchers can use the video to duplicate these results, extend them, and perform other analyses.
Video coding protocol for mask wearing
The research assistants (RAs) remotely observed the crowds and counted people. People were counted as wearing masks if they had a mask on covering their mouth up to their nose, and as not wearing if they did not have a mask on (properly or not, minimally covering their mouth).
Each RA observed the selected video using a video playback tool (e.g., VLC). The RAs were provided the criteria in Table 1 for counting pedestrians and noting their mask-wearing status. Pedestrians that were in frame on public sidewalks or streets were counted for the duration of the video, but non-public areas like bar patios were excluded. During moments when the camera panned, coding was paused until the camera stopped moving. Pedestrians were counted as unmasked, masked, or unclear, with the masked pedestrians required to have mouth and nose covered by the mask. Masks were not distinguished by type (e.g., surgical masks, neck gaiters), however, non-mask head coverings like motorcycle helmets were excluded. Coders were instructed to count someone as masked if the pedestrian began wearing a mask within ~5 seconds of eligibility (i.e., donning mask after removing helmet). People in or on vehicles (including motorcycles) were excluded until they become a pedestrian or talked with a pedestrian.
Coding scheme
Results
The 25-minute video from the first analysis showed 512 pedestrians over 25 minutes starting at 6:50PM local time on August 10, 2020. The video was high-quality and well-lit, and information from the video stream indicated the weather was clear and sunny. The first RA counted 12 mask wearers, a ratio of 2.3% mask-wearing during that span. The number of people observed during the short period of time was relatively high because of the large number attending and participating in the Sturgis Motorcycle Rally. These numbers are comparable to other Sturgis videos shot by private citizens and uploaded to YouTube as travelogues (see Table 2, Segment 5).
Results from 5 segments lasting 5 minutes and coded by 3 RAs using the coding protocol in Table 1
Source is the filename on Databrary.org, Time is local date and time at start of the segment in 24-hour notation, and Time coded is the time in the source video in mm:ss format. The “Sturgis walk” video was uploaded to YouTube by its presenter.
The coding protocol was checked for reliability through independent coding by 2 further RAs of 5 different 5-minute video segments from a variety of dates and times. There were 4 streams in Sturgis available at the time (3 from the city and 1 from a motorcycle shop). We did not include a systematic sample of video segments because this was a pilot project. However, we tested for coding reliability by adding 2 additional observers and comparing their observations on 5 different video sections.
Table 2 shows the results from each coder for each segment. The pair-wise correlations between all coders are shown in Table 3. While the correlation between the overall count was weaker in 1 instance, there were reliable pair-wise correlations between all 3 coders for the number of masked people counted and the overall proportion observed wearing a mask.
Correlations between all pair-wise combinations of coders for their responses on the 5 segments lasting 5 minutes
An asterisk (*) indicates a reliable correlation at P < 0.01 (N = 5).
Because there were many stimuli in the scenes, the RAs had trouble coding consistent counts for the overall number of people observed. While this did not prevent strong correlations between the coders for the count of masked subjects and the overall proportion, the lack of agreement for the count of the stimuli prevents us from using kappa to assess reliability.
A multiple correlation between coders on all 3 measures was also conductedFootnote i . The multiple correlation was reliable for the number of masked people observed (R2 adj = 0.990) and for the proportion of mask-wearing observed (R2 adj = 0.974). The correlation for total count was not reliable (R2 adj = 0.639).
These correlations show that with minimal training, RAs were able to get reliable counts of the signal (masked), however, their count of the noise (non-masked) was more variable. While reliability was high among the coders in the test segments, post-coding discussions with RAs informed us that the difference in count is likely due to differences in total time spent per video segment and the quality of the video of the edge cases (i.e., distant figures being included or excluded depending on RA).
Discussion and Conclusions
We were able to rapidly analyze public health behavior using online video. The results, if duplicated and extended, can be used to generate public health reports and to inform policy and simulations in this area. We found strong correlations (both pairwise and multiple) for masks observed and overall proportion of mask wearing when using 3 different coders across 5 segments. While this method does not provide precise measures, the data can be rapidly collected and disseminated to public health officials, which can help them to better understand the public response to public health mandates.
We characterize this method as ‘rapid’ because it was able to provide estimates of mask-wearing proportions within a single day once the coding protocol was established. The human coders took about 3 - 4 minutes per 1-minute of video. The rate-limiting factor was the human coder due to their additional responsibilities to ensure an accurate count (e.g., checking that someone is only counted once). With practice the coders would be able to code more quickly, and the current method is already likely to return results more quickly than other methods for gathering mask-wearing adherence (e.g., surveying people on self-reported mask-wearing compliance). Reference Fischer, Adrien, Silguero, Hopper, Chowdhury and Werler6 We found no other examples of direct counting methods to compare, and the time to perform this approach is consistent with times to do more complex video coding that takes 8 - 10 min/min. Reference Ericsson and Simon7,Reference Ritter and Larkin8
This approach has several applications for disaster management and public health. Understanding the prevalence of mask wearing and similar behavior at this and other events could be useful for allocating resources and predicting the impact of such mass gatherings. One way to gather this data is through observation of crowds using publicly available video from web cams. Other webcams that are in use in cities, mega-churches, and sporting events, for example, have a great potential to help predict spread and needed responses.
Limitations
There are some limitations to this approach that should be noted. A more formal method would require using more formal accounting and annotating methods to collect data. For example, the method could account for types of masks and how to handle partial compliance (i.e., mask covers mouth only). There are numerous tools for doing more detailed video coding protocols (e.g., CowLog), Reference Hänninen and Pastell9 some provide automatic or semi-automatic analyses. Coding could also be performed more collaboratively across coders to get agreement on each frame.
Data gathered with this method should also be analyzed cautiously to avoid over-stating its meaning. The mask-wearing proportions were reliable, however, this is still an unusual coding task with sparse data. This method is not designed to address adjacent measures (e.g., overall attendance at an event), the specific method described provides a rapid approximation of mask-wearing adherence for a given time and location. Reliability of the measure improves with additional time and effort expended, but we believe this method’s strength is providing near-real-time public health data with minimal investment.
This method uses publicly available data, where the identity of the individuals cannot be ascertained. This is an advantage in some circumstances and a disadvantage in others as this method is only applicable to public areas, while private areas are not available for inspection with this technique. Larger public crowds would suggest that private areas would be more crowded as well. Publicly available videos from the event on YouTube suggest that this is the case. This method may also not account for mixing across event attendee quaranteamFootnote ii groups.
Future Work
Other streams and recorded videos could be used. There are numerous videos on YouTube documenting behavior during the Sturgis Motorcycle Rally at other venues (e.g., campgrounds, bars) that could also be examined. Furthermore, more complex methods for coding and analyzing the data could be integrated into the process, for example, the prevalence of mask wearing at an event could be analyzed alongside infection data of intermediary locations between the attendee’s home region and the event region by using license plate recognition technology and return navigation forecasts to predict way point infection risk. Similar work has already begun using anonymous cell phone data to assess the home counties of the attendees of the event to identify possible future infection rates in those counties. 10 Future work could also assess the frequency of close contacts between individuals seen on videos. Analyzing pods of people at these events could provide additional information on likelihood of disease spread across groups, but this may require additional knowledge about the people being observed and more detailed video analysis.
This method could also be used to examine public health measures such as speeding on roads, helmet-wearing by motorcyclists, and behavior by police officers including their use of PPE. There are numerous webcams for public roads that would make this possible, though they range in image quality and frame rate (e.g., snapshots of I-90 via The National Weather Service, 11 and live HD videos in CA from the California Department of Transportation 12 ). Ideally, the analysis would concurrently record or would work from recordings, this method could also be compared across city centers (or similar locations) to perform similar analyses and be compared with the results from the present study.
This approach has several implications for disaster medicine. This approach could be used in a disaster situation to measure number of injuries. Where recordings are available from tourists, the number of buildings impacted by a hurricane or tornado could start to be computed.
Our recordings are stored in Databrary.org that allows later analyses of mask-wearing proportions across days to be computed, as well as other analyses by other researchers. Extending this analysis to indoor locations would also provide additional information on public health risks. It could be extended or repeated to provide estimates of BMI, race, and other demographic measures in a given place and time; at a minimum, videos could be cached of public events using this technique for later analysis.
Concluding comments
The use of public video is a useful technique for accessing disasters. This example shows that public video can be useful for public health disasters as well as more physical disasters. If replicated, this approach can document events such as superspreader events, and provides a tool for public health officials to gather low-investment rapid turnover data about public health compliance that can be used in predictive models. A summary example helps show how this method could be used: an analyst could estimate mask wearing at a local farmers’ market within 120 minutes (given a live camera stream) and report results within the day.
Finally, we can note that our hypothesis that the Sturgis Rally would classify as a superspreader event was borne out through epidemiological studies of movements of people to and from the rally Reference Dave, McNichols and Sabia13 and from genomic sequencing of cases in neighboring states showing associations with the motorcycle rally Reference Firestone, Wienkes and Garfin14 . With rapid adoption of the methodology proposed in this paper, public health experts could have had preliminary estimates of mask wearing before the event ended, and planned accordingly, or even after the first day.
Acknowledgements
Amanda Clase, Steve Croker, Ed Glantz, Clare Robson, and Deja Workman provided useful comments. We also thank Arunima Maheshwari and Clare Robson for their help in developing the video coding protocol and coding these videos.
Open access
