Online study design and recruitment

For those interested in exploring online research methods, Sauter et al^{Reference Sauter, Draschkow and Mack6} recently published a brief yet thorough guide to the necessary steps, tools and considerations for running online behavioural experiments. We designed our study in REDCap,^{Reference Harris, Taylor, Minor, Elliott, Fernandez and O'Neal7} a platform for research database management, which allows for participant-facing surveys and task participation via the web. Importantly, we adapted the reward task that was originally planned for use during functional magnetic resonance scanning. This task sought to measure both reward anticipation and initial response to reward for various types of images (erotic images, non-erotic pleasant images such as puppies and scenery, as well as neutral images such as furniture). REDCap's interface was well suited for this task, allowing for stimulus presentation along with user input of image ratings. A number of platforms for delivering novel, online tasks are available – from extensions of lab-based software such as PsychoPy and E-Prime to online-only platforms such as Testable – REDCap was sufficient for our needs and we were already familiar with using it. A list of various resources that can assist in online study design and implementation can be found in the Appendix.

Recruitment of participants occurred through various internet media, including university listservs, social media and the use of established platforms, such as ResearchMatch that connect researchers with potential volunteers and allow targeted advertising to specific volunteers (Appendix). Over a period 2.5 months, we recruited n = 1000 eligible participants, half were current users of hormonal contraceptives and half who did not use hormonal contraceptives. This provided us with a large data-set to apply multivariable statistical and reinforcement learning models in order to evaluate the relationship between contraception use and reward processing. Furthermore, the data-set serves as preliminary results that will hone our original hypotheses and design of an in-person, neuroimaging experiment with the most robust stimuli possible to distinguish differences between groups.

Advantages and limitations of online research

We found that one of the biggest advantages of online research is that it removes geographic barriers allowing researchers to recruit participants nationally and internationally as well as individuals from more isolated regions. The ability to complete studies from home may also eliminate issues of stigma that prevent individuals participating in research.^{Reference Bragard, Fisher and Curtis8} It also allows for the rapid accrual of large amounts of data. And because it requires minimal equipment and less investment on the part of the participant in terms of time and travel, it is also an economic research strategy that can be used by early career investigators and others who have yet to acquire significant funding.

Although online research has the obvious benefit of feasibility during the pandemic, it has known limitations and disadvantages.^{Reference Bragard, Fisher and Curtis8} The requirement for internet access and necessary computer skills is biased against low-income individuals with less education.^{Reference Singh, Girmay, Allender and Christine9} As all eligibility information is gathered by self-report with minimal interaction with research staff, there is increased risk of inadvertent erroneous reporting or frank deceptive practices. Some individuals may rush through study questions with little thought to their responses whereas others may even set up automated methods or ‘bots’ in order to redeem payment with minimal effort. These risks can be minimised by putting in quality control measures, such as using online CAPTCHA (Completely Automated Public Turing test to tell Computers and Humans Apart) tools and interspersing questions that deliberately ask for a certain response (i.e. ‘Please select option 4’).

Even with quality assurance strategies, there are differences in the environment in an online versus lab setting for cognitive tasking that need to be better understood. Whereas one participant might be participating on their computer in a quiet environment, another might be on their phone while watching television. Such variation might be particularly relevant when asking participants to complete tasks with greater cognitive demands.^{Reference Bauer, Iverson, Cernich, Binder, Ruff and Naugle10} However, researchers considering online experiments should be reassured by data suggesting that attention to tasks^{Reference Clifford and Jerit11} and data quality^{Reference Casler, Bickel and Hackett12,Reference Dandurand, Shultz and Onishi13} are similar between online and lab participants. Furthermore, anonymous, online tasking might promote more naturalistic behaviour, when the respondent is less influenced by an unfamiliar environment or the experimenter. The latter is known to lead to social desirability or demand effects, which may be particularly important in experiments like ours, in which we ask individuals to rate their experience of erotica.^{Reference Milani, Brotto and Kingstone14} In any event, as the pandemic has promoted a shift for online clinical and research assessment, our field will need to better understand its implications for study design, analysis and interpretation. Thus, ongoing comparisons of similar methods that are conducted in laboratory environments versus at home via the internet will be necessary.