Hostname: page-component-848d4c4894-75dct Total loading time: 0 Render date: 2024-05-28T09:41:21.040Z Has data issue: false hasContentIssue false

Novel digital methods for gathering intensive time series data in mental health research: scoping review of a rapidly evolving field

Published online by Cambridge University Press:  15 November 2022

Anita Schick
Affiliation:
Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Heidelberg, Germany
Christian Rauschenberg
Affiliation:
Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Heidelberg, Germany
Leonie Ader
Affiliation:
Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Heidelberg, Germany
Maud Daemen
Affiliation:
Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
Lena M. Wieland
Affiliation:
Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Heidelberg, Germany
Isabell Paetzold
Affiliation:
Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Heidelberg, Germany
Mary Rose Postma
Affiliation:
Department of Psychiatry and Neuropsychology, School for Mental Health and Neuroscience, Maastricht University, Maastricht, The Netherlands
Julia C. C. Schulte-Strathaus
Affiliation:
Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Heidelberg, Germany
Ulrich Reininghaus*
Affiliation:
Department of Public Mental Health, Medical Faculty Mannheim, Central Institute of Mental Health, Heidelberg University, Heidelberg, Germany Centre for Epidemiology and Public Health, Health Service and Population Research Department, Institute of Psychiatry, Psychology & Neuroscience, King's College London, London, UK ESRC Centre for Society and Mental Health, King's College London, London, UK
*
Author for correspondence: Ulrich Reininghaus, E-mail: ulrich.reininghaus@zi-mannheim.de
Rights & Permissions [Opens in a new window]

Abstract

Recent technological advances enable the collection of intensive longitudinal data. This scoping review aimed to provide an overview of methods for collecting intensive time series data in mental health research as well as basic principles, current applications, target constructs, and statistical methods for this type of data.

In January 2021, the database MEDLINE was searched. Original articles were identified that (1) used active or passive data collection methods to gather intensive longitudinal data in daily life, (2) had a minimum sample size of N ⩾ 100 participants, and (3) included individuals with subclinical or clinical mental health problems.

In total, 3799 original articles were identified, of which 174 met inclusion criteria. The most widely used methods were diary techniques (e.g. Experience Sampling Methodology), various types of sensors (e.g. accelerometer), and app usage data. Target constructs included affect, various symptom domains, cognitive processes, sleep, dysfunctional behaviour, physical activity, and social media use. There was strong evidence on feasibility of, and high compliance with, active and passive data collection methods in diverse clinical settings and groups. Study designs, sampling schedules, and measures varied considerably across studies, limiting the generalisability of findings.

Gathering intensive longitudinal data has significant potential to advance mental health research. However, more methodological research is required to establish and meet critical quality standards in this rapidly evolving field. Advanced approaches such as digital phenotyping, ecological momentary interventions, and machine-learning methods will be required to efficiently use intensive longitudinal data and deliver personalised digital interventions and services for improving public mental health.

Type
Invited Review
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press

Introduction

Smartphones, sensors, and wearables may play an important role in advancing mental health research by actively or passively collecting fine-grained, multi-modal intensive longitudinal data. Active data acquisition methods include modern diary techniques, such as Experience Sampling Methodology (ESM; Csikszentmihalyi & Larson, Reference Csikszentmihalyi and Larson1987; Myin-Germeys et al., Reference Myin-Germeys, Kasanova, Vaessen, Vachon, Kirtley, Viechtbauer and Reininghaus2018) or synonymously Ecological Momentary Assessment (Shiffman, Stone, & Hufford, Reference Shiffman, Stone and Hufford2008). These methods are built on the premise that subjective experience and behaviour is situated in context and, hence, are geared towards capturing moment-to-moment variation in thoughts, feelings, and behaviours in relation to the real-world context in which they occur, i.e., in daily life, outside the research laboratory (Myin-Germeys et al., Reference Myin-Germeys, Kasanova, Vaessen, Vachon, Kirtley, Viechtbauer and Reininghaus2018), thereby, generating time-intensive longitudinal data with limited recall bias and high ecological validity. Continuous time-intensive data can also be collected passively by using dedicated, high-grade, and research-driven sensors providing objective measures of physical or physiological parameters in daily life. Passive intensive longitudinal data can be further acquired through built-in sensors of mobile devices such as smartphones and wearables (Boonstra et al., Reference Boonstra, Nicholas, Wong, Shaw, Townsend and Christensen2018). Smartphones allow for logging device usage data, application usage, and communication. These passive data collection methods come with reduced burden as they do not require active user input and allow for a high sampling frequency, enabling the detection of temporal variation in trajectories of target constructs on micro-timescales, which has been posited to provide the basis for identifying ‘digital phenotypes’ (Insel, Reference Insel2017, Reference Insel2018; Jain, Powers, Hawkins, & Brownstein, Reference Jain, Powers, Hawkins and Brownstein2015) that may be relevant to mental ill-health (Jain et al., Reference Jain, Powers, Hawkins and Brownstein2015).

Intensive longitudinal data can also be used to investigate important risk and protective factors, including candidate momentary mechanisms that may contribute to the development of mental disorders (Rauschenberg et al., Reference Rauschenberg, van Os, Cremers, Goedhart, Schieveld and Reininghaus2017, Reference Reininghaus, Gayer-Anderson, Valmaggia, Kempton, Calem, Onyejiaka and Morgan2016b; Reininghaus, Depp, & Myin-Germeys, Reference Reininghaus, Depp and Myin-Germeys2016a). Allowing for the analysis of temporal variation within and between individuals, intensive longitudinal data provide detailed insights into trajectories of experience and behaviour as they occur in daily life, including their interaction with contextual or socio-environmental factors. Thus, this type of data can further our understanding of and generate evidence on, the social environment and how it contributes to our mental health (Myin-Germeys et al., Reference Myin-Germeys, Oorschot, Collip, Lataster, Delespaul and van Os2009, Reference Myin-Germeys, Kasanova, Vaessen, Vachon, Kirtley, Viechtbauer and Reininghaus2018; Reininghaus, Reference Reininghaus2018).

Methods for collecting intensive time series data have a wide range of applications in mental health research, including digital monitoring, reporting, and feedback (Kramer et al., Reference Kramer, Simons, Hartmann, Menne-Lothmann, Viechtbauer, Peeters and Wichers2014; Rauschenberg et al., Reference Rauschenberg, Hirjak, Ganslandt, Schulte-Strathaus, Schick, Meyer-Lindenberg and Reininghaus2021a). The aim of the present scoping review is to provide an extensive overview of methods for collecting intensive longitudinal data in mental health research, including basic principles, current applications, target constructs, and statistical methods for this type of data.

Methods

In January 2021, a combined search was conducted in the MEDLINE database for terms related to (a) mental disorders and, more generally, psychopathological domains (e.g. anxiety, depression), and (b) assessment methods that allow for intensive time series data collection (e.g. ESM, sensor-based technologies) (see online Supplementary Material, Table S1 for the full list of search terms). Search strings were developed and tested using MeSH terms, Boolean operators, and text words to conduct a broad search and identify relevant articles. In sum, 3799 titles and abstracts were screened for inclusion by independent reviewers (AS, CR, JSS, MD, MRP, IP, LA, CA, NM) using EndNote (TE, Reference TE2013). The references were screened and categorised as ‘eligible’, ‘query’, and ‘not eligible’. Full texts of articles categorised as eligible or query were obtained, read, and assessed against the full list of inclusion criteria. Grey literature and manuscripts from preprint servers were excluded. The study selection process is displayed as a PRISMA flow diagram in Fig. 1.

Fig. 1. Study selection. Notes: MEDLINE was searched on 30.01.2021. Reasons for exclusion were not meeting inclusion criteria (e.g. studies investigating samples with neurological disorders). Fifteen studies were excluded as they did not meet the criteria for assessment frequency (i.e. less than 20 data points).

Selection criteria

Inclusion criteria

Studies were included if they met the following inclusion criteria: (1) published in a peer-reviewed journal; (2) written in English, Dutch, or German; (3) contained original findings examining active (i.e. diary) or passive (i.e. sensors, mobile sensing) methods for collecting intensive time series data in daily life (i.e. defined as ⩾20 assessments per person, with a maximum time interval of one week between two assessments); (4) individuals with a diagnosis of, or at-risk for, mental disorder (i.e. first degree relatives of service users with a mental disorder, individuals with psychometric risk or an at-risk mental state); (5) published between January 2007 and January 2021; and (6) included a sample of at least 100 participants.

Exclusion criteria

We excluded studies that (1) used qualitative methods, single case studies, and studies with less than 100 participants, reviews, non-peer-reviewed articles, manuscripts, dissertations, PhD theses, conference proceedings, and book chapters; (2) investigated individuals from the general population without any documented psychometric risk or mental health problem; (3) focused on health-related problems without meeting criteria for a full clinical diagnosis of mental disorder; (4) investigated mHealth interventions for mental health promotion or universal prevention; (5) exclusively focused on service users that suffer neurological disorders or other medical conditions.

Results

The search strategy yielded 3799 potential articles of interest. Following title and abstract screening, 572 full text articles were assessed for eligibility (see Fig. 1). Five studies from low- and middle-income countries were identified. They included less than 100 participants and, hence, are reported in online Supplementary Table S5. In total, 174 articles were included in the final qualitative synthesis.

Data extraction

In total, active data collection methods were used in more than half of the included studies (61%, see online Supplementary Table S2). Twenty-nine publications (17%) reported findings from dedicated sensors (see online Supplementary Table S3), and 8 studies (5%) from mobile sensing (see online Supplementary Table S4). In 30 studies (17%), a combination of active and passive methods for collecting intensive time series data was used (see online Supplementary Table S5).

Active data collection methods

The most commonly used active data collection method was the ESM (in 96 of 108 studies). Various sampling techniques were used in these studies, including event-contingent designs (e.g. Tasca et al., Reference Tasca, Illing, Balfour, Krysanski, Demidenko, Nowakowski and Bissada2009), time-contingent designs (e.g. Collip et al., Reference Collip, Wigman, van Os, Oorschot, Jacobs, Derom and Myin-Germeys2014) with (semi-) random or fixed sampling schedules, or hybrid designs (i.e. combining event- and time-contingent designs) (e.g. Smyth et al., Reference Smyth, Wonderlich, Sliwinski, Crosby, Engel, Mitchell and Calogero2009). In ESM studies, the sampling frequency ranged from three to ten assessments per day, whereas in the twelve telephone/ email studies included, the sampling frequency was between four times per day to once per week. The assessment period ranged between two days to two years (see online Supplementary Table S2). Notably, there was considerable heterogeneity in sampling designs and ESM measures.

Applications and target constructs

Next, we extracted the most common target constructs in the identified studies.

Most studies (i.e. 75 studies), used the ESM to capture self-reported positive and negative affect (e.g. Collip et al. Reference Collip, Nicolson, Lardinois, Lataster, van Os and Myin-Germeys2011a; Fitzsimmons-Craft et al. Reference Fitzsimmons-Craft, Accurso, Ciao, Crosby, Cao, Pisetsky and Wonderlich2015; Hartmann et al. Reference Hartmann, Wichers, Menne-Lothmann, Kramer, Viechtbauer, Peeters and Simons2015; Haynos et al. Reference Haynos, Crosby, Engel, Lavender, Wonderlich, Mitchell and Le Grange2015; Lavender et al. Reference Lavender, Utzinger, Crosby, Goldschmidt, Ellison, Wonderlich and Le Grange2016). To this end, e.g. items from the Positive and Negative Affect Schedule (PANAS; Watson, Clark, and Tellegen, Reference Watson, Clark and Tellegen1988) have been used. In addition, systematic variation in affective states over time (sometimes referred to as emotional instability) was frequently investigated (e.g. Johns et al. Reference Johns, Di, Merikangas, Cui, Swendsen and Zipunnikov2019, Solhan, Trull, Jahng, & Wood, Reference Solhan, Trull, Jahng and Wood2009; Wonderlich et al. Reference Wonderlich, Lavender, Wonderlich, Peterson, Crow, Engel and Crosby2015). As an alternative to assessing discrete emotions, ratings of valence and arousal have been used to capture affective states (Becker, Fischer, Crosby, Engel, & Wonderlich, Reference Becker, Fischer, Crosby, Engel and Wonderlich2018). Affective processes have been examined in at-risk samples, or samples of service users diagnosed with various mental disorders, such as depressive (Hartmann et al., Reference Hartmann, Wichers, Menne-Lothmann, Kramer, Viechtbauer, Peeters and Simons2015; Kordy et al., Reference Kordy, Wolf, Aulich, Burgy, Hegerl, Husing and Backenstrass2016; Simons et al., Reference Simons, Hartmann, Kramer, Menne-Lothmann, Hohn, van Bemmel and Wichers2015), bipolar (Tsanas et al., Reference Tsanas, Saunders, Bilderbeck, Palmius, Osipov, Clifford and De Vos2016), psychotic (Collip et al., Reference Collip, van Winkel, Peerbooms, Lataster, Thewissen, Lardinois and Myin-Germeys2011c; Lataster et al., Reference Lataster, van Os, de Haan, Thewissen, Bak, Lataster and Myin-Germeys2011; Oorschot et al., Reference Oorschot, Lataster, Thewissen, Lardinois, van Os, Delespaul and Myin-Germeys2012), eating (Berner et al., Reference Berner, Crosby, Cao, Engel, Lavender, Mitchell and Wonderlich2017; Fitzsimmons-Craft et al., Reference Fitzsimmons-Craft, Accurso, Ciao, Crosby, Cao, Pisetsky and Wonderlich2015; Haynos et al., Reference Haynos, Crosby, Engel, Lavender, Wonderlich, Mitchell and Le Grange2015), anxiety (Silk et al., Reference Silk, Tan, Ladouceur, Meller, Siegle, McMakin and Ryan2018), and personality (Andrewes, Hulbert, Cotton, Betts, & Chanen, Reference Andrewes, Hulbert, Cotton, Betts and Chanen2017; Chapman, Rosenthal, Dixon-Gordon, Turner, & Kuppens, Reference Chapman, Rosenthal, Dixon-Gordon, Turner and Kuppens2017) disorders (see supplementary tables).

ESM has also been applied to assess cognitive processes in daily life. In psychosis research, psychotic experiences [e.g. subclinical expressions of disordered thinking, paranoia, delusions (Collip et al., Reference Collip, Nicolson, Lardinois, Lataster, van Os and Myin-Germeys2011a; Collip et al., Reference Collip, van Winkel, Peerbooms, Lataster, Thewissen, Lardinois and Myin-Germeys2011c; Klippel et al., Reference Klippel, Myin-Germeys, Chavez-Baldini, Preacher, Kempton, Valmaggia and Reininghaus2017; Reininghaus et al., Reference Reininghaus, Gayer-Anderson, Valmaggia, Kempton, Calem, Onyejiaka and Morgan2016b; Reininghaus et al., Reference Reininghaus, Kempton, Valmaggia, Craig, Garety, Onyejiaka and Morgan2016c)], threat anticipation (Klippel et al., Reference Klippel, Myin-Germeys, Chavez-Baldini, Preacher, Kempton, Valmaggia and Reininghaus2017; Perez Arribas, Goodwin, Geddes, Lyons, & Saunders, Reference Perez Arribas, Goodwin, Geddes, Lyons and Saunders2018), and aberrant salience (Klippel et al., Reference Klippel, Myin-Germeys, Chavez-Baldini, Preacher, Kempton, Valmaggia and Reininghaus2017; Reininghaus et al., Reference Reininghaus, Kempton, Valmaggia, Craig, Garety, Onyejiaka and Morgan2016c) are important target constructs that have been captured using ESM. In eating disorder research, momentary assessments of social comparison processes and thoughts of compensatory behaviour added evidence to existing theories with high ecological validity (Leahey, Crowther, & Ciesla, Reference Leahey, Crowther and Ciesla2011). In addition, ESM has been used to capture worrying or rumination (Khazanov, Ruscio, & Swendsen, Reference Khazanov, Ruscio and Swendsen2019; Ruscio et al., Reference Ruscio, Gentes, Jones, Hallion, Coleman and Swendsen2015), and dissociation (Mason et al., Reference Mason, Lavender, Wonderlich, Steiger, Cao, Engel and Crosby2017). More recently, experimental experience sampling tasks have been developed to measure momentary cognition (Reininghaus et al., Reference Reininghaus, Oorschot, Moritz, Gayer-Anderson, Kempton, Valmaggia and Myin-Germeys2019). However, compared to affective processes, overall, cognitive processes have been less frequently studied using ESM in mental health research. This might be due to more frequent fluctuations in cognitive constructs and to difficulty accessing these processes using traditional ESM measures (Daniels et al., Reference Daniels, Bartels, Verhagen, Van Knippenberg, De Vugt and Delespaul2020).

In 31 studies, ESM has also been used to assess the occurrence of specific – often dysfunctional – behavioural patterns in daily life [e.g. self-harm (Muehlenkamp et al., Reference Muehlenkamp, Engel, Wadeson, Crosby, Wonderlich, Simonich and Mitchell2009)]. Momentary behaviour has either been assessed by event-contingent or time-contingent sampling schemes. Offering an appealing alternative to retrospective self-report measures, some studies assessed momentary behaviour, such as substance use (Fatseas, Serre, Swendsen, & Auriacombe, Reference Fatseas, Serre, Swendsen and Auriacombe2018; Serre, Fatseas, Denis, Swendsen, & Auriacombe, Reference Serre, Fatseas, Denis, Swendsen and Auriacombe2018), intoxication (Mackesy-Amiti & Donenberg, Reference Mackesy-Amiti and Donenberg2020; Pisetsky et al., Reference Pisetsky, Crosby, Cao, Fitzsimmons-Craft, Mitchell, Engel and Peterson2016), and mode of consumption (Mackesy-Amiti & Donenberg, Reference Mackesy-Amiti and Donenberg2020) in the daily lives of individuals with substance use disorders. Moreover, ESM has gained increasing attention in research focusing on dysfunctional behaviour in the spectrum of eating disorders to assess, for example, restrictive eating, binge eating, and purging (Fitzsimmons-Craft et al., Reference Fitzsimmons-Craft, Accurso, Ciao, Crosby, Cao, Pisetsky and Wonderlich2015; Lavender et al., Reference Lavender, Utzinger, Crosby, Goldschmidt, Ellison, Wonderlich and Le Grange2016; Schaefer et al., Reference Schaefer, Smith, Anderson, Cao, Crosby, Engel and Wonderlich2020; Smyth et al., Reference Smyth, Wonderlich, Sliwinski, Crosby, Engel, Mitchell and Calogero2009; Zunker et al., Reference Zunker, Peterson, Crosby, Cao, Engel, Mitchell and Wonderlich2011).

In the included studies, a strong emphasis was placed on contextual factors such as participants' current location (Mackesy-Amiti & Donenberg, Reference Mackesy-Amiti and Donenberg2020; Rintala, Wampers, Myin-Germeys, & Viechtbauer, Reference Rintala, Wampers, Myin-Germeys and Viechtbauer2019) and activities (Leendertse et al., Reference Leendertse, Myin-Germeys, Lataster, Simons, Oorschot and Lardinois2018; Oorschot et al., Reference Oorschot, Lataster, Thewissen, Lardinois, van Os, Delespaul and Myin-Germeys2012), but also social context [e.g. being alone or in company, interpersonal stressful events (Collip et al., Reference Collip, Oorschot, Thewissen, Van Os, Bentall and Myin-Germeys2011b; Morgan et al., Reference Morgan, Lee, Wright, Gilchrist, Forbes, McMakin and Silk2017; Tasca et al., Reference Tasca, Illing, Balfour, Krysanski, Demidenko, Nowakowski and Bissada2008)]. Moving beyond the assessment of context, appraisals of the context have gained increasing attention. For example, appraisals of the unpleasantness of events, activities, and social situations have been used to operationalise different types of stress (Collip et al., Reference Collip, Nicolson, Lardinois, Lataster, van Os and Myin-Germeys2011a; Klippel et al., Reference Klippel, Viechtbauer, Reininghaus, Wigman, van Borkulo and Wichers2018; Peerbooms et al., Reference Peerbooms, Rutten, Collip, Lardinois, Lataster, Thewissen and van Winkel2012; Reininghaus et al., Reference Reininghaus, Gayer-Anderson, Valmaggia, Kempton, Calem, Onyejiaka and Morgan2016b). In addition, specific processes such as social satisfaction (Collip et al., Reference Collip, Wigman, van Os, Oorschot, Jacobs, Derom and Myin-Germeys2014) or perceived rejection (Scott et al., Reference Scott, Wright, Beeney, Lazarus, Pilkonis and Stepp2017) have been examined using ESM.

Thus, overall, ESM research to date has commonly examined a combination of affective, cognitive, and behavioural processes taking into account contextual factors to further elucidate the role of candidate momentary mechanisms (as situated in context) in the development and maintenance of mental health problems and their subjective experience (Erwin, Dennis, Coughlin, Calhoun, & Beckham, Reference Erwin, Dennis, Coughlin, Calhoun and Beckham2019; Khazanov et al., Reference Khazanov, Ruscio and Swendsen2019; Oorschot et al., Reference Oorschot, Lataster, Thewissen, Lardinois, van Os, Delespaul and Myin-Germeys2012). For example, the construct of affective and psychotic stress reactivity, defined as an increased intensity of negative affect and psychotic experiences in response to minor daily stressors, has been widely studied as an important putative momentary mechanism in psychosis research (Collip et al., Reference Collip, van Winkel, Peerbooms, Lataster, Thewissen, Lardinois and Myin-Germeys2011c; Frissen et al., Reference Frissen, Lieverse, Drukker, Delespaul, Lataster, Myin-Germeys and van Os2014; Reininghaus et al., Reference Reininghaus, Kempton, Valmaggia, Craig, Garety, Onyejiaka and Morgan2016c) and in other fields [e.g. eating disorders (Pearson et al., Reference Pearson, Lavender, Cao, Wonderlich, Crosby, Engel and Crow2017), personality disorders (Glaser, Van Os, Mengelers, & Myin-Germeys, Reference Glaser, Van Os, Mengelers and Myin-Germeys2008)], and with respect to transdiagnostic phenotypes (Rauschenberg et al., Reference Rauschenberg, van Os, Cremers, Goedhart, Schieveld and Reininghaus2017)). Investigating the interplay of affective processes and contexts in daily life offers the advantage of operationalising important symptom domains and their subjective experience, as has been done, for example, for negative symptoms such as anhedonia (Oorschot et al. (Reference Oorschot, Lataster, Thewissen, Lardinois, van Os, Delespaul and Myin-Germeys2012). Insights on the momentary dynamics of affective experiences and dysfunctional behaviour have advanced our understanding of the emergence of disordered eating behaviour such as restrictive or binge eating episodes and purging (Berg et al., Reference Berg, Cao, Crosby, Engel, Peterson, Crow and Wonderlich2017; Engel et al., Reference Engel, Wonderlich, Crosby, Mitchell, Crow, Peterson and Gordon2013; Fitzsimmons-Craft et al., Reference Fitzsimmons-Craft, Accurso, Ciao, Crosby, Cao, Pisetsky and Wonderlich2015; Goldschmidt et al., Reference Goldschmidt, Wonderlich, Crosby, Engel, Lavender, Peterson and Mitchell2014; Haynos et al., Reference Haynos, Crosby, Engel, Lavender, Wonderlich, Mitchell and Le Grange2015; Schaefer et al., Reference Schaefer, Smith, Anderson, Cao, Crosby, Engel and Wonderlich2020). ESM measures of temporal dynamics in affect, craving, and substance use behaviour in daily life can now be used to inform treatment and relapse prevention in substance use disorders (Fatseas et al., Reference Fatseas, Serre, Swendsen and Auriacombe2018; Serre et al., Reference Serre, Fatseas, Denis, Swendsen and Auriacombe2018). An overview on psychometric quality of ESM measures is provided in the supplementary material.

Passive, sensor-based data collection methods

Overall, accelerometers were the most frequently used dedicated sensors to capture time intensive data in the included studies (i.e. in 27 of 29 studies, see online Supplementary Table S3). Only two studies reported pedometer data, i.e., estimating individuals' physical activity based on step count. Other studies using sensors capable of detecting other psychophysiological parameters, such as heart rate or electrodermal activity were not included. The assessment period in the included studies utilising dedicated sensors ranged from two hours to 22 days, and the sampling frequency was between two to 60 s epochs (see online Supplementary Table S3).

Applications and target constructs

Most studies (i.e. 18 out of 29, see online Supplementary Table S3) used sensors to passively monitor physical activity. In the included studies, daytime physical activity was operationalised as a gradient of intensity that includes sleep, sedentary behaviour, light physical activity, moderate to vigorous physical activity, and high intensity physical activity. In addition, the included studies reported frequency and duration of specific types of physical activity, such as sedentary behaviour assessed by a pedometer (Piette et al., Reference Piette, Valenstein, Himle, Duffy, Torres, Vogel and Richardson2011) or accelerometer (Stubbs, Ku, Chung, & Chen, Reference Stubbs, Ku, Chung and Chen2017), as well as gross motor activity (Difrancesco et al., Reference Difrancesco, Lamers, Riese, Merikangas, Beekman, van Hemert and Penninx2019). Further, some of the included studies quantified physical activity on a daily basis using step count (Baerg et al., Reference Baerg, Cairney, Hay, Rempel, Mahlberg and Faught2011), or by calculating a mean activity score from accelerometer data (Benard et al., Reference Benard, Etain, Vaiva, Boudebesse, Yeim, Benizri and Geoffroy2019). Others (Geoffroy et al., Reference Geoffroy, Micoulaud Franchi, Maruani, Philip, Boudebesse, Benizri and Etain2019) reported the average activity during the most active 10-h period.

Another construct that has been extensively investigated using sensors is sleep (11 out of 29 studies, see online Supplementary Table S3). There are different operationalisations for sleep duration. As shown in online Supplementary Table S3, different parameters have been applied to measure sleep quality and duration including e.g., sleep efficiency and start and end time of the rest period (e.g. Blake et al. Reference Blake, Snoep, Raniti, Schwartz, Waloszek, Simmons and Allen2017, Reference Blake, Blake, Schwartz, Raniti, Waloszek, Murray and Allen2018; Fang et al. Reference Fang, Suzuki, Lim, Chung, Ku and Chen2016; Goodlin-Jones, Waters, & Anders, Reference Goodlin-Jones, Waters and Anders2009; McCrae et al. Reference McCrae, Williams, Roditi, Anderson, Mundt, Miller and Robinson2019; Owens et al. Reference Owens, Sangal, Sutton, Bakken, Allen and Kelsey2009; Robillard et al. Reference Robillard, Naismith, Smith, Rogers, White, Terpening and Hickie2014; Verkooijen et al. Reference Verkooijen, Stevelink, Abramovic, Vinkers, Ophoff, Kahn and van Haren2017; Wallace et al. Reference Wallace, McMakin, Tan, Rosen, Forbes, Ladouceur and Silk2017; Wallen, Park, Krumlauf, & Brooks, Reference Wallen, Park, Krumlauf and Brooks2019; Wichniak et al. Reference Wichniak, Skowerska, Chojnacka-Wojtowicz, Taflinski, Wierzbicka, Jernajczyk and Jarema2011). Among the included studies, several measures intended to assess sleep disruption, including the fragmentation index, which quantifies sleep continuity (Benard et al., Reference Benard, Etain, Vaiva, Boudebesse, Yeim, Benizri and Geoffroy2019; Geoffroy et al., Reference Geoffroy, Micoulaud Franchi, Maruani, Philip, Boudebesse, Benizri and Etain2019). Further, two parameters were used to describe the transition period between sleep and wakefulness: sleep onset latency, which is the time required to fall asleep after going to bed (e.g. Bergwerff, Luman, & Oosterlaan, Reference Bergwerff, Luman and Oosterlaan2016; Blake et al. Reference Blake, Blake, Schwartz, Raniti, Waloszek, Murray and Allen2018), and sleep inertia, which is the time spent awake between sleep offset and getting out of bed [e.g. (Verkooijen et al., Reference Verkooijen, Stevelink, Abramovic, Vinkers, Ophoff, Kahn and van Haren2017)]. As demonstrated by the included studies, sensor data can also be informative to determine individuals' day-night rhythm or circadian patterns, when physical activity and sleep data are combined. The inter-daily variability (e.g. Benard et al. Reference Benard, Etain, Vaiva, Boudebesse, Yeim, Benizri and Geoffroy2019; Geoffroy et al. Reference Geoffroy, Micoulaud Franchi, Maruani, Philip, Boudebesse, Benizri and Etain2019; Shou et al. Reference Shou, Cui, Hickie, Lameira, Lamers, Zhang and Merikangas2017 is one measure to quantify consistency in sleep-wake pattern across days, whereas intra-daily variability (e.g. Geoffroy et al., Reference Geoffroy, Micoulaud Franchi, Maruani, Philip, Boudebesse, Benizri and Etain2019) represents an indicator for rhythm fragmentation, which relates to daytime napping or night time activity. Difrancesco et al. (Reference Difrancesco, Lamers, Riese, Merikangas, Beekman, van Hemert and Penninx2019) developed an index for circadian rhythm, also known as chronotype, or the proclivity to be asleep at a particular time of the day.

Mobile sensing

Only three included studies applied mobile sensing (see online Supplementary Table S4) and used log data (i.e. ingoing/outgoing calls and text messages), mobility measures (GPS, cell tower IDs, e.g. Friedmann et al., Reference Friedmann, Santangelo, Ebner-Priemer, Hill, Neubauer, Rausch and Priebe2020) to investigate mental health outcomes. For example, Pratap et al. (Reference Pratap, Atkins, Renn, Tanana, Mooney, Anguera and Arean2019) made use of machine learning to predict prospective group and person-level daily mood via passive smartphone data. Using GPS to capture mobility was reported as one of the most encouraging and important features in the study sample.

Five included studies investigated smartphone usage data and mainly focused on linguistic characteristics of social media usage (Birnbaum, Ernala, Rizvi, De Choudhury, & Kane, Reference Birnbaum, Ernala, Rizvi, De Choudhury and Kane2017; Cheng, Li, Kwok, Zhu, & Yip, Reference Cheng, Li, Kwok, Zhu and Yip2017; Hswen, Naslund, Brownstein, & Hawkins, Reference Hswen, Naslund, Brownstein and Hawkins2018; Hswen et al., Reference Hswen, Naslund, Chandrashekar, Siegel, Brownstein and Hawkins2017; Reece et al., Reference Reece, Reagan, Lix, Dodds, Danforth and Langer2017). The investigation of communication patterns on popular social media outlets has been used for (1) predicting the emergence of poor mental health (Eichstaedt et al., Reference Eichstaedt, Smith, Merchant, Ungar, Crutchley, Preotiuc-Pietro and Schwartz2018; Pratap et al., Reference Pratap, Atkins, Renn, Tanana, Mooney, Anguera and Arean2019; Reece et al., Reference Reece, Reagan, Lix, Dodds, Danforth and Langer2017), (2) supporting early detection and intervention (Cheng et al., Reference Cheng, Li, Kwok, Zhu and Yip2017; Hswen et al., Reference Hswen, Naslund, Brownstein and Hawkins2018), (3) identifying individuals at-risk for, or with a diagnosis of, mental disorders (Birnbaum et al., Reference Birnbaum, Ernala, Rizvi, De Choudhury and Kane2017; Hswen et al., Reference Hswen, Naslund, Chandrashekar, Siegel, Brownstein and Hawkins2017), and (4) to identify important social-environmental risk and resilience factors (Birnbaum et al., Reference Birnbaum, Ernala, Rizvi, De Choudhury and Kane2017; Friedmann et al., Reference Friedmann, Santangelo, Ebner-Priemer, Hill, Neubauer, Rausch and Priebe2020; Hswen et al., Reference Hswen, Naslund, Chandrashekar, Siegel, Brownstein and Hawkins2017).

Active and passive data acquisition methods combined

The findings of our review further indicated that, to date, it is primarily sleep research that has pioneered the joint use of active and passive data acquisition methods in mental health research (i.e. 26 out of 30 included studies, see online Supplementary Table S5). The validation of measures can be accomplished by combining sensor data with self-report data e.g. on sleep (e.g. Lovato, Lack, Wright, & Kennaway, Reference Lovato, Lack, Wright and Kennaway2014; McMakin et al., Reference McMakin, Ricketts, Forbes, Silk, Ladouceur, Siegle and Dahl2019 or on other constructs such as pain (McCrae et al., Reference McCrae, Williams, Roditi, Anderson, Mundt, Miller and Robinson2019), affect (Merikangas et al., Reference Merikangas, Swendsen, Hickie, Cui, Shou, Merikangas and Zipunnikov2019; Wallace et al., Reference Wallace, McMakin, Tan, Rosen, Forbes, Ladouceur and Silk2017), or stress (Wallace et al., Reference Wallace, McMakin, Tan, Rosen, Forbes, Ladouceur and Silk2017).

Analysis

Intensive longitudinal data typically has a multilevel structure, with repeated measurements nested within individuals. Therefore, associations among the constructs of interest can be examined on at least two levels. Analyses at the cluster level (i.e. individuals or groups) reveal information on between-person differences in individuals' average responses (e.g. those who experience more stress in their daily life are, on average, more likely to report psychotic experiences (e.g. Glaser, Van Os, Thewissen, & Myin-Germeys, Reference Glaser, Van Os, Thewissen and Myin-Germeys2010; Reininghaus et al., Reference Reininghaus, Gayer-Anderson, Valmaggia, Kempton, Calem, Onyejiaka and Morgan2016b, Reference Reininghaus, Kempton, Valmaggia, Craig, Garety, Onyejiaka and Morgan2016c). Analyses at the within-person level account for potential variability in individuals' experience and behaviour over time (i.e. from one measurement occasion to another). These analyses therefore allow for investigating temporal trajectories and uncovering event-related or context-dependent relations among the constructs under scrutiny (e.g. whether an individual has a high risk for binge eating when experiencing high levels of negative affect (e.g. Berg et al., Reference Berg, Cao, Crosby, Engel, Peterson, Crow and Wonderlich2017; Crosby et al., Reference Crosby, Wonderlich, Engel, Simonich, Smyth and Mitchell2009; Selby et al., Reference Selby, Doyle, Crosby, Wonderlich, Engel, Mitchell and Le Grange2012).

To date, ESM in the field of mental health research has primarily reported findings based on the analyses of between-person differences i.e., aggregating ratings on target constructs across measurement occasions (e.g. Blum et al., Reference Blum, Vakhrusheva, Saperstein, Khan, Chang, Hansen and Kimhy2015; Engel et al., Reference Engel, Wonderlich, Crosby, Mitchell, Crow, Peterson and Gordon2013, Kimhy et al., Reference Kimhy, Vakhrusheva, Khan, Chang, Hansen, Ballon and Gross2014; Kuepper et al., Reference Kuepper, Oorschot, Myin-Germeys, Smits, van Os and Henquet2013; Muehlenkamp et al., Reference Muehlenkamp, Engel, Wadeson, Crosby, Wonderlich, Simonich and Mitchell2009; Pearson et al., Reference Pearson, Pisetsky, Goldschmidt, Lavender, Wonderlich, Crosby and Peterson2016; Pisetsky et al., Reference Pisetsky, Crosby, Cao, Fitzsimmons-Craft, Mitchell, Engel and Peterson2016). Most of the included studies conducted these types of analyses to examine the effectiveness of an intervention (e.g. comparing treatment v. control conditions (e.g. Chapman et al., Reference Chapman, Rosenthal, Dixon-Gordon, Turner and Kuppens2017; Kordy et al., Reference Kordy, Wolf, Aulich, Burgy, Hegerl, Husing and Backenstrass2016; Schlam, Baker, Smith, Cook, & Piper, Reference Schlam, Baker, Smith, Cook and Piper2020; Silk et al., Reference Silk, Tan, Ladouceur, Meller, Siegle, McMakin and Ryan2018; Simons et al., Reference Simons, Hartmann, Kramer, Menne-Lothmann, Hohn, van Bemmel and Wichers2015), or to examine differences in target constructs (e.g. the experience of stress, or negative affect) across service users and healthy controls (e.g. Blum et al., Reference Blum, Vakhrusheva, Saperstein, Khan, Chang, Hansen and Kimhy2015; Goldschmidt et al., Reference Goldschmidt, Peterson, Wonderlich, Crosby, Engel, Mitchell and Berg2013; Johns et al., Reference Johns, Di, Merikangas, Cui, Swendsen and Zipunnikov2019; Leraas et al., Reference Leraas, Smith, Utzinger, Cao, Engel, Crosby and Wonderlich2018; Morgan et al., Reference Morgan, Lee, Wright, Gilchrist, Forbes, McMakin and Silk2017; Oorschot et al., Reference Oorschot, Lataster, Thewissen, Lardinois, van Os, Delespaul and Myin-Germeys2012; Reininghaus et al., Reference Reininghaus, Gayer-Anderson, Valmaggia, Kempton, Calem, Onyejiaka and Morgan2016b; Tsanas et al., Reference Tsanas, Saunders, Bilderbeck, Palmius, Osipov, Clifford and De Vos2016). However, the full benefit of analysing intensive longitudinal data collected using ESM, arguably, comes into play when also considering temporal fluctuations in the relationship between an independent variable [e.g. affective experience (Anestis et al., Reference Anestis, Selby, Crosby, Wonderlich, Engel and Joiner2010; Berner et al., Reference Berner, Crosby, Cao, Engel, Lavender, Mitchell and Wonderlich2017; Karr et al., Reference Karr, Crosby, Cao, Engel, Mitchell, Simonich and Wonderlich2013)] and some outcome of interest [e.g. maladaptive behaviour (Anestis et al., Reference Anestis, Selby, Crosby, Wonderlich, Engel and Joiner2010; Berner et al., Reference Berner, Crosby, Cao, Engel, Lavender, Mitchell and Wonderlich2017; Karr et al., Reference Karr, Crosby, Cao, Engel, Mitchell, Simonich and Wonderlich2013; Ruscio et al., Reference Ruscio, Gentes, Jones, Hallion, Coleman and Swendsen2015)] that unfold at the within-person level. This approach also provides a means of identifying processes and situations that precede a critical event [e.g. incidents of self-injury (Muehlenkamp et al., Reference Muehlenkamp, Engel, Wadeson, Crosby, Wonderlich, Simonich and Mitchell2009), dietary restrictions (Engel et al., Reference Engel, Wonderlich, Crosby, Mitchell, Crow, Peterson and Gordon2013), aggressive urges or behaviour (Scott et al., Reference Scott, Wright, Beeney, Lazarus, Pilkonis and Stepp2017)]. Multi-level mixed-effect models further allow for the inclusion of random effects to account for person- and day-level differences, for example, in the association between negative affect and aggressive urges by modelling random intercepts and slopes. In this way, it can be shown that there are between-person differences in complex within-person associations. For example, it has been reported (Scott et al., Reference Scott, Wright, Beeney, Lazarus, Pilkonis and Stepp2017) that an increase in perceived rejection was associated with an increase in the experience of negative affect (i.e. within-person association). This association was stronger for individuals with more pronounced borderline personality symptoms (i.e. between-person difference). Finally, examining time-lagged associations between independent variables and outcomes provide insights into the development of these associations over time. Despite this advantage, only a minority of the included studies used time-lagged analyses (24 studies, e.g. Jahng et al., Reference Jahng, Solhan, Tomko, Wood, Piasecki and Trull2011; Klippel et al., Reference Klippel, Schick, Myin-Germeys, Rauschenberg, Vaessen and Reininghaus2021; Wigman et al., Reference Wigman, van Os, Borsboom, Wardenaar, Epskamp, Klippel and Wichers2015). Gerritsen et al. (Reference Gerritsen, Bagby, Sanches, Kiang, Maheandiran, Prce and Mizrahi2019), for instance, showed that high levels of activity-related stress experienced at time t n−1 predicted increases in anhedonia at time t n. Another study revealed that post-traumatic stress disorder symptom severity at time t n was not predicted by the experience of negative affect at time t n−1, but conversely, that symptom severity at time t n−1 predicted the experience of negative affect at time t n (Erwin et al., Reference Erwin, Dennis, Coughlin, Calhoun and Beckham2019). More recently, Klippel et al. (Reference Klippel, Schick, Myin-Germeys, Rauschenberg, Vaessen and Reininghaus2021) applied cross-lagged moderated multilevel mediation analyses in order to systematically test the temporal association between momentary stress, negative affect, and psychotic experiences.

Common approaches to analyse sensor data also include multi-level modelling. However, in most studies, parameters are aggregated prior to analysis by, for example, calculating the mean score for approximating individuals' physical activity from step counts collected on several consecutive days (Benard et al., Reference Benard, Etain, Vaiva, Boudebesse, Yeim, Benizri and Geoffroy2019). There is also a recent move towards utilising more complex methodological approaches, including supervised machine learning algorithms (e.g. Wallen et al., Reference Wallen, Park, Krumlauf and Brooks2019, Zebin, Peek, & Casson, Reference Zebin, Peek and Casson2019). In particular, in long time series derived from multiple sources (e.g. several sensors) machine learning approaches using prediction models including Bayesian networks and recurrent neural networks may be applied (Koppe, Guloksuz, Reininghaus, & Durstewitz, Reference Koppe, Guloksuz, Reininghaus and Durstewitz2019) These novel approaches are also increasingly being used for classifying individuals, e.g., into individuals with mental health problems and controls, based on mobile sensing data (Birnbaum et al., Reference Birnbaum, Ernala, Rizvi, De Choudhury and Kane2017). To this end, different algorithms have been applied including Support Vector Machines, Bayesian classifiers, random forest, and other decision trees.

Discussion

The aim of this scoping review was to provide a comprehensive overview of methods used for gathering time series data in mental health research. We identified a broad range of methods, comprising self-report and various passive, sensor-based technologies. These methods have been utilised in diverse populations and settings across the full spectrum of mental ill-health. Compliance with, active and passive data collection methods in diverse clinical settings and groups was high. Most frequently studied target constructs included positive and negative affect, symptom domains, cognitive processes, sleep, and dysfunctional behaviour, as well as physical activity and social media use. Overall, our findings indicate that the included studies were highly heterogeneous in terms of design, sampling schemes, and operationalisation of target constructs – even when largely comparable constructs (e.g. negative affect) were studied. Furthermore, our review highlights that, so far, the full potential of the data captured by these methods has not been fully exploited, as often only aggregated data were analysed. The reported relationships were largely correlational in nature and only a small number of studies used more advanced statistical methods to investigate, for instance, temporality or other criteria for establishing causality. In addition, only a minority of studies applied a combination of methods.

Methodological considerations

The current review and its findings must be viewed in light of some limitations. First, the overarching aim of the review was to provide a comprehensive overview of the various methods currently used to collect intensive longitudinal data in mental health research. However, the definition used for intensive longitudinal data may differ from field to field. In the present work, we included only studies with more than 20 assessments per person, with a maximum time interval of one week between two assessments. Although an arbitrary cut-off, this criterion aimed to exclude studies with longitudinal designs such as longitudinal cohort designs, in which data are collected over time periods of several years, and, hence, do not reflect a design for collecting intensive longitudinal data. Further, given tens of thousands of studies published on this subject, only a restricted time period, in which studies were published, was considered (i.e. January 2007 and January 2021). We focused on those with large sample size (i.e. equal to or more than 100 individuals). Thus, important studies published before 2007 or with small sample size or studies that used cost-intensive sensors (e.g. high-grade heart-rate sensors) may have failed to identify.

Second, we did not perform hand-searching and scanning of reference lists of the included articles. Also, the results were not subjected to a second independent review. While this may have led to selection bias, it is in line with recommendations for conceptual and methodological reviews of a vast and disparate literature (Lilford et al., Reference Lilford, Richardson, Stevens, Fitzpatrick, Edwards, Rock and Hutton2001; Morgan, Burns, Fitzpatrick, Pinfold, & Priebe, Reference Morgan, Burns, Fitzpatrick, Pinfold and Priebe2007; Reininghaus & Priebe, Reference Reininghaus and Priebe2012).

Third, the synthesis of evidence, for example, on psychometric properties of ESM measures, was hampered by use of inappropriate psychometric methods (e.g. principal component analysis for multilevel data). This reflects a limitation of the conclusion that can be drawn about the psychometric quality of ESM measures based on our review. Overall, only a relatively small number of studies investigated some psychometric domains suggested by the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) initiative (Mokkink et al., Reference Mokkink, Terwee, Patrick, Alonso, Stratford, Knol and de Vet2010), e.g., responsiveness, interpretability and test-retest reliability were not investigated at all.

Fourth, we identified only few original articles from low- and middle-income countries despite applying more liberal eligibility criteria with regard to sample size for studies from these countries (see online Supplementary Material Table S5). This may imply that ESM studies may be less feasible or have been conducted on a smaller scale in large parts of the world, limiting the generalisability of reported findings. This may indicate the need of technology transfer or open software facilitating its application as digital monitoring and interventions may present an opportunity for global health settings by facilitating remote access to mental health services, for example, for difficult-to-reach populations (Naslund et al., Reference Naslund, Aschbrenner, Araya, Marsch, Unutzer, Patel and Bartels2017; Rauschenberg et al., Reference Rauschenberg, Schick, Hirjak, Seidler, Paetzold, Apfelbacher and Reininghaus2021b). Future research may benefit from the use of widely available consumer rather than dedicated research devices, and facilitated by country-specific implementation strategies. Practical steps may include engagement of multiple stakeholders in user-centred designs and transdisciplinary research, including mental health practitioners, service users, digital industry, and interdisciplinary research teams.

Finally, the constructs and methods that were reported in the included studies were heterogeneous – which may further limit the generalisability of reported findings. On the one hand, this may be a result of insufficient reporting and less of an issue in future studies when recently published reporting guidelines will hopefully be followed more closely (e.g. Trull & Ebner-Priemer, Reference Trull and Ebner-Priemer2020). On the other hand, this may in part be imminent to a rapidly growing field of research. However, with the advent of open science practices, studies in this field may be more commonly documented in a transparent and openly accessible way, as it has been common practice in other fields (e.g. randomised controlled trials) for a long time. This in turn, may provide the basis for direct replications, which are urgently needed in this rapidly evolving and methodologically diverse field. Item repositories (Hall, Scherner, Kreidel, & Rubel, Reference Hall, Scherner, Kreidel and Rubel2021; Kirtley, Lafit, Achterhof, Hiekkaranta, & Myin-Germeys, Reference Kirtley, Lafit, Achterhof, Hiekkaranta and Myin-Germeys2021) may aid in the organisation, validation, and utilisation of ESM items. In the long run, open science practices may also facilitate collaboration, which may foster the use of more comparable methods (e.g. items, sampling frequencies, devices). The research community and scientific associations should work towards defining standards and reach agreement, particularly in the rapidly growing field of mobile sensing. Additional research on measurement quality and further optimisations are required to fully exploit the advancements in methods for gathering longitudinal intensive data.

Future outlook

To date, the evidence on clinical benefits of ESM and sensor methods remains very limited. Digital monitoring may increase individual's awareness about symptoms and their interaction with the environment. As time series data allow for investigating within-person variation, patterns of associations may be revealed and personalised feedback provided based on ESM monitoring data (Rauschenberg et al., Reference Rauschenberg, Hirjak, Ganslandt, Schulte-Strathaus, Schick, Meyer-Lindenberg and Reininghaus2021a). This, in turn, may empower service users to actively participate in clinical decision-making, which is an important feature of standard health care (National Institute for Health and Care Excellence, 2021). While there is some evidence on the efficacy of ESM-derived feedback in the treatment of depression (Kramer et al., Reference Kramer, Simons, Hartmann, Menne-Lothmann, Viechtbauer, Peeters and Wichers2014), further well-designed and adequately powered RCTs are needed to examine benefits for service users.

Furthermore, ESM and sensor data have been used to trigger digital interventions known as Ecological Momentary Interventions (EMIs; Heron & Smyth, Reference Heron and Smyth2010; Myin-Germeys, Birchwood, & Kwapil, Reference Myin-Germeys, Birchwood and Kwapil2011; Myin-Germeys, Klippel, Steinhart, & Reininghaus, Reference Myin-Germeys, Klippel, Steinhart and Reininghaus2016; Reininghaus, Reference Reininghaus2018). Thereby EMIs are adaptive, and can be personalised based on the dynamics of individuals' experience and behaviour (Heron & Smyth, Reference Heron and Smyth2010; Myin-Germeys et al., Reference Myin-Germeys, Klippel, Steinhart and Reininghaus2016, Reference Myin-Germeys, Kasanova, Vaessen, Vachon, Kirtley, Viechtbauer and Reininghaus2018; Reininghaus, Reference Reininghaus2018; Reininghaus et al., Reference Reininghaus, Depp and Myin-Germeys2016a). This also allows for testing ecological interventionist causal models (Reininghaus et al., Reference Reininghaus, Depp and Myin-Germeys2016a) by examining whether targeting candidate mechanisms in daily life result in lasting changes in mental health outcomes. Remote monitoring and digital interventions recently received increasing attention as tools for tracking and mitigating the negative impact of the COVID-19 pandemic (Rauschenberg et al., Reference Rauschenberg, Schick, Hirjak, Seidler, Paetzold, Apfelbacher and Reininghaus2021b). Intensive time series data – passive data collection methods in particular – may be used to monitor system- or population-level mental health or to inform more targeted programs of mental health promotion. However, as there may be a potential of scaling-up the application of ESM and sensor methods in clinical care, technical problems and adverse device effects need to be minimised, as also reflected in regulatory requirements such as those set out by the EU Medical Device Regulation.

Another aspect that has not yet come to bear, is the combination of various types of intensive time series data that may help advance our understanding of critical determinants, developmental candidate mechanisms, and the persistence of mental health problems. The combination of ESM with sensor-based assessments may enable a deeper understanding of context specific influences. Furthermore, mobile sensing and digital phenotyping may have the potential to advance mental health research, particularly when passive data is collected concurrently with self-report data (Myin-Germeys et al., Reference Myin-Germeys, Kasanova, Vaessen, Vachon, Kirtley, Viechtbauer and Reininghaus2018; Trull & Ebner-Priemer, Reference Trull and Ebner-Priemer2014). However, this also bears privacy risks and users need to be adequately informed and educated about the applied privacy settings. These methods may therefore empower users also with respect to data and digital health literacy when applied according to current regulations. Careful attention needs to be paid to data safety and privacy issues and users need to be adequately informed about privacy settings of sensor methods. It is notable that only very few included studies have taken advantage of the potential for combining active and passive methods for collecting intensive time series data. This is true even though it opens up new avenues for more context-sensitive sampling strategies that link experience to specific events or behavioural patterns, such as GPS-triggered ESM reports (Tost et al., Reference Tost, Reichert, Braun, Reinhard, Peters, Lautenbach and Meyer-Lindenberg2019). However, the added value of combining active and passive data collection methods must be demonstrated in future studies.

Conclusion

While technological advancements have significantly increased the opportunities for collecting intensive time series data in mental health research, the field continues to face critical challenges in the years to come. This includes current reporting practices, the use of insufficient statistical approaches to fully exploit the potential of multimodal longitudinal data, and establishing best practices for studies that purposefully combine various modes of data collection. Open science practices have the potential to increase transparency, generalisability, and reproducibility in this rapidly evolving field. Further, the field requires a consensus on the operationalisation of constructs and robust evidence on the psychometric quality of existing measures are critical next steps. The use of ESM and other intensive longitudinal data collections methods have enormous potential for digital monitoring and personalised feedback on service users' experience and behaviour that can be used meaningfully by service users and clinicians. This may include empowering individuals with mental health conditions to more effectively manage their mental and physical health, as well as informing and extending face-to-face sessions to real-world situations and more personalised treatment based on adaptive, ecological momentary interventions. How the research community will address these opportunities and challenges will determine whether the digital transformation of public mental health provision results in tangible benefits for users, carers, and practitioners.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/S0033291722003336

Acknowledgements

This work is funded by a DFG project grant (no. 389626655) and DFG Heisenberg professorship (no. 389624707) to UR. We further want to thank the student assistants Nina Mueller and Carina Achtnich for helping with screening the literature and extracting the data.

Author contributions

UR, CR, AS, JSS, MRP, MD, IP, LA and LW conceptualised, wrote and revised this manuscript. All authors approved the final version.

Financial support

This work is funded by a DFG project grant (no. 389626655) and DFG Heisenberg professorship (no. 389624707) to UR.

Conflict of interest

None.

References

Andrewes, H. E., Hulbert, C., Cotton, S. M., Betts, J., & Chanen, A. M. (2017). An ecological momentary assessment investigation of complex and conflicting emotions in youth with borderline personality disorder. Psychiatry Research, 252, 102110. doi: 10.1016/j.psychres.2017.01.100CrossRefGoogle ScholarPubMed
Anestis, M. D., Selby, E. A., Crosby, R. D., Wonderlich, S. A., Engel, S. G., & Joiner, T. E. (2010). A comparison of retrospective self-report versus ecological momentary assessment measures of affective lability in the examination of its relationship with bulimic symptomatology. Behavior Research and Therapy, 48, 607613. doi: 10.1016/j.brat.2010.03.012CrossRefGoogle ScholarPubMed
Baerg, S., Cairney, J., Hay, J., Rempel, L., Mahlberg, N., & Faught, B. E. (2011). Evaluating physical activity using accelerometry in children at risk of developmental coordination disorder in the presence of attention deficit hyperactivity disorder. Research in Developmental Disabilities, 32, 13431350. doi: 10.1016/j.ridd.2011.02.009CrossRefGoogle ScholarPubMed
Becker, K. R., Fischer, S., Crosby, R. D., Engel, S. G., & Wonderlich, S. A. (2018). Dimensional analysis of emotion trajectories before and after disordered eating behaviors in a sample of women with bulimia nervosa. Psychiatry Research, 268, 490500. doi: 10.1016/j.psychres.2018.08.008CrossRefGoogle Scholar
Benard, V., Etain, B., Vaiva, G., Boudebesse, C., Yeim, S., Benizri, C., … Geoffroy, P. A. (2019). Sleep and circadian rhythms as possible trait markers of suicide attempt in bipolar disorders: An actigraphy study. Journal of Affective Disorders, 244, 18. doi: 10.1016/j.jad.2018.09.054CrossRefGoogle ScholarPubMed
Berg, K. C., Cao, L., Crosby, R. D., Engel, S. G., Peterson, C. B., Crow, S. J., … Wonderlich, S. A. (2017). Negative affect and binge eating: Reconciling differences between two analytic approaches in ecological momentary assessment research. International Journal of Eating Disorders, 50, 12221230. doi: 10.1002/eat.22770CrossRefGoogle ScholarPubMed
Bergwerff, C. E., Luman, M., & Oosterlaan, J. (2016). No objectively measured sleep disturbances in children with attention-deficit/hyperactivity disorder. Journal of Sleep Research, 25, 534540. doi: 10.1111/jsr.12399CrossRefGoogle ScholarPubMed
Berner, L. A., Crosby, R. D., Cao, L., Engel, S. G., Lavender, J. M., Mitchell, J. E., & Wonderlich, S. A. (2017). Temporal associations between affective instability and dysregulated eating behavior in bulimia nervosa. Journal of Psychiatric Research, 92, 183190. doi: 10.1016/j.jpsychires.2017.04.009CrossRefGoogle ScholarPubMed
Birnbaum, M. L., Ernala, S. K., Rizvi, A. F., De Choudhury, M., & Kane, J. M. (2017). A collaborative approach to identifying social media markers of schizophrenia by employing machine learning and clinical appraisals. Journal of Medical Internet Research, 19, e289. doi: 10.2196/jmir.7956CrossRefGoogle ScholarPubMed
Blake, M. J., Blake, L. M., Schwartz, O., Raniti, M., Waloszek, J. M., Murray, G., … Allen, N. B. (2018). Who benefits from adolescent sleep interventions? Moderators of treatment efficacy in a randomized controlled trial of a cognitive-behavioral and mindfulness-based group sleep intervention for at-risk adolescents. Journal of Child Psychology and Psychiatry, 59, 637649. doi: 10.1111/jcpp.12842CrossRefGoogle Scholar
Blake, M. J., Snoep, L., Raniti, M., Schwartz, O., Waloszek, J. M., Simmons, J. G., … Allen, N. B. (2017). A cognitive-behavioral and mindfulness-based group sleep intervention improves behavior problems in at-risk adolescents by improving perceived sleep quality. Behavior Research and Therapy, 99, 147156. doi: 10.1016/j.brat.2017.10.006CrossRefGoogle ScholarPubMed
Blum, L. H., Vakhrusheva, J., Saperstein, A., Khan, S., Chang, R. W., Hansen, M. C., … Kimhy, D. (2015). Depressed mood in individuals with schizophrenia: A comparison of retrospective and real-time measures. Psychiatry Research, 227, 318323. doi: 10.1016/j.psychres.2015.03.008CrossRefGoogle ScholarPubMed
Boonstra, T. W., Nicholas, J., Wong, Q. J., Shaw, F., Townsend, S., & Christensen, H. (2018). Using mobile phone sensor technology for mental health research: Integrated analysis to identify hidden challenges and potential solutions. Journal of Medical Internet Research, 20, e10131. doi: 10.2196/10131CrossRefGoogle ScholarPubMed
Chapman, A. L., Rosenthal, M. Z., Dixon-Gordon, K. L., Turner, B. J., & Kuppens, P. (2017). Borderline personality disorder and the effects of instructed emotional avoidance or acceptance in daily life. Journal of Personality Disorders, 31, 483502, Retrieved from: Chapman2017BPDAT.pdf (kuleuven.be).CrossRefGoogle ScholarPubMed
Cheng, Q., Li, T. M., Kwok, C. L., Zhu, T., & Yip, P. S. (2017). Assessing suicide risk and emotional distress in Chinese social media: A text mining and machine learning study. Journal of Medical Internet Research, 19, e243. doi: 10.2196/jmir.7276CrossRefGoogle ScholarPubMed
Collip, D., Nicolson, N. A., Lardinois, M., Lataster, T., van Os, J., Myin-Germeys, I., & G.R.O.U.P. (2011a). Daily cortisol, stress reactivity and psychotic experiences in individuals at above average genetic risk for psychosis. Psychological Medicine, 41, 23052315. doi: 10.1017/S0033291711000602CrossRefGoogle ScholarPubMed
Collip, D., Oorschot, M., Thewissen, V., Van Os, J., Bentall, R., & Myin-Germeys, I. (2011b). Social world interactions: How company connects to paranoia. Psychological Medicine, 41, 911921. doi: 10.1017/S0033291710001558CrossRefGoogle ScholarPubMed
Collip, D., van Winkel, R., Peerbooms, O., Lataster, T., Thewissen, V., Lardinois, M., … Myin-Germeys, I. (2011c). COMT Val158Met-stress interaction in psychosis: Role of background psychosis risk. CNS Neuroscience and Therapeutics, 17, 612619. doi: 10.1111/j.1755-5949.2010.00213.xCrossRefGoogle ScholarPubMed
Collip, D., Wigman, J. T., van Os, J., Oorschot, M., Jacobs, N., Derom, C., … Myin-Germeys, I. (2014). Positive emotions from social company in women with persisting subclinical psychosis: Lessons from daily life. Acta Psychiatrica Scandinavica, 129, 202210. doi: 10.1111/acps.12151CrossRefGoogle ScholarPubMed
Crosby, R. D., Wonderlich, S. A., Engel, S. G., Simonich, H., Smyth, J., & Mitchell, J. E. (2009). Daily mood patterns and bulimic behaviors in the natural environment. Behavior Research and Therapy, 47, 181188. doi: 10.1016/j.brat.2008.11.006CrossRefGoogle ScholarPubMed
Csikszentmihalyi, M., & Larson, R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Disease, 175, 526536. doi: 10.1007/978-94-017-9088-8_3CrossRefGoogle ScholarPubMed
Daniels, N. E. M., Bartels, S. L., Verhagen, S. J. W., Van Knippenberg, R. J. M., De Vugt, M. E., & Delespaul, P. (2020). Digital assessment of working memory and processing speed in everyday life: Feasibility, validation, and lessons-learned. Internet Interventions, 19, 100300. doi: 10.1016/j.invent.2019.100300CrossRefGoogle ScholarPubMed
Difrancesco, S., Lamers, F., Riese, H., Merikangas, K. R., Beekman, A. T. F., van Hemert, A. M., … Penninx, B. (2019). Sleep, circadian rhythm, and physical activity patterns in depressive and anxiety disorders: A 2-week ambulatory assessment study. Depression and Anxiety, 36, 975986. doi: 10.1002/da.22949CrossRefGoogle ScholarPubMed
Eichstaedt, J. C., Smith, R. J., Merchant, R. M., Ungar, L. H., Crutchley, P., Preotiuc-Pietro, D., … Schwartz, H. A. (2018). Facebook language predicts depression in medical records. Proceedings of National Academy of Science of the United States of America, 115, 1120311208. doi: 10.1073/pnas.1802331115CrossRefGoogle ScholarPubMed
Engel, S. G., Wonderlich, S. A., Crosby, R. D., Mitchell, J. E., Crow, S., Peterson, C. B., … Gordon, K. H. (2013). The role of affect in the maintenance of anorexia nervosa: Evidence from a naturalistic assessment of momentary behaviors and emotion. Journal of Abnormal Psychology, 122, 709719. doi: 10.1037/a0034010CrossRefGoogle ScholarPubMed
Erwin, M. C., Dennis, P. A., Coughlin, L. N., Calhoun, P. S., & Beckham, J. C. (2019). Examining the relationship between negative affect and posttraumatic stress disorder symptoms among smokers using ecological momentary assessment. Journal of Affective Disorders, 253, 285291. doi: 10.1016/j.jad.2019.04.035CrossRefGoogle ScholarPubMed
Fang, S. H., Suzuki, K., Lim, C. L., Chung, M. S., Ku, P. W., & Chen, L. J. (2016). Associations between sleep quality and inflammatory markers in patients with schizophrenia. Psychiatry Research, 246, 154160. doi: 10.1016/j.psychres.2016.09.032CrossRefGoogle ScholarPubMed
Fatseas, M., Serre, F., Swendsen, J., & Auriacombe, M. (2018). Effects of anxiety and mood disorders on craving and substance use among patients with substance use disorder: An ecological momentary assessment study. Drug and Alcohol Dependence, 187, 242248. doi: 10.1016/j.drugalcdep.2018.03.008CrossRefGoogle ScholarPubMed
Fitzsimmons-Craft, E. E., Accurso, E. C., Ciao, A. C., Crosby, R. D., Cao, L., Pisetsky, E. M., … Wonderlich, S. A. (2015). Restrictive eating in anorexia nervosa: Examining maintenance and consequences in the natural environment. International Journal of Eating Disorders, 48, 923931. doi: 10.1002/eat.22439CrossRefGoogle ScholarPubMed
Friedmann, F., Santangelo, P., Ebner-Priemer, U., Hill, H., Neubauer, A. B., Rausch, S., … Priebe, K. (2020). Life within a limited radius: Investigating activity space in women with a history of child abuse using global positioning system tracking. PLoS One, 15, e0232666. doi: 10.1371/journal.pone.0232666CrossRefGoogle ScholarPubMed
Frissen, A., Lieverse, R., Drukker, M., Delespaul, P., Lataster, T., Myin-Germeys, I., & van Os, J. (2014). Evidence that childhood urban environment is associated with blunted stress reactivity across groups of patients with psychosis, relatives of patients and controls. Social Psychiatry and Psychiatric Epidemiology, 49, 15791587. doi: 10.1007/s00127-014-0859-3CrossRefGoogle ScholarPubMed
Geoffroy, P. A., Micoulaud Franchi, J. A., Maruani, J., Philip, P., Boudebesse, C., Benizri, C., … Etain, B. (2019). Clinical characteristics of obstructive sleep apnea in bipolar disorders. Journal of Affective Disorders, 245, 17. doi: 10.1016/j.jad.2018.10.096CrossRefGoogle ScholarPubMed
Gerritsen, C., Bagby, R. M., Sanches, M., Kiang, M., Maheandiran, M., Prce, I., & Mizrahi, R. (2019). Stress precedes negative symptom exacerbations in clinical high risk and early psychosis: A time-lagged experience sampling study. Schizophrenia Research, 210, 5258. doi: 10.1016/j.schres.2019.06.015CrossRefGoogle ScholarPubMed
Glaser, J. P., Van Os, J., Mengelers, R., & Myin-Germeys, I. (2008). A momentary assessment study of the reputed emotional phenotype associated with borderline personality disorder. Psychological Medicine, 38, 12311239. doi: 10.1017/S0033291707002322CrossRefGoogle ScholarPubMed
Glaser, J. P., Van Os, J., Thewissen, V., & Myin-Germeys, I. (2010). Psychotic reactivity in borderline personality disorder. Acta Psychiatrica Scandinavica, 121, 125134. doi: 10.1111/j.1600-0447.2009.01427.xCrossRefGoogle ScholarPubMed
Goldschmidt, A. B., Peterson, C. B., Wonderlich, S. A., Crosby, R. D., Engel, S. G., Mitchell, J. E., … Berg, K. C. (2013). Trait-level and momentary correlates of bulimia nervosa with a history of anorexia nervosa. International Journal of Eating Disorders, 46, 140146. doi: 10.1002/eat.22054CrossRefGoogle ScholarPubMed
Goldschmidt, A. B., Wonderlich, S. A., Crosby, R. D., Engel, S. G., Lavender, J. M., Peterson, C. B., … Mitchell, J. E. (2014). Ecological momentary assessment of stressful events and negative affect in bulimia nervosa. Journal of Consulting and Clinical Psychology, 82, 3039. doi: 10.1037/a0034974CrossRefGoogle ScholarPubMed
Goodlin-Jones, B. L., Waters, S., & Anders, T. F. (2009). Objective sleep measurement in typically and atypically developing preschool children with ADHD-like profiles. Child Psychiatry and Human Development, 40, 257268. doi: 10.1007/s10578-009-0124-2CrossRefGoogle ScholarPubMed
Hall, M., Scherner, P. V., Kreidel, Y., & Rubel, J. A. (2021). A systematic review of momentary assessment designs for mood and anxiety symptoms. Frontiers in Psychology, 12, 113. doi: 10.3389/fpsyg.2021.642044CrossRefGoogle ScholarPubMed
Hartmann, J. A., Wichers, M., Menne-Lothmann, C., Kramer, I., Viechtbauer, W., Peeters, F., … Simons, C. J. (2015). Experience sampling-based personalized feedback and positive affect: A randomized controlled trial in depressed patients. PLoS One, 10, e0128095. doi: 10.1371/journal.pone.0128095CrossRefGoogle ScholarPubMed
Haynos, A. F., Crosby, R. D., Engel, S. G., Lavender, J. M., Wonderlich, S. A., Mitchell, J. E., … Le Grange, D. (2015). Initial test of an emotional avoidance model of restriction in anorexia nervosa using ecological momentary assessment. Journal of Psychiatric Research, 68, 134139. doi: 10.1016/j.jpsychires.2015.06.016CrossRefGoogle ScholarPubMed
Heron, K. E., & Smyth, J. M. (2010). Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments. British Journal of Health Psychology, 15, 139. doi: 10.1348/135910709X466063CrossRefGoogle ScholarPubMed
Hswen, Y., Naslund, J. A., Brownstein, J. S., & Hawkins, J. B. (2018). Online communication about depression and anxiety among twitter users with schizophrenia: Preliminary findings to inform a digital phenotype using social media. Psychiatric Quaterly, 89, 569580. doi: 10.1007/s11126-017-9559-yGoogle ScholarPubMed
Hswen, Y., Naslund, J. A., Chandrashekar, P., Siegel, R., Brownstein, J. S., & Hawkins, J. B. (2017). Exploring online communication about cigarette smoking among Twitter users who self-identify as having schizophrenia. Psychiatry Research, 257, 479484. doi: 10.1016/j.psychres.2017.08.002CrossRefGoogle ScholarPubMed
Insel, T. R. (2017). Digital phenotyping: Technology for a new science of behavior. JAMA, 318, 12151216. doi: 10.1001/jama.2017.11295CrossRefGoogle ScholarPubMed
Insel, T. R. (2018). Digital phenotyping: A global tool for psychiatry. World Psychiatry, 17, 276277. doi: 10.1002/wps.20550CrossRefGoogle ScholarPubMed
Jahng, S., Solhan, M. B., Tomko, R. L., Wood, P. K., Piasecki, T. M., & Trull, T. J. (2011). Affect and alcohol use: An ecological momentary assessment study of outpatients with borderline personality disorder. Journal of Abnormal Psychology, 120, 572584. doi: 10.1037/a0024686CrossRefGoogle ScholarPubMed
Jain, S. H., Powers, B. W., Hawkins, J. B., & Brownstein, J. S. (2015). The digital phenotype. Nature Biotechnology, 33, 462463. doi: 10.1038/nbt.3223CrossRefGoogle ScholarPubMed
Johns, J. T., Di, J., Merikangas, K., Cui, L., Swendsen, J., & Zipunnikov, V. (2019). Fragmentation as a novel measure of stability in normalized trajectories of mood and attention measured by ecological momentary assessment. Psychological Assessment, 31, 329339. doi: 10.1037/pas0000661CrossRefGoogle ScholarPubMed
Karr, T. M., Crosby, R. D., Cao, L., Engel, S. G., Mitchell, J. E., Simonich, H., & Wonderlich, S. A. (2013). Posttraumatic stress disorder as a moderator of the association between negative affect and bulimic symptoms: An ecological momentary assessment study. Comprehensive Psychiatry, 54, 6169. doi: 10.1016/j.comppsych.2012.05.011CrossRefGoogle ScholarPubMed
Khazanov, G. K., Ruscio, A. M., & Swendsen, J. (2019). The “Brightening” effect: Reactions to positive events in the daily lives of individuals with major depressive disorder and generalized anxiety disorder. Behavior Therapy, 50, 270284. doi: 10.1016/j.beth.2018.05.008CrossRefGoogle Scholar
Kimhy, D., Vakhrusheva, J., Khan, S., Chang, R. W., Hansen, M. C., Ballon, J. S., … Gross, J. J. (2014). Emotional granularity and social functioning in individuals with schizophrenia: An experience sampling study. Journal of Psychiatric Research, 53, 141148. doi: 10.1016/j.jpsychires.2014.01.020CrossRefGoogle ScholarPubMed
Kirtley, O. J., Lafit, G., Achterhof, R., Hiekkaranta, A. P., & Myin-Germeys, I. (2021). Making the black box transparent: A template and tutorial for registration of studies using experience-sampling methods. Advances in Methods and Practices in Psychological Science, 4(1), 116. doi: 10.1177/2515245920924686CrossRefGoogle Scholar
Klippel, A., Myin-Germeys, I., Chavez-Baldini, U., Preacher, K. J., Kempton, M., Valmaggia, L., … Reininghaus, U. (2017). Modeling the interplay between psychological processes and adverse, stressful contexts and experiences in pathways to psychosis: An experience sampling study. Schizophrenia Bulletin, 43, 302315. doi: 10.1093/schbul/sbw185CrossRefGoogle ScholarPubMed
Klippel, A., Schick, A., Myin-Germeys, I., Rauschenberg, C., Vaessen, T., & Reininghaus, U. (2021). Modelling the temporal interplay between stress and affective disturbances in pathways to psychosis: An experience sampling study. Psychological Medicine, 110. doi: 10.1017/S0033291720004894Google ScholarPubMed
Klippel, A., Viechtbauer, W., Reininghaus, U., Wigman, J., van Borkulo, C., MERGE, … Wichers, M. (2018). The cascade of stress: A network approach to explore differential dynamics in populations varying in risk for psychosis. Schizophrenia Bulletin, 44, 328337. doi: 10.1093/schbul/sbx037CrossRefGoogle Scholar
Koppe, G., Guloksuz, S., Reininghaus, U., & Durstewitz, D. (2019). Recurrent neural networks in mobile sampling and intervention. Schizophrenia Bulletin, 45, 272276. doi: 10.1093/schbul/sby171CrossRefGoogle ScholarPubMed
Kordy, H., Wolf, M., Aulich, K., Burgy, M., Hegerl, U., Husing, J., … Backenstrass, M. (2016). Internet-Delivered disease management for recurrent depression: A multicenter randomized controlled trial. Psychotherapy and Psychosomatics, 85, 9198. doi: 10.1159/000441951CrossRefGoogle ScholarPubMed
Kramer, I., Simons, C. J., Hartmann, J. A., Menne-Lothmann, C., Viechtbauer, W., Peeters, F., … Wichers, M. (2014). A therapeutic application of the experience sampling method in the treatment of depression: A randomized controlled trial. World Psychiatry, 13, 6877. doi: 10.1002/wps.20090CrossRefGoogle ScholarPubMed
Kuepper, R., Oorschot, M., Myin-Germeys, I., Smits, M., van Os, J., & Henquet, C. (2013). Is psychotic disorder associated with increased levels of craving for cannabis? An experience sampling study. Acta Psychiatrica Scandinavia, 128, 448456. doi: 10.1111/acps.12078CrossRefGoogle ScholarPubMed
Lataster, J., van Os, J., de Haan, L., Thewissen, V., Bak, M., Lataster, T., … Myin-Germeys, I. (2011). Emotional experience and estimates of D2 receptor occupancy in psychotic patients treated with haloperidol, risperidone, or olanzapine: An experience sampling study. Journal of Clinical Psychiatry, 72, 13971404. doi: 10.4088/JCP.09m05466yelCrossRefGoogle ScholarPubMed
Lavender, J. M., Utzinger, L. M., Crosby, R. D., Goldschmidt, A. B., Ellison, J., Wonderlich, S. A., … Le Grange, D. (2016). A naturalistic examination of the temporal patterns of affect and eating disorder behaviors in anorexia nervosa. International Journal of Eating Disorders, 49, 7783. doi: 10.1002/eat.22447CrossRefGoogle ScholarPubMed
Leahey, T. M., Crowther, J. H., & Ciesla, J. A. (2011). An ecological momentary assessment of the effects of weight and shape social comparisons on women with eating pathology, high body dissatisfaction, and low body dissatisfaction. Behavior Therapy, 42, 197210. doi: 10.1016/j.beth.2010.07.003CrossRefGoogle ScholarPubMed
Leendertse, P., Myin-Germeys, I., Lataster, T., Simons, C. J. P., Oorschot, M., Lardinois, M., … G.R.O.U.P. Investigators. (2018). Subjective quality of life in psychosis: Evidence for an association with real world functioning? Psychiatry Research, 261, 116123. doi: 10.1002/mpr.1352CrossRefGoogle ScholarPubMed
Leraas, B. C., Smith, K. E., Utzinger, L. M., Cao, L., Engel, S. G., Crosby, R. D., … Wonderlich, S. A. (2018). Affect-based profiles of bulimia nervosa: The utility and validity of indicators assessed in the natural environment. Psychiatry Research, 259, 210215. doi: 10.1016/j.psychres.2017.09.080CrossRefGoogle ScholarPubMed
Lilford, R. J., Richardson, A., Stevens, A., Fitzpatrick, R., Edwards, S., Rock, F., & Hutton, J. L. (2001). Issues in methodological research: Perspectives from researchers and commissioners. Health Technology Assessment, 5, 157. doi: 10.3310/hta5080CrossRefGoogle ScholarPubMed
Lovato, N., Lack, L., Wright, H., & Kennaway, D. J. (2014). Evaluation of a brief treatment program of cognitive behavior therapy for insomnia in older adults. Sleep, 37, 117126. doi: 10.5665/sleep.3320CrossRefGoogle ScholarPubMed
Mackesy-Amiti, M. E., & Donenberg, G. (2020). Negative affect and emotion dysregulation among people who inject drugs: An ecological momentary assessment study. Psychology of Addictive Behaviors, 34, 650659. doi: 10.1037/adb0000577CrossRefGoogle ScholarPubMed
Mason, T. B., Lavender, J. M., Wonderlich, S. A., Steiger, H., Cao, L., Engel, S. G., … Crosby, R. D. (2017). Comfortably numb: The role of momentary dissociation in the experience of negative affect around binge eating. Journal Nervous Mental Disorders, 205, 335339. doi: 10.1097/NMD.0000000000000658CrossRefGoogle ScholarPubMed
McCrae, C. S., Williams, J., Roditi, D., Anderson, R., Mundt, J. M., Miller, M. B., … Robinson, M. E. (2019). Cognitive behavioral treatments for insomnia and pain in adults with comorbid chronic insomnia and fibromyalgia: Clinical outcomes from the SPIN randomized controlled trial. Sleep, 42, 01. doi: 10.1093/sleep/zsy234CrossRefGoogle ScholarPubMed
McMakin, D. L., Ricketts, E. J., Forbes, E. E., Silk, J. S., Ladouceur, C. D., Siegle, G. J., … Dahl, R. E. (2019). Anxiety treatment and targeted sleep enhancement to address sleep disturbance in pre/early adolescents with anxiety. Journal of Clinical Child and Adolescent Psychology, 48, S284S297. doi: 10.1080/15374416.2018.1463534CrossRefGoogle ScholarPubMed
Merikangas, K. R., Swendsen, J., Hickie, I. B., Cui, L., Shou, H., Merikangas, A. K., … Zipunnikov, V. (2019). Real-time mobile monitoring of the dynamic associations among motor activity, energy, mood, and sleep in adults with bipolar disorder. JAMA Psychiatry, 76, 190198. doi: 10.1001/jamapsychiatry.2018.3546CrossRefGoogle ScholarPubMed
Mokkink, L. B., Terwee, C. B., Patrick, D. L., Alonso, J., Stratford, P. W., Knol, D. L., … de Vet, H. C. (2010). The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. Journal of Clinical Epidemiology, 63, 737745. doi: 10.1016/j.jclinepi.2010.02.006CrossRefGoogle ScholarPubMed
Morgan, C., Burns, T., Fitzpatrick, R., Pinfold, V., & Priebe, S. (2007). Social exclusion and mental health: Conceptual and methodological review. British Journal of Psychiatry, 191, 477483. doi: 10.1192/bjp.bp.106.034942CrossRefGoogle ScholarPubMed
Morgan, J. K., Lee, G. E., Wright, A. G. C., Gilchrist, D. E., Forbes, E. E., McMakin, D. L., … Silk, J. S. (2017). Altered positive affect in clinically anxious youth: The role of social context and anxiety subtype. Journal of Abnormal Child Psychology, 45, 14611472. doi: 10.1007/s10802-016-0256-3CrossRefGoogle ScholarPubMed
Muehlenkamp, J. J., Engel, S. G., Wadeson, A., Crosby, R. D., Wonderlich, S. A., Simonich, H., & Mitchell, J. E. (2009). Emotional states preceding and following acts of non-suicidal self-injury in bulimia nervosa patients. Behavior Research and Therapy, 47, 8387. doi: 10.1016/j.brat.2008.10.011CrossRefGoogle ScholarPubMed
Myin-Germeys, I., Birchwood, M., & Kwapil, T. (2011). From environment to therapy in psychosis: A real-world momentary assessment approach. Schizophrenia Bulletin, 37(2), 244247. doi: 10.1093/schbul/sbq164CrossRefGoogle ScholarPubMed
Myin-Germeys, I., Kasanova, Z., Vaessen, T., Vachon, H., Kirtley, O., Viechtbauer, W., & Reininghaus, U. (2018). Experience sampling methodology in mental health research: New insights and technical developments. World Psychiatry, 17, 123132. doi: 10.1002/wps.20513CrossRefGoogle ScholarPubMed
Myin-Germeys, I., Klippel, A., Steinhart, H., & Reininghaus, U. (2016). Ecological momentary interventions in psychiatry. Current Opinion in Psychiatry, 29, 258263. doi: 10.1097/YCO.0000000000000255CrossRefGoogle ScholarPubMed
Myin-Germeys, I., Oorschot, M., Collip, D., Lataster, J., Delespaul, P., & van Os, J. (2009). Experience sampling research in psychopathology: Opening the black box of daily life. Psychological Medicine, 39, 15331547. doi: 10.1017/S0033291708004947CrossRefGoogle ScholarPubMed
Naslund, J. A., Aschbrenner, K. A., Araya, R., Marsch, L. A., Unutzer, J., Patel, V., & Bartels, S. J. (2017). Digital technology for treating and preventing mental disorders in low-income and middle-income countries: A narrative review of the literature. The Lancet. Psychiatry, 4, 486500. doi: 10.1016/S2215-0366(17)30096-2CrossRefGoogle ScholarPubMed
National Institute for Health and Care Excellence, N. (2021). NICE guideline [NG197]. Shared decision making. Retrieved from http://www.nice.org.uk/guidance/ng197.Google Scholar
Oorschot, M., Lataster, T., Thewissen, V., Lardinois, M., van Os, J., Delespaul, P. A., & Myin-Germeys, I. (2012). Symptomatic remission in psychosis and real-life functioning. British Journal of Psychiatry, 201, 215220. doi: 10.1371/journal.pone.0086652CrossRefGoogle ScholarPubMed
Owens, J., Sangal, R. B., Sutton, V. K., Bakken, R., Allen, A. J., & Kelsey, D. (2009). Subjective and objective measures of sleep in children with attention-deficit/hyperactivity disorder. Sleep Medicine, 10, 446456. doi: 10.1016/j.sleep.2008.03.013CrossRefGoogle ScholarPubMed
Pearson, C. M., Lavender, J. M., Cao, L., Wonderlich, S. A., Crosby, R. D., Engel, S. G., … Crow, S. J. (2017). Associations of borderline personality disorder traits with stressful events and emotional reactivity in women with bulimia nervosa. Journal of Abnormal Psychology, 126, 531539. doi: 10.1037/abn0000225CrossRefGoogle ScholarPubMed
Pearson, C. M., Pisetsky, E. M., Goldschmidt, A. B., Lavender, J. M., Wonderlich, S. A., Crosby, R. D., … Peterson, C. B. (2016). Personality psychopathology differentiates risky behaviors among women with bulimia nervosa. International Journal of Eating Disorders, 49, 681688. doi: 10.1002/eat.22570CrossRefGoogle ScholarPubMed
Peerbooms, O., Rutten, B. P., Collip, D., Lardinois, M., Lataster, T., Thewissen, V., … van Winkel, R. (2012). Evidence that interactive effects of COMT and MTHFR moderate psychotic response to environmental stress. Acta Psychiatrica Scandinavica, 125, 247256. doi: 10.1111/j.1600-0447.2011.01806.xCrossRefGoogle ScholarPubMed
Perez Arribas, I., Goodwin, G. M., Geddes, J. R., Lyons, T., & Saunders, K. E. A. (2018). A signature-based machine learning model for distinguishing bipolar disorder and borderline personality disorder. Translational Psychiatry, 8, 274. doi: 10.1038/s41398-018-0334-0CrossRefGoogle ScholarPubMed
Piette, J. D., Valenstein, M., Himle, J., Duffy, S., Torres, T., Vogel, M., & Richardson, C. (2011). Clinical complexity and the effectiveness of an intervention for depressed diabetes patients. Chronic Illness, 7, 267278. doi: 10.1177/1742395311409259CrossRefGoogle ScholarPubMed
Pisetsky, E. M., Crosby, R. D., Cao, L., Fitzsimmons-Craft, E. E., Mitchell, J. E., Engel, S. G., … Peterson, C. B. (2016). An examination of affect prior to and following episodes of getting drunk in women with bulimia nervosa. Psychiatry Research, 240, 202208. doi: 10.1016/j.psychres.2016.04.044CrossRefGoogle ScholarPubMed
Pratap, A., Atkins, D. C., Renn, B. N., Tanana, M. J., Mooney, S. D., Anguera, J. A., & Arean, P. A. (2019). The accuracy of passive phone sensors in predicting daily mood. Depression and Anxiety, 36, 7281. doi: 10.1002/da.22822CrossRefGoogle ScholarPubMed
Rauschenberg, C., Hirjak, D., Ganslandt, T., Schulte-Strathaus, J. C., Schick, A., Meyer-Lindenberg, A., & Reininghaus, U. (2021a). Digital forms of service delivery for personalized crisis resolution and home treatment. Der Nervenarzt, 93(3), 279287. doi: 10.1007/s00115-021-01100-5CrossRefGoogle ScholarPubMed
Rauschenberg, C., Schick, A., Hirjak, D., Seidler, A., Paetzold, I., Apfelbacher, C., … Reininghaus, U. (2021b). Evidence synthesis of digital interventions to mitigate the negative impact of the COVID-19 pandemic on public mental health: Rapid meta-review. Journal of Medical Internet Research, 23, e23365. doi: 10.2196/23365CrossRefGoogle ScholarPubMed
Rauschenberg, C., van Os, J., Cremers, D., Goedhart, M., Schieveld, J. N. M., & Reininghaus, U. (2017). Stress sensitivity as a putative mechanism linking childhood trauma and psychopathology in youth's daily life. Acta Psychiatrica Scandinavica, 136, 373388. doi: 10.1111/acps.12775CrossRefGoogle ScholarPubMed
Reece, A. G., Reagan, A. J., Lix, K. L. M., Dodds, P. S., Danforth, C. M., & Langer, E. J. (2017). Forecasting the onset and course of mental illness with Twitter data. Scientific Reports, 7, 13006. doi: 10.1038/s41598-017-12961-9CrossRefGoogle ScholarPubMed
Reininghaus, U. (2018). Ecological momentary interventions in psychiatry: The momentum for change in daily social context. Journal Psychiatrische Praxis, 45, 5961. doi: 10.1055/s-0044-101986Google ScholarPubMed
Reininghaus, U., Depp, C. A., & Myin-Germeys, I. (2016a). Ecological interventionist causal models in psychosis: Targeting psychological mechanisms in daily life. Schizophrenia Bulletin, 42, 264269. doi: 10.1093/schbul/sbv193CrossRefGoogle ScholarPubMed
Reininghaus, U., Gayer-Anderson, C., Valmaggia, L., Kempton, M. J., Calem, M., Onyejiaka, A., … Morgan, C. (2016b). Psychological processes underlying the association between childhood trauma and psychosis in daily life: An experience sampling study. Psychological Medicine, 46, 27992813. doi: 10.1017/S003329171600146XCrossRefGoogle ScholarPubMed
Reininghaus, U., Kempton, M. J., Valmaggia, L., Craig, T. K., Garety, P., Onyejiaka, A., … Morgan, C. (2016c). Stress sensitivity, aberrant salience, and threat anticipation in early psychosis: An experience sampling study. Schizophrenia Bulletin, 42, 712722. doi: 10.1093/schbul/sbv190CrossRefGoogle ScholarPubMed
Reininghaus, U., Oorschot, M., Moritz, S., Gayer-Anderson, C., Kempton, M. J., Valmaggia, L., … Myin-Germeys, I. (2019). Liberal acceptance bias, momentary aberrant salience, and psychosis: An experimental experience sampling study. Schizophrenia Bulletin, 45, 871882. doi: 10.1093/schbul/sby116CrossRefGoogle ScholarPubMed
Reininghaus, U., & Priebe, S. (2012). Measuring patient-reported outcomes in psychosis: Conceptual and methodological review. British Journal of Psychiatry, 201, 262267. doi: 10.1192/bjp.bp.111.107615CrossRefGoogle ScholarPubMed
Rintala, A., Wampers, M., Myin-Germeys, I., & Viechtbauer, W. (2019). Response compliance and predictors thereof in studies using the experience sampling method. Psychological Assessment, 31, 226235. doi: 10.1037/pas0000662CrossRefGoogle ScholarPubMed
Robillard, R., Naismith, S. L., Smith, K. L., Rogers, N. L., White, D., Terpening, Z., … Hickie, I. B. (2014). Sleep-wake cycle in young and older persons with a lifetime history of mood disorders. PLoS One, 9, e87763. doi: 10.1371/journal.pone.0087763CrossRefGoogle Scholar
Ruscio, A. M., Gentes, E. L., Jones, J. D., Hallion, L. S., Coleman, E. S., & Swendsen, J. (2015). Rumination predicts heightened responding to stressful life events in major depressive disorder and generalized anxiety disorder. Journal of Abnormal Psychology, 124, 1726. doi: 10.1037/abn0000025CrossRefGoogle ScholarPubMed
Schaefer, L. M., Smith, K. E., Anderson, L. M., Cao, L., Crosby, R. D., Engel, S. G., … Wonderlich, S. A. (2020). The role of affect in the maintenance of binge-eating disorder: Evidence from an ecological momentary assessment study. Journal of Abnormal Psychology, 129, 387396. doi: 10.1037/abn0000517CrossRefGoogle ScholarPubMed
Schlam, T. R., Baker, T. B., Smith, S. S., Cook, J. W., & Piper, M. E. (2020). Anxiety sensitivity and distress tolerance in smokers: Relations with tobacco dependence, withdrawal, and quitting successdagger. Nicotine and Tobacco Research, 22, 5865. doi: 10.1093/ntr/ntz070CrossRefGoogle Scholar
Scott, L. N., Wright, A. G. C., Beeney, J. E., Lazarus, S. A., Pilkonis, P. A., & Stepp, S. D. (2017). Borderline personality disorder symptoms and aggression: A within-person process model. Journal of Abnormal Psychology, 126, 429440. doi: 10.1037/abn0000272CrossRefGoogle ScholarPubMed
Selby, E. A., Doyle, P., Crosby, R. D., Wonderlich, S. A., Engel, S. G., Mitchell, J. D., & Le Grange, D. (2012). Momentary emotion surrounding bulimic behaviors in women with bulimia nervosa and borderline personality disorder. Journal of Psychiatric Research, 46, 14921500. doi: 10.1016/j.jpsychires.2012.08.014CrossRefGoogle ScholarPubMed
Serre, F., Fatseas, M., Denis, C., Swendsen, J., & Auriacombe, M. (2018). Predictors of craving and substance use among patients with alcohol, tobacco, cannabis or opiate addictions: Commonalities and specificities across substances. Addictive Behavior, 83, 123129. doi: 10.1016/j.addbeh.2018.01.041CrossRefGoogle ScholarPubMed
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 132. doi: 10.1146/annurev.clinpsy.3.022806.091415CrossRefGoogle ScholarPubMed
Shou, H., Cui, L., Hickie, I., Lameira, D., Lamers, F., Zhang, J., … Merikangas, K. R. (2017). Dysregulation of objectively assessed 24-hour motor activity patterns as a potential marker for bipolar I disorder: Results of a community-based family study. Translational Psychiatry, 7, e1211. doi: 10.1038/tp.2017.136CrossRefGoogle ScholarPubMed
Silk, J. S., Tan, P. Z., Ladouceur, C. D., Meller, S., Siegle, G. J., McMakin, D. L., … Ryan, N. D. (2018). A randomized clinical trial comparing individual cognitive behavioral therapy and child-centered therapy for child anxiety disorders. Journal of Clinical Child and Adolescent Psychology, 47, 542554. doi: 10.1080/15374416.2016.1138408CrossRefGoogle ScholarPubMed
Simons, C. J., Hartmann, J. A., Kramer, I., Menne-Lothmann, C., Hohn, P., van Bemmel, A. L., … Wichers, M. (2015). Effects of momentary self-monitoring on empowerment in a randomized controlled trial in patients with depression. European Psychiatry, 30, 900906. doi: 10.1016/j.eurpsy.2015.09.004CrossRefGoogle Scholar
Smyth, J. M., Wonderlich, S. A., Sliwinski, M. J., Crosby, R. D., Engel, S. G., Mitchell, J. E., & Calogero, R. M. (2009). Ecological momentary assessment of affect, stress, and binge-purge behaviors: Day of week and time of day effects in the natural environment. International Journal of Eating Disorders, 42, 429436. doi: 10.1002/eat.v42:510.1002/eat.20623CrossRefGoogle ScholarPubMed
Solhan, M. B., Trull, T. J., Jahng, S., & Wood, P. K. (2009). Clinical assessment of affective instability: Comparing EMA indices, questionnaire reports, and retrospective recall. Psychological Assessment, 21, 425436. doi: 10.1037/a0016869CrossRefGoogle ScholarPubMed
Stubbs, B., Ku, P. W., Chung, M. S., & Chen, L. J. (2017). Relationship between objectively measured sedentary behavior and cognitive performance in patients with schizophrenia Vs controls. Schizophrenia Bulletin, 43, 566574. doi: 10.1093/schbul/sbw126Google ScholarPubMed
Tasca, G. A., Illing, V., Balfour, L., Krysanski, V., Demidenko, N., Nowakowski, J., & Bissada, H. (2009). Psychometric properties of self-monitoring of eating disorder urges among treatment seeking women: Ecological momentary assessment using a daily diary method. Eating Behavior, 10, 5961. doi: 10.1016/j.eatbeh.2008.10.004CrossRefGoogle ScholarPubMed
TE, T. (2013). Endnote. Philadelphia, PA: Clarivate Analytics.Google Scholar
Tost, H., Reichert, M., Braun, U., Reinhard, I., Peters, R., Lautenbach, S., … Meyer-Lindenberg, A. (2019). Neural correlates of individual differences in affective benefit of real-life urban green space exposure. Nature Neuroscience, 22, 13891393. doi: 10.1038/s41593-019-0451-yCrossRefGoogle ScholarPubMed
Trull, T. J., & Ebner-Priemer, U. (2014). The role of ambulatory assessment in psychological science. Current Directions in Psychological Science, 23, 466470. doi: 10.1177/0963721414550706CrossRefGoogle ScholarPubMed
Trull, T. J., & Ebner-Priemer, U. W. (2020). Ambulatory assessment in psychopathology research: A review of recommended reporting guidelines and current practices. Journal of Abnormal Psychology, 129(1), 5663. doi: 10.1037/abn0000473CrossRefGoogle ScholarPubMed
Tsanas, A., Saunders, K. E., Bilderbeck, A. C., Palmius, N., Osipov, M., Clifford, G. D., … De Vos, M. (2016). Daily longitudinal self-monitoring of mood variability in bipolar disorder and borderline personality disorder. Journal of Affective Disorders, 205, 225233. doi: 10.1016/j.jad.2016.06.065CrossRefGoogle ScholarPubMed
Verkooijen, S., Stevelink, R., Abramovic, L., Vinkers, C. H., Ophoff, R. A., Kahn, R. S., … van Haren, N. E. (2017). The association of sleep and physical activity with integrity of white matter microstructure in bipolar disorder patients and healthy controls. Psychiatry Research: Neuroimaging, 262, 7180. doi: 10.1016/j.pscychresns.2017.01.013CrossRefGoogle ScholarPubMed
Wallace, M. L., McMakin, D. L., Tan, P. Z., Rosen, D., Forbes, E. E., Ladouceur, C. D., … Silk, J. S. (2017). The role of day-to-day emotions, sleep, and social interactions in pediatric anxiety treatment. Behavior Research and Therapy, 90, 8795. doi: 10.1016/j.brat.2016.12.012CrossRefGoogle ScholarPubMed
Wallen, G. R., Park, J., Krumlauf, M., & Brooks, A. T. (2019). Identification of distinct latent classes related to sleep, PTSD, depression, and anxiety in individuals diagnosed with severe alcohol use disorder. Behavior Sleep Medicine, 17, 514523. doi: 10.1080/15402002.2018.1425867CrossRefGoogle ScholarPubMed
Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54, 10631070. doi: 10.1037/0022-3514.54.6.1063CrossRefGoogle ScholarPubMed
Wichniak, A., Skowerska, A., Chojnacka-Wojtowicz, J., Taflinski, T., Wierzbicka, A., Jernajczyk, W., & Jarema, M. (2011). Actigraphic monitoring of activity and rest in schizophrenic patients treated with olanzapine or risperidone. Journal of Psychiatric Research, 45, 13811386. doi: 10.1016/j.jpsychires.2011.05.009CrossRefGoogle ScholarPubMed
Wigman, J. T., van Os, J., Borsboom, D., Wardenaar, K. J., Epskamp, S., Klippel, A., … Wichers, M. (2015). Exploring the underlying structure of mental disorders: Cross-diagnostic differences and similarities from a network perspective using both a top-down and a bottom-up approach. Psychological Medicine, 45, 23752387. doi: 10.1017/S0033291715000331CrossRefGoogle ScholarPubMed
Wonderlich, J. A., Lavender, J. M., Wonderlich, S. A., Peterson, C. B., Crow, S. J., Engel, S. G., … Crosby, R. D. (2015). Examining convergence of retrospective and ecological momentary assessment measures of negative affect and eating disorder behaviors. International Journal of Eating Disorders, 48, 305311. doi: 10.1002/eat.22352CrossRefGoogle ScholarPubMed
Zebin, T., Peek, N., & Casson, A. J. (2019). Physical activity based classification of serious mental illness group participants in the UK Biobank using ensemble dense neural networks. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 2019, 12511254. doi: 10.1109/EMBC.2019.8857532Google ScholarPubMed
Zunker, C., Peterson, C. B., Crosby, R. D., Cao, L., Engel, S. G., Mitchell, J. E., & Wonderlich, S. A. (2011). Ecological momentary assessment of bulimia nervosa: Does dietary restriction predict binge eating? Behavior Research and Therapy, 49, 714717. doi: 10.1037/a0034974CrossRefGoogle ScholarPubMed
Figure 0

Fig. 1. Study selection. Notes: MEDLINE was searched on 30.01.2021. Reasons for exclusion were not meeting inclusion criteria (e.g. studies investigating samples with neurological disorders). Fifteen studies were excluded as they did not meet the criteria for assessment frequency (i.e. less than 20 data points).

Supplementary material: File

Schick et al. supplementary material

Schick et al. supplementary material

Download Schick et al. supplementary material(File)
File 490.1 KB