Hostname: page-component-8678677fb7-2d78d Total loading time: 0 Render date: 2026-10-05T23:58:04.719Z Has data issue: false hasContentIssue false
  • English
  • Français

Predicting states of elevated negative affect in adolescents from smartphone sensors: a novel personalized machine learning approach

Published online by Cambridge University Press:  27 July 2022

Boyu Ren ,
Emma G. Balkind ,
Brianna Pastro ,
Elana S. Israel ,
Diego A. Pizzagalli ,
Habiballah Rahimi-Eichi ,
Justin T. Baker  and
Christian A. Webb Open the ORCID record for Christian A. Webb [Opens in a new window]
Boyu Ren
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
Emma G. Balkind
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
Brianna Pastro
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
Elana S. Israel
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
Diego A. Pizzagalli
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
Habiballah Rahimi-Eichi
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
Justin T. Baker
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
Christian A. Webb*
Affiliation:
Department of Psychiatry, Harvard Medical School, Boston, MA, USA McLean Hospital, Belmont, MA, USA
*
Author for correspondence: Christian A. Webb, E-mail: cwebb@mclean.harvard.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Background

Adolescence is characterized by profound change, including increases in negative emotions. Approximately 84% of American adolescents own a smartphone, which can continuously and unobtrusively track variables potentially predictive of heightened negative emotions (e.g. activity levels, location, pattern of phone usage). The extent to which built-in smartphone sensors can reliably predict states of elevated negative affect in adolescents is an open question.

Methods

Adolescent participants (n = 22; ages 13–18) with low to high levels of depressive symptoms were followed for 15 weeks using a combination of ecological momentary assessments (EMAs) and continuously collected passive smartphone sensor data. EMAs probed negative emotional states (i.e. anger, sadness and anxiety) 2–3 times per day every other week throughout the study (total: 1145 EMA measurements). Smartphone accelerometer, location and device state data were collected to derive 14 discrete estimates of behavior, including activity level, percentage of time spent at home, sleep onset and duration, and phone usage.

Results

A personalized ensemble machine learning model derived from smartphone sensor data outperformed other statistical approaches (e.g. linear mixed model) and predicted states of elevated anger and anxiety with acceptable discrimination ability (area under the curve (AUC) = 74% and 71%, respectively), but demonstrated more modest discrimination ability for predicting states of high sadness (AUC = 66%).

Conclusions

To the extent that smartphone data could provide reasonably accurate real-time predictions of states of high negative affect in teens, brief ‘just-in-time’ interventions could be immediately deployed via smartphone notifications or mental health apps to alleviate these states.

Keywords

Adolescentsecological momentary assessmentnegative affectpredictionsmartphone sensors

Information

Type
Original Article
Information
Psychological Medicine , Volume 53 , Issue 11 , August 2023 , pp. 5146 - 5154
Check for updates
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Airola, A., Pahikkala, T., Waegeman, W., De Baets, B., & Salakoski, T. (2009). A comparison of AUC estimators in small-sample studies. Machine Learning in Systems Biology, 1, 3–13.Google Scholar
Avenevoli, S., Swendsen, J., He, J.-P., Burstein, M., & Merikangas, K. R. (2015). Major depression in the national comorbidity survey–adolescent supplement: Prevalence, correlates, and treatment. Journal of the American Academy of Child & Adolescent Psychiatry, 54(1), 37–44.e2. https://doi.org/10.1016/j.jaac.2014.10.010.CrossRefGoogle ScholarPubMed
Bailen, N. H., Green, L. M., & Thompson, R. J. (2019). Understanding emotion in adolescents: A review of emotional frequency, intensity, instability, and clarity. Emotion Review, 11(1), 63–73.10.1177/1754073918768878CrossRefGoogle Scholar
Boehmke, B., & Greenwell, B. M. (2019). Hands-on machine learning with R (1st ed.). Boca Raton: Chapman and Hall/CRC.10.1201/9780367816377CrossRefGoogle Scholar
Borghuis, J., Denissen, J. J., Oberski, D., Sijtsma, K., Meeus, W. H., Branje, S., … Bleidorn, W. (2017). Big five personality stability, change, and codevelopment across adolescence and early adulthood. Journal of Personality and Social Psychology, 113(4), 641.10.1037/pspp0000138CrossRefGoogle ScholarPubMed
Cao, J., Truong, A. L., Banu, S., Shah, A. A., Sabharwal, A., & Moukaddam, N. (2020). Tracking and predicting depressive symptoms of adolescents using smartphone-based self-reports, parental evaluations, and passive phone sensor data: Development and usability study. JMIR Mental Health, 7(1), e14045.10.2196/14045CrossRefGoogle ScholarPubMed
Christodoulou, E., Ma, J., Collins, G. S., Steyerberg, E. W., Verbakel, J. Y., & Van Calster, B. (2019). A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. Journal of Clinical Epidemiology, 110, 12–22.10.1016/j.jclinepi.2019.02.004CrossRefGoogle ScholarPubMed
Chwastiak, L., Ehde, D. M., Gibbons, L. E., Sullivan, M., Bowen, J. D., & Kraft, G. H. (2002). Depressive symptoms and severity of illness in multiple sclerosis: Epidemiologic study of a large community sample. American Journal of Psychiatry, 159(11), 1862–1868. https://doi.org/10.1176/appi.ajp.159.11.1862.CrossRefGoogle ScholarPubMed
Fisher, A. J., Medaglia, J. D., & Jeronimus, B. F. (2018). Lack of group-to-individual generalizability is a threat to human subjects research. Proceedings of the National Academy of Sciences, 115(27), E6106–E6115.10.1073/pnas.1711978115CrossRefGoogle ScholarPubMed
Forbes, E. E., Hariri, A. R., Martin, S. L., Silk, J. S., Moyles, D. L., Fisher, P. M., … Dahl, R. E. (2009). Altered striatal activation predicting real-world positive affect in adolescent major depressive disorder. American Journal of Psychiatry, 166(1), 64–73. https://doi.org/10.1176/appi.ajp.2008.07081336.CrossRefGoogle ScholarPubMed
Forbes, E. E., Stepp, S. D., Dahl, R. E., Ryan, N. D., Whalen, D., Axelson, D. A., … Silk, J. S. (2012). Real-world affect and social context as predictors of treatment response in child and adolescent depression and anxiety: An ecological momentary assessment study. Journal of Child and Adolescent Psychopharmacology, 22(1), 37–47. https://doi.org/10.1089/cap.2011.0085.CrossRefGoogle ScholarPubMed
Frost, A., Hoyt, L. T., Chung, A. L., & Adam, E. K. (2015). Daily life with depressive symptoms: Gender differences in adolescents’ everyday emotional experiences. Journal of Adolescence, 43, 132–141. https://doi.org/10.1016/j.adolescence.2015.06.001.CrossRefGoogle ScholarPubMed
Griffith, J. M., Clark, H. M., Haraden, D. A., Young, J. F., & Hankin, B. L. (2021). Affective development from middle childhood to late adolescence: Trajectories of mean-level change in negative and positive affect. Journal of Youth and Adolescence, 50(8), 1550–1563.10.1007/s10964-021-01425-zCrossRefGoogle ScholarPubMed
Jacobson, N. C., & Chung, Y. J. (2020). Passive sensing of prediction of moment-to-moment depressed mood among undergraduates with clinical levels of depression sample using smartphones. Sensors, 20(12), 3572.10.3390/s20123572CrossRefGoogle ScholarPubMed
Kaufman, J., Birmaher, B., Axelson, D., Perepletchikova, F., Brent, D., & Ryan, N. (2013). Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL 2013, DSM-5). Pittsburgh, PA: Western Psychiatric Institute and Yale University.Google Scholar
Kessler, R. C., & Luedtke, A. (2021). Pragmatic precision psychiatry-a new direction for optimizing treatment selection. JAMA Psychiatry, 78(12), 1384–1390. https://doi.org/10.1001/jamapsychiatry.2021.2500.CrossRefGoogle ScholarPubMed
Larson, R. W., Moneta, G., Richards, M. H., & Wilson, S. (2002). Continuity, stability, and change in daily emotional experience across adolescence. Child Development, 73(4), 1151–1165.10.1111/1467-8624.00464CrossRefGoogle ScholarPubMed
Laurent, J., Catanzaro, S. J., Joiner, T. E. Jr., Rudolph, K. D., Potter, K. I., Lambert, S., … Gathright, T. (1999). A measure of positive and negative affect for children: Scale development and preliminary validation. Psychological Assessment, 11(3), 326–338. https://doi.org/10.1037/1040-3590.11.3.326.CrossRefGoogle Scholar
Perry, B. I., Osimo, E. F., Upthegrove, R., Mallikarjun, P. K., Yorke, J., Stochl, J., … Khandaker, G. M. (2021). Development and external validation of the psychosis metabolic risk calculator (PsyMetRiC): A cardiometabolic risk prediction algorithm for young people with psychosis. The Lancet Psychiatry, 8(7), 589–598. https://doi.org/10.1016/S2215-0366(21)00114-0.CrossRefGoogle ScholarPubMed
Radloff, L. S. (1977). The CES-D scale a self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401. https://doi.org/10.1177/014662167700100306.CrossRefGoogle Scholar
Rahimi-Eichi, H., Iii, G. C., Bustamante, C. M. V., Onnela, J.-P., Baker, J. T., & Buckner, R. L. (2021). Open-source longitudinal sleep analysis from accelerometer data (DPSleep): Algorithm development and validation. JMIR MHealth and UHealth, 9(10), e29849. https://doi.org/10.2196/29849.CrossRefGoogle ScholarPubMed
Ren, B., Patil, P., Dominici, F., Parmigiani, G., & Trippa, L. (2021). Cross-study learning for generalist and specialist predictions. ArXiv:2007.12807 [Math, Stat]. Retrieved from http://arxiv.org/abs/2007.12807.Google Scholar
Rideout, V., & Robb, M. B. (2019). The common sense census: Media use by tweens and teens, 2019 | Common Sense Media. Retrieved August 19, 2020, from https://www.commonsensemedia.org/research/the-common-sense-census-media-use-by-tweens-and-teens-2019.Google Scholar
Sequeira, L., Perrotta, S., LaGrassa, J., Merikangas, K., Kreindler, D., Kundur, D., … Strauss, J. (2020). Mobile and wearable technology for monitoring depressive symptoms in children and adolescents: A scoping review. Journal of Affective Disorders, 265, 314–324.10.1016/j.jad.2019.11.156CrossRefGoogle ScholarPubMed
Shah, R. V., Grennan, G., Zafar-Khan, M., Alim, F., Dey, S., Ramanathan, D., & Mishra, J. (2021). Personalized machine learning of depressed mood using wearables. Translational Psychiatry, 11(1), 1–18. https://doi.org/10.1038/s41398-021-01445-0.CrossRefGoogle ScholarPubMed
Snaith, R. P., Hamilton, M., Morley, S., Humayan, A., Hargreaves, D., & Trigwell, P. (1995). A scale for the assessment of hedonic tone the Snaith-Hamilton Pleasure Scale. The British Journal of Psychiatry, 167(1), 99–103. https://doi.org/10.1192/bjp.167.1.99.CrossRefGoogle ScholarPubMed
Soto, C. J. (2016). The little six personality dimensions from early childhood to early adulthood: Mean-level age and gender differences in parents’ reports. Journal of Personality, 84(4), 409–422.10.1111/jopy.12168CrossRefGoogle ScholarPubMed
Soto, C. J., John, O. P., Gosling, S. D., & Potter, J. (2011). Age differences in personality traits from 10 to 65: Big five domains and facets in a large cross-sectional sample. Journal of Personality and Social Psychology, 100(2), 330.10.1037/a0021717CrossRefGoogle Scholar
Squeglia, L. M., Ball, T. M., Jacobus, J., Brumback, T., McKenna, B. S., Nguyen-Louie, T. T., … Tapert, S. F. (2016). Neural predictors of initiating alcohol use during adolescence. American Journal of Psychiatry, 174(2), 172–185. https://doi.org/10.1176/appi.ajp.2016.15121587.CrossRefGoogle ScholarPubMed
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., … Kattan, M. W. (2010). Assessing the performance of prediction models: A framework for some traditional and novel measures. Epidemiology (Cambridge, Mass.), 21(1), 128–138. https://doi.org/10.1097/EDE.0b013e3181c30fb2.CrossRefGoogle ScholarPubMed
Torous, J., Kiang, M. V., Lorme, J., & Onnela, J.-P. (2016). New tools for new research in psychiatry: A scalable and customizable platform to empower data-driven smartphone research. JMIR Mental Health, 3(2), e5165. https://doi.org/10.2196/mental.5165.CrossRefGoogle ScholarPubMed
Trøstheim, M., Eikemo, M., Meir, R., Hansen, I., Paul, E., Kroll, S. L., … Leknes, S. (2020). Assessment of anhedonia in adults with and without mental illness: A systematic review and meta-analysis. JAMA Network Open, 3(8), e2013233. https://doi.org/10.1001/jamanetworkopen.2020.13233.CrossRefGoogle ScholarPubMed
Unal, I. (2017). Defining an optimal cut-point value in ROC analysis: An alternative approach. Computational and Mathematical Methods in Medicine, 6, 1–14. https://doi.org/10.1155/2017/3762651.CrossRefGoogle Scholar
van Buuren, S., & Groothuis-Oudshoorn, K. (2011). Mice: Multivariate imputation by chained equations in R. Journal of Statistical Software, 45, 1–67. https://doi.org/10.18637/jss.v045.i03.Google Scholar
Vickers, A. J., Calster, B. V., & Steyerberg, E. W. (2016). Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ, 352, i6. https://doi.org/10.1136/bmj.i6.CrossRefGoogle Scholar
Vickers, A. J., van Calster, B., & Steyerberg, E. W. (2019). A simple, step-by-step guide to interpreting decision curve analysis. Diagnostic and Prognostic Research, 3(1), 18. https://doi.org/10.1186/s41512-019-0064-7.CrossRefGoogle ScholarPubMed
Vinograd, M., Williams, A., Sun, M., Bobova, L., Wolitzky-Taylor, K. B., Vrshek-Schallhorn, S., … Craske, M. G. (2020). Neuroticism and interpretive bias as risk factors for anxiety and depression. Clinical Psychological Science, 8(4), 641–656.10.1177/2167702620906145CrossRefGoogle ScholarPubMed
Webb, C. A., & Cohen, Z. D. (2021). Progress towards clinically informative data-driven decision support tools in psychotherapy. The Lancet Digital Health, 3(4), e207–e208.10.1016/S2589-7500(21)00042-XCrossRefGoogle ScholarPubMed
Webb, C. A., Israel, E. S., Belleau, E., Appleman, L., Forbes, E. E., & Pizzagalli, D. A. (2021). Mind-wandering in adolescents predicts worse affect and is linked to aberrant default mode network–salience network connectivity. Journal of the American Academy of Child & Adolescent Psychiatry, 60(3), 377–387. https://doi.org/10.1016/j.jaac.2020.03.010.CrossRefGoogle ScholarPubMed
Wright, A. G., & Woods, W. C. (2020). Personalized models of psychopathology. Annual Review of Clinical Psychology, 16, 49–74.10.1146/annurev-clinpsy-102419-125032CrossRefGoogle ScholarPubMed
Zinbarg, R. E., Mineka, S., Bobova, L., Craske, M. G., Vrshek-Schallhorn, S., Griffith, J. W., … Anand, D. (2016). Testing a hierarchical model of neuroticism and its cognitive facets: Latent structure and prospective prediction of first onsets of anxiety and unipolar mood disorders during 3 years in late adolescence. Clinical Psychological Science, 4(5), 805–824.10.1177/2167702615618162CrossRefGoogle Scholar
Supplementary material: File

Ren et al. supplementary material

Ren et al. supplementary material

Download Ren et al. supplementary material(File)
File 925 KB