No CrossRef data available.
Article contents
Factual and counterfactual learning in major adolescent depressive disorder, evidence from an instrumental learning study
Published online by Cambridge University Press: 10 May 2023
Abstract
Background
The incidence of adolescent depressive disorder is globally skyrocketing in recent decades, albeit the causes and the decision deficits depression incurs has yet to be well-examined. With an instrumental learning task, the aim of the current study is to investigate the extent to which learning behavior deviates from that observed in healthy adolescent controls and track the underlying mechanistic channel for such a deviation.
Methods
We recruited a group of adolescents with major depression and age-matched healthy control subjects to carry out the learning task with either gain or loss outcome and applied a reinforcement learning model that dissociates valence (positive v. negative) of reward prediction error and selection (chosen v. unchosen).
Results
The results demonstrated that adolescent depressive patients performed significantly less well than the control group. Learning rates suggested that the optimistic bias that overall characterizes healthy adolescent subjects was absent for the depressive adolescent patients. Moreover, depressed adolescents exhibited an increased pessimistic bias for the counterfactual outcome. Lastly, individual difference analysis suggested that these observed biases, which significantly deviated from that observed in normal controls, were linked with the severity of depressive symoptoms as measured by HAMD scores.
Conclusions
By leveraging an incentivized instrumental learning task with computational modeling within a reinforcement learning framework, the current study reveals a mechanistic decision-making deficit in adolescent depressive disorder. These findings, which have implications for the identification of behavioral markers in depression, could support the clinical evaluation, including both diagnosis and prognosis of this disorder.
- Type
- Original Article
- Information
- Copyright
- Copyright © The Author(s), 2023. Published by Cambridge University Press
Footnotes
*
These authors contributed equally to this work.
References
Auerbach, R. P., Pagliaccio, D., & Pizzagalli, D. A. (2019). Toward an improved understanding of anhedonia. JAMA Psychiatry, 76(6), 571–573. doi:10.1001/jamapsychiatry.2018.4600.CrossRefGoogle ScholarPubMed
Bakic, J., Pourtois, G., Jepma, M., Duprat, R., De Raedt, R., & Baeken, C. (2017). Spared internal but impaired external reward prediction error signals in major depressive disorder during reinforcement learning. Depression and Anxiety, 34(1), 89–96. doi:10.1002/da.22576.CrossRefGoogle ScholarPubMed
Bao, H. W. S. (2022). bruceR: Broadly useful convenien and efficient R functions. R package version 0.8.x. Retrieved from https://CRAN.R-project.org/package=bruceR.Google Scholar
Bavard, S., Rustichini, A., & Palminteri, S. (2021). Two sides of the same coin: Beneficial and detrimental consequences of range adaptation in human reinforcement learning. Science Advances, 7(14), eabe0340. doi:10.1126/sciadv.abe0340.CrossRefGoogle ScholarPubMed
Berwian, I. M., Wenzel, J. G., Collins, A. G., Seifritz, E., Stephan, K. E., Walter, H., & Huys, Q. J. (2020). Computational mechanisms of effort and reward decisions in patients with depression and their association with relapse after antidepressant discontinuation. JAMA Psychiatry, 77(5), 513–522. doi:10.1001/jamapsychiatry.2019.4971.CrossRefGoogle ScholarPubMed
Bishop, S. J., & Gagne, C. (2018). Anxiety, depression, and decision making: A computational perspective. Annual Review of Neuroscience, 41, 371–388. doi:10.1146/annurev-neuro-080317-062007.CrossRefGoogle ScholarPubMed
Brolsma, S. C., Vassena, E., Vrijsen, J. N., Sescousse, G., Collard, R. M., van Eijndhoven, P. F., & …Cools, R. (2021). Negative learning bias in depression revisited: Enhanced neural response to surprising reward across psychiatric disorders. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 6(3), 280–289. doi:10.1016/j.bpsc.2020.08.011.Google ScholarPubMed
Bromberg-Martin, E. S., & Sharot, T. (2020). The value of beliefs. Neuron, 106(4), 561–565. doi:10.1016/j.neuron.2020.05.001.CrossRefGoogle ScholarPubMed
Broomhall, A. G., Phillips, W. J., Hine, D. W., & Loi, N. M. (2017). Upward counterfactual thinking and depression: A meta-analysis. Clinical Psychology Review, 55, 56–73. doi:10.1016/j.cpr.2017.04.010.CrossRefGoogle ScholarPubMed
Chambon, V., Thero, H., Vidal, M., Vandendriessche, H., Haggard, P., & Palminteri, S. (2020). Information about action outcomes differentially affects learning from self-determined versus imposed choices. Nature Human Behaviour, 4(10), 1067–1079. doi:10.1038/s41562-020-0919-5.CrossRefGoogle ScholarPubMed
Chase, H. W., Frank, M. J., Michael, A., Bullmore, E. T., Sahakian, B. J., & Robbins, T. W. (2010). Approach and avoidance learning in patients with major depression and healthy controls: Relation to anhedonia. Psychological Medicine, 40(3), 433–440. doi:10.1017/S0033291709990468.CrossRefGoogle ScholarPubMed
Clayborne, Z. M., Varin, M., & Colman, I. (2019). Systematic review and meta-analysis: Adolescent depression and long-term psychosocial outcomes. Journal of the American Academy of Child & Adolescent Psychiatry, 58(1), 72–79. doi:10.1016/j.jaac.2018.07.896.CrossRefGoogle ScholarPubMed
Fontanesi, L., Gluth, S., Spektor, M. S., & Rieskamp, J. (2019). A reinforcement learning diffusion decision model for value-based decisions. Psychonomic Bulletin & Review, 26(4), 1099–1121. doi:10.3758/s13423-018-1554-2.CrossRefGoogle ScholarPubMed
Frank, M. J., Seeberger, L. C., & O'reilly, R. C. (2004). By carrot or by stick: Cognitive reinforcement learning in Parkinsonism. Science (New York, N.Y.), 306(5703), 1940–1943. doi:10.1126/science.1102941.CrossRefGoogle ScholarPubMed
Gaure, S. (2013). lfe: Linear group fixed effects. The R Journal, 5(2), 104–116. doi:10.32614/RJ-2013-031.CrossRefGoogle Scholar
Gillan, C. M., Otto, A. R., Phelps, E. A., & Daw, N. D. (2015). Model-based learning protects against forming habits. Cognitive, Affective, & Behavioral Neuroscience, 15(3), 523–536. doi:10.3758/s13415-015-0347-6.CrossRefGoogle ScholarPubMed
Harrell, F. E. Jr. (2021). rms: Regression Modeling Strategies. R package version 6.2-0. Retrieved from https://CRAN.R-project.org/package=rms.Google Scholar
Hartzmark, S. M., Hirshman, S. D., & Imas, A. (2021). Ownership, learning, and beliefs. The Quarterly Journal of Economics, 136(3), 1665–1717. doi:10.1093/qje/qjab010.CrossRefGoogle Scholar
Kappes, A., Harvey, A. H., Lohrenz, T., Montague, P. R., & Sharot, T. (2019). Confirmation bias in the utilization of others’ opinion strength. Nature Neuroscience, 23(1), 130–137. doi:10.1038/s41593-019-0549-2.CrossRefGoogle ScholarPubMed
Korn, C. W., Sharot, T., Walter, H., Heekeren, H. R., & Dolan, R. J. (2014). Depression is related to an absence of optimistically biased belief updating about future life events. Psychological Medicine, 44(3), 579–592. doi:10.1017/S0033291713001074.CrossRefGoogle Scholar
Kraines, M. A., Krug, C. P., & Wells, T. T. (2017). Decision justification theory in depression: Regret and self-blame. Cognitive Therapy and Research, 41(4), 556–561. doi:10.1007/s10608-017-9836-y.CrossRefGoogle Scholar
Kube, T., Schwarting, R., Rozenkrantz, L., Glombiewski, J. A., & Rief, W. (2020). Distorted cognitive processes in major depression: A predictive processing perspective. Biological Psychiatry, 87(5), 388–398. doi:10.1016/j.biopsych.2019.07.017.CrossRefGoogle ScholarPubMed
Kumar, P., Goer, F., Murray, L., Dillon, D. G., Beltzer, M. L., Cohen, A. L., … Pizzagalli, D. A. (2018). Impaired reward prediction error encoding and striatal-midbrain connectivity in depression. Neuropsychopharmacology, 43(7), 1581–1588. doi:10.1038/s41386-018-0032-x.CrossRefGoogle ScholarPubMed
Lefebvre, G., Lebreton, M., Meyniel, F., Bourgeois-Gironde, S., & Palminteri, S. (2017). Behavioural and neural characterization of optimistic reinforcement learning. Nature Human Behaviour, 1(4), 1–9. doi:10.1038/s41562-017-0067.CrossRefGoogle Scholar
Lefebvre, G., Summerfield, C., & Bogacz, R. (2021). A normative account of confirmation bias during reinforcement learning. Neural Computation, 34(2), 1–31. doi:10.1162/neco_a_01455.Google Scholar
Lu, W. (2019). Adolescent depression: National trends, risk factors, and healthcare disparities. American Journal of Health Behavior, 43(1), 181–194. doi:10.5993/AJHB.43.1.15.CrossRefGoogle ScholarPubMed
Ma, Y., Li, S., Wang, C., Liu, Y., Li, W., Yan, X., … Han, S. (2016). Distinct oxytocin effects on belief updating in response to desirable and undesirable feedback. Proceedings of the National Academy of Sciences, 113(33), 9256–9261. doi:10.1073/pnas.1604285113.CrossRefGoogle ScholarPubMed
McFadden, D. (1973). Conditional logit analysis of qualitative choice behavior. In Zarembka, P. (Ed.), Frontiers in econometrics (pp. 105–142). New York: Academic Press.Google Scholar
Miletić, S., Boag, R. J., Trutti, A. C., Stevenson, N., Forstmann, B. U., & Heathcote, A. (2021). A new model of decision processing in instrumental learning tasks. Elife, 10, e63055. doi:10.7554/eLife.63055.CrossRefGoogle ScholarPubMed
Miller, L., & Campo, J. V. (2021). Depression in adolescents. New England Journal of Medicine, 385(5), 445–449. doi:10.1056/NEJMra2033475.CrossRefGoogle ScholarPubMed
Montague, P. R., Dolan, R. J., Friston, K. J., & Dayan, P. (2012). Computational psychiatry. Trends in Cognitive Sciences, 16(1), 72–80. doi:10.1016/j.tics.2011.11.018.CrossRefGoogle ScholarPubMed
Mukherjee, D., Filipowicz, A. L. S., Vo, K., Satterthwaite, T. D., & Kable, J. W. (2020). Reward and punishment reversal-learning in major depressive disorder. Journal of Abnormal Psychology, 129(8), 810–823. doi:10.1037/abn0000641.CrossRefGoogle ScholarPubMed
Mullen, K., Ardia, D., Gil, D. L., Windover, D., & Cline, J. (2011). DEoptim: An R package for global optimization by differential evolution. Journal of Statistical Software, 40(6), 1–26. doi:10.18637/jss.v040.i06.CrossRefGoogle Scholar
Ng, T. H., Alloy, L. B., & Smith, D. V. (2019). Meta-analysis of reward processing in major depressive disorder reveals distinct abnormalities within the reward circuit. Translational Psychiatry, 9(1), 293. doi:10.1038/s41398-019-0644-x.CrossRefGoogle ScholarPubMed
Nielson, D. M., Keren, H., O'Callaghan, G., Jackson, S. M., Douka, I., Vidal-Ribas, P., … Stringaris, A. (2021). Great expectations: A critical review of and suggestions for the study of reward processing as a cause and predictor of depression. Biological Psychiatry, 89(2), 134–143. doi:10.1016/j.biopsych.2020.06.012.CrossRefGoogle Scholar
Niv, Y., Edlund, J. A., Dayan, P., & O'Doherty, J. P. (2012). Neural prediction errors reveal a risk-sensitive reinforcement-learning process in the human brain. Journal of Neuroscience, 32(2), 551–562. doi:10.1523/JNEUROSCI.5498-10.2012.CrossRefGoogle ScholarPubMed
Palminteri, S., Kilford, E. J., Coricelli, G., & Blakemore, S. J. (2016). The computational development of reinforcement learning during adolescence. PLoS Computational Biology, 12(6), e1004953. doi:10.1371/journal.pcbi.1004953.CrossRefGoogle ScholarPubMed
Palminteri, S., Lefebvre, G., Kilford, E. J., & Blakemore, S. J. (2017). Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing. PLoS Computational Biology, 13(8), e1005684. doi:10.1371/journal.pcbi.1005684.CrossRefGoogle ScholarPubMed
Paus, T., Keshavan, M., & Giedd, J. N. (2008). Why do many psychiatric disorders emerge during adolescence? Nature Reviews Neuroscience, 9(12), 947–957. doi:10.1038/nrn2513.CrossRefGoogle ScholarPubMed
Pedersen, M. L., & Frank, M. J. (2020). Simultaneous hierarchical Bayesian parameter estimation for reinforcement learning and drift diffusion models: A tutorial and links to neural data. Computational Brain & Behavior, 3(4), 458–471. doi:10.1007/s42113-020-00084-w.CrossRefGoogle ScholarPubMed
Pedersen, M. L., Frank, M. J., & Biele, G. (2017). The drift diffusion model as the choice rule in reinforcement learning. Psychonomic Bulletin & Review, 24(4), 1234–1251. doi:10.3758/s13423-016-1199-y.CrossRefGoogle ScholarPubMed
Pizzagalli, D. A., Iosifescu, D., Hallett, L. A., Ratner, K. G., & Fava, M. (2008). Reduced hedonic capacity in major depressive disorder: Evidence from a probabilistic reward task. Journal of Psychiatric Research, 43(1), 76–87. doi:10.1016/j.jpsychires.2008.03.001.CrossRefGoogle ScholarPubMed
Raio, C. M., Hartley, C. A., Orederu, T. A., Li, J., & Phelps, E. A. (2017). Stress attenuates the flexible updating of aversive value. Proceedings of the National Academy of Sciences, 114(42), 11241–11246. doi:10.1073/pnas.1702565114.CrossRefGoogle ScholarPubMed
Raven, J. C., & Court, J. H. (1998). Raven's progressive matrices and vocabulary scales (pp. 223–237). Oxford: Oxford Pyschologists Press.Google Scholar
Roese, N. J., Epstude, K. A. I., Fessel, F., Morrison, M., Smallman, R., Summerville, A., … Segerstrom, S. (2009). Repetitive regret, depression, and anxiety: Findings from a nationally representative survey. Journal of Social and Clinical Psychology, 28(6), 671–688. doi:10.1521/jscp.2009.28.6.671.CrossRefGoogle Scholar
Santomauro, D. F., Herrera, A. M. M., Shadid, J., Zheng, P., Ashbaugh, C., Pigott, D. M., … Ferrari, A. J. (2021). Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. The Lancet, 398(10312), 1700–1712. doi:10.1016/s0140-6736(21)02143-7.CrossRefGoogle Scholar
Seidel, E. M., Satterthwaite, T. D., Eickhoff, S. B., Schneider, F., Gur, R. C., Wolf, D. H., … Derntl, B. (2012). Neural correlates of depressive realism—An fMRI study on causal attribution in depression. Journal of Affective Disorders, 138(3), 268–276. doi:10.1016/j.jad.2012.01.041.CrossRefGoogle Scholar
Sharot, T. (2011). The optimism bias. Current Biology, 21(23), R941–R945. doi:10.1016/j.cub.2011.10.030.CrossRefGoogle ScholarPubMed
Sharot, T., & Garrett, N. (2016). Forming beliefs: Why valence matters. Trends in Cognitive Sciences, 20(1), 25–33. doi:10.1016/j.tics.2015.11.002.CrossRefGoogle ScholarPubMed
Sharot, T., Riccardi, A. M., Raio, C. M., & Phelps, E. A. (2007). Neural mechanisms mediating optimism bias. Nature, 450(7166), 102–105. doi:10.1016/j.cub.2011.10.030.CrossRefGoogle ScholarPubMed
Sharot, T., Velasquez, C. M., & Dolan, R. J. (2010). Do decisions shape preference? Evidence from blind choice. Psychological Science, 21(9), 1231–1235. doi:10.1177/0956797610379235.CrossRefGoogle ScholarPubMed
Stevanovic, D., Jancic, J., & Lakic, A. (2011). The impact of depression and anxiety disorder symptoms on the health-related quality of life of children and adolescents with epilepsy. Epilepsia, 52(8), e75–e78. doi:10.1111/j.1528-1167.2011.03133.x.CrossRefGoogle ScholarPubMed
Stringaris, A., Vidal-Ribas Belil, P., Artiges, E., Lemaitre, H., Gollier-Briant, F., & Wolke, S., … IMAGEN Consortium. (2015). The brain's response to reward anticipation and depression in adolescence: Dimensionality, specificity, and longitudinal predictions in a community-based sample. American Journal Psychiatry, 172(12), 1215–1223. doi:10.1176/appi.ajp.2015.14101298.CrossRefGoogle Scholar
Sugawara, M., & Katahira, K. (2021). Dissociation between asymmetric value updating and perseverance in human reinforcement learning. Scientific Reports, 11(1), 3574. doi:10.1038/s41598-020-80593-7.CrossRefGoogle ScholarPubMed
Sutton, R. S., & Barto, A. G. (2018). Reinforcement learning: An introduction. Boston: MIT press.Google Scholar
Tarantola, T. O., Folke, T., Boldt, A., Perez, O. D., & De Martino, B. (2021). Confirmation bias optimizes reward learning. bioRxiv, 2021-02. doi:10.1101/2021.02.27.433214.Google Scholar
Twenge, J. M., Cooper, A. B., Joiner, T. E., Duffy, M. E., & Binau, S. G. (2019). Age, period, and cohort trends in mood disorder indicators and suicide-related outcomes in a nationally representative dataset, 2005–2017. Journal of Abnormal Psychology, 128(3), 185–199. doi:10.1287/mnsc.2017.2931.CrossRefGoogle Scholar
Webb, R. (2019). The (neural) dynamics of stochastic choice. Management Science, 65(1), 230–255. doi:10.1287/mnsc.2017.2931.CrossRefGoogle Scholar
Wiehler, A., Chakroun, K., & Peters, J. (2021). Attenuated directed exploration during reinforcement learning in gambling disorder. Journal of Neuroscience, 41(11), 2512–2522. doi:10.1523/JNEUROSCI.1607-20.2021.CrossRefGoogle ScholarPubMed
Zhang, M., & He, Y. (2015). Psychiatric rating scale manual. Changsha: Hunan Science and Technology Press.Google Scholar
Shen et al. supplementary material
Shen et al. supplementary material
File
18.7 MB