Skip to main content Accessibility help
×
Home
Hostname: page-component-6c8bd87754-hvdfp Total loading time: 0.707 Render date: 2022-01-18T08:47:19.144Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Beyond “incentive hope”: Information sampling and learning under reward uncertainty

Published online by Cambridge University Press:  19 March 2019

Maya Zhe Wang
Affiliation:
Department of Brain and Cognitive Sciences and Center for Visual Sciences, University of Rochester, Rochester, NY 14627 Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455. wang8200@umn.edubenhayden@gmail.comhaydenlab.com
Benjamin Y. Hayden
Affiliation:
Department of Neuroscience and Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, MN 55455. wang8200@umn.edubenhayden@gmail.comhaydenlab.com

Abstract

Information seeking, especially when motivated by strategic learning and intrinsic curiosity, could render the new mechanism “incentive hope” proposed by Anselme & Güntürkün sufficient, but not necessary to explain how reward uncertainty promotes reward seeking and consumption. Naturalistic and foraging-like tasks can help parse motivational processes that bridge learning and foraging behaviors and identify their neural underpinnings.

Type
Open Peer Commentary
Copyright
Copyright © Cambridge University Press 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abe, H. & Lee, D. (2011) Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70(4):731–41. http://doi.org/10.1016/j.neuron.2011.03.026.CrossRefGoogle ScholarPubMed
Bateson, M. & Kacelnik, A. (1997) Starlings’ preference for predictable and unpredictable delays to food. Animal Behaviour 53(6):1129–42. https://doi.org/10.1006/anbe.1996.0388.CrossRefGoogle Scholar
Behrens, T. E. J., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. S. (2007) Learning the value of information in an uncertain world. Nature Neuroscience 10(9):1214–21. http://doi.org/10.1038/nn1954.CrossRefGoogle Scholar
Blanchard, T. C., Hayden, B. Y. & Bromberg-Martin, E. S. (2015a) Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity. Neuron 85(3):602–14. http://doi.org/10.1016/j.neuron.2014.12.050.CrossRefGoogle Scholar
Blanchard, T. C., Strait, C. E. & Hayden, B. Y. (2015b) Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision. Journal of Neurophysiology 114:2439–49. http://doi:10.1152/jn.00711.2015.CrossRefGoogle Scholar
Blanchard, T. C., Wilke, A. & Hayden, B. Y. (2014) Hot-hand bias in rhesus monkeys. Journal of Experimental Psychology: Animal Learning and Cognition 40(3):280–86. http://doi.org/10.1037/xan0000033.Google ScholarPubMed
Boorman, E. D., Behrens, T. E. & Rushworth, M. F. (2011) Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex. PLoS Biology 9(6):e100109313. http://doi.org/10.1371/journal.pbio.1001093.CrossRefGoogle Scholar
Bromberg-Martin, E. S., Matsumoto, M. & Hikosaka, O. (2010) Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron 68(5):815–34. http://doi.org/10.1016/j.neuron.2010.11.022.CrossRefGoogle ScholarPubMed
Calhoun, A. J. & Hayden, B. Y. (2015) The foraging brain. Current Opinion in Behavioral Science 5:2431. http://dx.doi.org/10.1016/j.cobeha.2015.07.003.CrossRefGoogle Scholar
Chang, C. Y., Gardner, M., Di Tillio, M. G. & Schoenbaum, G. (2017) Optogenetic blockade of dopamine transients prevents learning induced by changes in reward features. Curbio 27(22):3480–86.Google ScholarPubMed
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. (2006) Cortical substrates for exploratory decisions in humans. Nature 441(7095):876–79. http://doi.org/10.1038/nature04766.CrossRefGoogle ScholarPubMed
De Petrillo, F., Ventricelli, M., Ponsi, G. & Addessi, E. (2015) Do tufted capuchin monkeys play the odds? Flexible risk preferences in Sapajus spp. Animal Cognition 18(1):119–30.CrossRefGoogle ScholarPubMed
Gershman, S. J. & Schoenbaum, G. (2017) Rethinking dopamine prediction errors. bioRxiv 239731 preprint. doi: https://doi.org/10.1101/239731.Google Scholar
Hayden, B. Y. (2018) Economic choice: The foraging perspective. Current Opinion in Behavioral Sciences 24:16.CrossRefGoogle Scholar
Hayden, B. Y., Pearson, J. M. & Platt, M. L. (2009) Fictive reward signals in the anterior cingulate cortex. Science 324(5929):948–50. http://doi.org/10.1126/science.1168488.CrossRefGoogle ScholarPubMed
Kacelnik, A. & Bateson, M. (1997) Risk-sensitivity: Crossroads for theories of decision-making. Trends in Cognitive Sciences 1(8):304309. http://doi.org/10.1016/S1364-6613(97)01093-0.CrossRefGoogle ScholarPubMed
Kaplan, R., Schuck, N. W. & Doeller, C. F. (2017) The role of mental maps in decision-making. Trends in Neurosciences 40(5):14. http://doi.org/10.1016/j.tins.2017.03.002.CrossRefGoogle ScholarPubMed
Kiani, R. & Shadlen, M. N. (2009) Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324(5928):759–64. http://doi.org/10.1126/science.1169405.CrossRefGoogle ScholarPubMed
Kidd, C. & Hayden, B. Y. (2015) The psychology and neuroscience of curiosity. Neuron 88(3):449–60. http://doi.org/10.1016/j.neuron.2015.09.010.CrossRefGoogle Scholar
Killian, N. J., Jutras, M. J. & Buffalo, E. A. (2012) A map of visual space in the primate entorhinal cortex. Nature 491(7426):761.CrossRefGoogle ScholarPubMed
Kornell, N., Son, L. K. & Terrace, H. S. (2007) Transfer of metacognitive skills and hint seeking in monkeys. Psychological Science 18(1):6471.CrossRefGoogle ScholarPubMed
Langdon, A. J., Sharpe, M. J., Schoenbaum, G. & Niv, Y. (2018) Model-based predictions for dopamine. Current Opinion in Neurobiology 49:17.CrossRefGoogle ScholarPubMed
Minderer, M. & Harvey, C. D. (2016) Neuroscience: Virtual reality explored. Nature 533(7603):324–24. http://doi.org/10.1038/nature17899.CrossRefGoogle ScholarPubMed
Mobbs, D., Trimmer, P. C., Blumstein, D. T. & Dayan, P. (2018) Foraging for foundations in decision neuroscience: Insights from ethology. Neuroscience 13(18):19.Google Scholar
Musall, S., Kaufman, M. T., Gluf, S. & Churchland, A. K. (2018) Movement-related activity dominates cortex during sensory-guided decision making. bioRxiv preprint. https://doi.org/10.1101/308288.Google Scholar
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J. & Rushworth, M. F. S. (2010) Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proceedings of the National Academy of Sciences USA 107(47):20547–52. http://doi.org/10.1073/pnas.1012246107.CrossRefGoogle ScholarPubMed
Pearson, J. M., Hayden, B. Y., Raghavachari, S. & Platt, M. L. (2009) Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Current Biology 19(18):1532–37. http://doi.org/10.1016/j.cub.2009.07.048.CrossRefGoogle Scholar
Pitkow, X. & Angelaki, D. (2017) How the brain might work: Statistics flowing in redundant population codes. arXiv:1702.03492Google Scholar
Pouget, A., Drugowitsch, J. & Kepecs, A. (2016) Confidence and certainty: Distinct probabilistic quantities for different goals. Nature Neuroscience 19(3):366–74. http://doi.org/10.1038/nn.4240.CrossRefGoogle ScholarPubMed
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E. & Behrens, T. E. (2011) Frontal cortex and reward-guided learning and decision-making. Neuron 70(6):1054–69. http://doi.org/10.1016/j.neuron.2011.05.014.CrossRefGoogle ScholarPubMed
Sadacca, B. F., Jones, J. L. & Schoenbaum, G. (2016) Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework. eLife 016;5:e13665. http://doi.org/10.7554/eLife.13665.Google Scholar
Schonberg, T., Fox, C. R. & Poldrack, R. A. (2011) Mind the gap: Bridging economic and naturalistic risk-taking with cognitive neuroscience. Trends in Cognitive Sciences 15(1):1119. http://doi.org/10.1016/j.tics.2010.10.002.CrossRefGoogle ScholarPubMed
Stephens, D. W. & Krebs, J. R. (1986) Foraging theory. Princeton University Press.Google Scholar
Strait, C. E., Sleezer, B. J., Blanchard, T. C., Azab, H., Castagno, M. D. & Hayden, B. Y. (2016) Neuronal selectivity for spatial position of offers and choices in five reward regions. Journal of Neurophysiology 115:10981111.CrossRefGoogle ScholarPubMed
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M. & Schoenbaum, G. (2017) Dopamine neurons respond to errors in the prediction of sensory features of expected rewards. Neuron 95(6):13951405.CrossRefGoogle ScholarPubMed
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H. & Rushworth, M. F. S. (2010) Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65(6):927–39. http://doi.org/10.1016/j.neuron.2010.02.027.CrossRefGoogle ScholarPubMed
Wang, M. Z. & Hayden, B. (2018) Monkeys are curious about counterfactual outcomes. bioRxiv preprint. http://doi.org/10.1101/291708.CrossRefGoogle Scholar
Wang, M. Z. & Hayden, B. Y. (2017) Reactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices. Nature Communications 8:15821. http://doi.org/10.1038/ncomms15821.CrossRefGoogle ScholarPubMed
Wirth, S., Baraduc, P., Planté, A., Pinède, S., & Duhamel, J.-R. (2017) Gaze-informed, task-situated representation of space in primate hippocampus during virtual navigation. PLoS Biology 15(2):e2001045. http://doi.org/10.1371/journal.pbio.2001045.CrossRefGoogle ScholarPubMed

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Beyond “incentive hope”: Information sampling and learning under reward uncertainty
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Beyond “incentive hope”: Information sampling and learning under reward uncertainty
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Beyond “incentive hope”: Information sampling and learning under reward uncertainty
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *