Skip to main content Accessibility help
×
Home

Beyond “incentive hope”: Information sampling and learning under reward uncertainty

  • Maya Zhe Wang (a1) (a2) and Benjamin Y. Hayden (a2)

Abstract

Information seeking, especially when motivated by strategic learning and intrinsic curiosity, could render the new mechanism “incentive hope” proposed by Anselme & Güntürkün sufficient, but not necessary to explain how reward uncertainty promotes reward seeking and consumption. Naturalistic and foraging-like tasks can help parse motivational processes that bridge learning and foraging behaviors and identify their neural underpinnings.

Copyright

References

Hide All
Abe, H. & Lee, D. (2011) Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron 70(4):731–41. http://doi.org/10.1016/j.neuron.2011.03.026.
Bateson, M. & Kacelnik, A. (1997) Starlings’ preference for predictable and unpredictable delays to food. Animal Behaviour 53(6):1129–42. https://doi.org/10.1006/anbe.1996.0388.
Behrens, T. E. J., Woolrich, M. W., Walton, M. E. & Rushworth, M. F. S. (2007) Learning the value of information in an uncertain world. Nature Neuroscience 10(9):1214–21. http://doi.org/10.1038/nn1954.
Blanchard, T. C., Hayden, B. Y. & Bromberg-Martin, E. S. (2015a) Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity. Neuron 85(3):602–14. http://doi.org/10.1016/j.neuron.2014.12.050.
Blanchard, T. C., Strait, C. E. & Hayden, B. Y. (2015b) Ramping ensemble activity in dorsal anterior cingulate neurons during persistent commitment to a decision. Journal of Neurophysiology 114:2439–49. http://doi:10.1152/jn.00711.2015.
Blanchard, T. C., Wilke, A. & Hayden, B. Y. (2014) Hot-hand bias in rhesus monkeys. Journal of Experimental Psychology: Animal Learning and Cognition 40(3):280–86. http://doi.org/10.1037/xan0000033.
Boorman, E. D., Behrens, T. E. & Rushworth, M. F. (2011) Counterfactual choice and learning in a neural network centered on human lateral frontopolar cortex. PLoS Biology 9(6):e100109313. http://doi.org/10.1371/journal.pbio.1001093.
Bromberg-Martin, E. S., Matsumoto, M. & Hikosaka, O. (2010) Dopamine in motivational control: Rewarding, aversive, and alerting. Neuron 68(5):815–34. http://doi.org/10.1016/j.neuron.2010.11.022.
Calhoun, A. J. & Hayden, B. Y. (2015) The foraging brain. Current Opinion in Behavioral Science 5:2431. http://dx.doi.org/10.1016/j.cobeha.2015.07.003.
Chang, C. Y., Gardner, M., Di Tillio, M. G. & Schoenbaum, G. (2017) Optogenetic blockade of dopamine transients prevents learning induced by changes in reward features. Curbio 27(22):3480–86.
Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. (2006) Cortical substrates for exploratory decisions in humans. Nature 441(7095):876–79. http://doi.org/10.1038/nature04766.
De Petrillo, F., Ventricelli, M., Ponsi, G. & Addessi, E. (2015) Do tufted capuchin monkeys play the odds? Flexible risk preferences in Sapajus spp. Animal Cognition 18(1):119–30.
Gershman, S. J. & Schoenbaum, G. (2017) Rethinking dopamine prediction errors. bioRxiv 239731 preprint. doi: https://doi.org/10.1101/239731.
Hayden, B. Y. (2018) Economic choice: The foraging perspective. Current Opinion in Behavioral Sciences 24:16.
Hayden, B. Y., Pearson, J. M. & Platt, M. L. (2009) Fictive reward signals in the anterior cingulate cortex. Science 324(5929):948–50. http://doi.org/10.1126/science.1168488.
Kacelnik, A. & Bateson, M. (1997) Risk-sensitivity: Crossroads for theories of decision-making. Trends in Cognitive Sciences 1(8):304309. http://doi.org/10.1016/S1364-6613(97)01093-0.
Kaplan, R., Schuck, N. W. & Doeller, C. F. (2017) The role of mental maps in decision-making. Trends in Neurosciences 40(5):14. http://doi.org/10.1016/j.tins.2017.03.002.
Kiani, R. & Shadlen, M. N. (2009) Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324(5928):759–64. http://doi.org/10.1126/science.1169405.
Kidd, C. & Hayden, B. Y. (2015) The psychology and neuroscience of curiosity. Neuron 88(3):449–60. http://doi.org/10.1016/j.neuron.2015.09.010.
Killian, N. J., Jutras, M. J. & Buffalo, E. A. (2012) A map of visual space in the primate entorhinal cortex. Nature 491(7426):761.
Kornell, N., Son, L. K. & Terrace, H. S. (2007) Transfer of metacognitive skills and hint seeking in monkeys. Psychological Science 18(1):6471.
Langdon, A. J., Sharpe, M. J., Schoenbaum, G. & Niv, Y. (2018) Model-based predictions for dopamine. Current Opinion in Neurobiology 49:17.
Minderer, M. & Harvey, C. D. (2016) Neuroscience: Virtual reality explored. Nature 533(7603):324–24. http://doi.org/10.1038/nature17899.
Mobbs, D., Trimmer, P. C., Blumstein, D. T. & Dayan, P. (2018) Foraging for foundations in decision neuroscience: Insights from ethology. Neuroscience 13(18):19.
Musall, S., Kaufman, M. T., Gluf, S. & Churchland, A. K. (2018) Movement-related activity dominates cortex during sensory-guided decision making. bioRxiv preprint. https://doi.org/10.1101/308288.
Noonan, M. P., Walton, M. E., Behrens, T. E. J., Sallet, J., Buckley, M. J. & Rushworth, M. F. S. (2010) Separate value comparison and learning mechanisms in macaque medial and lateral orbitofrontal cortex. Proceedings of the National Academy of Sciences USA 107(47):20547–52. http://doi.org/10.1073/pnas.1012246107.
Pearson, J. M., Hayden, B. Y., Raghavachari, S. & Platt, M. L. (2009) Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Current Biology 19(18):1532–37. http://doi.org/10.1016/j.cub.2009.07.048.
Pitkow, X. & Angelaki, D. (2017) How the brain might work: Statistics flowing in redundant population codes. arXiv:1702.03492
Pouget, A., Drugowitsch, J. & Kepecs, A. (2016) Confidence and certainty: Distinct probabilistic quantities for different goals. Nature Neuroscience 19(3):366–74. http://doi.org/10.1038/nn.4240.
Rushworth, M. F. S., Noonan, M. P., Boorman, E. D., Walton, M. E. & Behrens, T. E. (2011) Frontal cortex and reward-guided learning and decision-making. Neuron 70(6):1054–69. http://doi.org/10.1016/j.neuron.2011.05.014.
Sadacca, B. F., Jones, J. L. & Schoenbaum, G. (2016) Midbrain dopamine neurons compute inferred and cached value prediction errors in a common framework. eLife 016;5:e13665. http://doi.org/10.7554/eLife.13665.
Schonberg, T., Fox, C. R. & Poldrack, R. A. (2011) Mind the gap: Bridging economic and naturalistic risk-taking with cognitive neuroscience. Trends in Cognitive Sciences 15(1):1119. http://doi.org/10.1016/j.tics.2010.10.002.
Stephens, D. W. & Krebs, J. R. (1986) Foraging theory. Princeton University Press.
Strait, C. E., Sleezer, B. J., Blanchard, T. C., Azab, H., Castagno, M. D. & Hayden, B. Y. (2016) Neuronal selectivity for spatial position of offers and choices in five reward regions. Journal of Neurophysiology 115:10981111.
Takahashi, Y. K., Batchelor, H. M., Liu, B., Khanna, A., Morales, M. & Schoenbaum, G. (2017) Dopamine neurons respond to errors in the prediction of sensory features of expected rewards. Neuron 95(6):13951405.
Walton, M. E., Behrens, T. E. J., Buckley, M. J., Rudebeck, P. H. & Rushworth, M. F. S. (2010) Separable learning systems in the macaque brain and the role of orbitofrontal cortex in contingent learning. Neuron 65(6):927–39. http://doi.org/10.1016/j.neuron.2010.02.027.
Wang, M. Z. & Hayden, B. (2018) Monkeys are curious about counterfactual outcomes. bioRxiv preprint. http://doi.org/10.1101/291708.
Wang, M. Z. & Hayden, B. Y. (2017) Reactivation of associative structure specific outcome responses during prospective evaluation in reward-based choices. Nature Communications 8:15821. http://doi.org/10.1038/ncomms15821.
Wirth, S., Baraduc, P., Planté, A., Pinède, S., & Duhamel, J.-R. (2017) Gaze-informed, task-situated representation of space in primate hippocampus during virtual navigation. PLoS Biology 15(2):e2001045. http://doi.org/10.1371/journal.pbio.2001045.

Beyond “incentive hope”: Information sampling and learning under reward uncertainty

  • Maya Zhe Wang (a1) (a2) and Benjamin Y. Hayden (a2)

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed