No CrossRef data available.
Article contents
The reinforcement metalearner as a biologically plausible meta-learning framework
Published online by Cambridge University Press: 23 September 2024
Abstract
We argue that the type of meta-learning proposed by Binz et al. generates models with low interpretability and falsifiability that have limited usefulness for neuroscience research. An alternative approach to meta-learning based on hyperparameter optimization obviates these concerns and can generate empirically testable hypotheses of biological computations.
Information
- Type
- Open Peer Commentary
- Information
- Copyright
- Copyright © The Author(s), 2024. Published by Cambridge University Press
References
Castelvecchi, D. (2016). Can we open the black box of AI? Nature, 538, 20–23. https://doi.org/10.1038/538020aCrossRefGoogle ScholarPubMed
Daddaoua, N., Lopes, M., & Gottlieb, J. (2016). Intrinsically motivated oculomotor exploration guided by uncertainty reduction and conditioned reinforcement in non-human primates. Scientific Reports, 6(1), Article 1. https://doi.org/10.1038/srep20202CrossRefGoogle ScholarPubMed
Doya, K. (2002). Metalearning and neuromodulation. Neural Networks, 15(4–6), 495–506. https://doi.org/10.1016/s0893-6080(02)00044-8CrossRefGoogle ScholarPubMed
Horan, M., Daddaoua, N., & Gottlieb, J. (2019). Parietal neurons encode information sampling based on decision uncertainty. Nature Neuroscience, 22(8), 1327–1335. https://doi.org/10.1038/s41593-019-0440-1CrossRefGoogle ScholarPubMed
Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359–366. https://doi.org/10.1016/0893-6080(89)90020-8CrossRefGoogle Scholar
Marblestone, A. H., Wayne, G., & Kording, K. P. (2017). Understand the cogs to understand cognition. Behavioral and Brain Sciences, 40, e272. https://doi.org/10.1017/S0140525X17000218CrossRefGoogle ScholarPubMed
Shackman, A. J., Salomons, T. V., Slagter, H. A., Fox, A. S., Winter, J. J., & Davidson, R. J. (2011). The integration of negative affect, pain, and cognitive control in the cingulate cortex. Nature Reviews. Neuroscience, 12(3), 154–167. https://doi.org/10.1038/nrn2994CrossRefGoogle ScholarPubMed
Silvetti, M., Lasaponara, S., Daddaoua, N., Horan, M., & Gottlieb, J. (2023). A reinforcement meta-learning framework of executive function and information demand. Neural Networks, 157, 103–113. https://doi.org/10.1016/j.neunet.2022.10.004CrossRefGoogle ScholarPubMed
Silvetti, M., Seurinck, R., van Bochove, M., & Verguts, T. (2013). The influence of the noradrenergic system on optimal control of neural plasticity. Frontiers in Behavioral Neuroscience, 7, 160. https://www.frontiersin.org/articles/10.3389/fnbeh.2013.00160CrossRefGoogle ScholarPubMed
Silvetti, M., Vassena, E., Abrahamse, E., & Verguts, T. (2018). Dorsal anterior cingulate-brainstem ensemble as a reinforcement meta-learner. PLoS Computational Biology, 14(8), e1006370. https://doi.org/10.1371/journal.pcbi.1006370CrossRefGoogle ScholarPubMed
Varazzani, C., San-Galli, A., Gilardeau, S., & Bouret, S. (2015). Noradrenaline and dopamine neurons in the reward/effort trade-off: A direct electrophysiological comparison in behaving monkeys. The Journal of Neuroscience, 35(20), 7866–7877. https://doi.org/10.1523/JNEUROSCI.0454-15.2015CrossRefGoogle ScholarPubMed
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C., & Wager, T. D. (2011). Large-scale automated synthesis of human functional neuroimaging data. Nature Methods, 8(8), 665–670. https://doi.org/10.1038/nmeth.1635CrossRefGoogle ScholarPubMed
Target article
Meta-learned models of cognition
Related commentaries (22)
Bayes beyond the predictive distribution
Challenges of meta-learning and rational analysis in large worlds
Combining meta-learned models with process models of cognition
Integrative learning in the lens of meta-learned models of cognition: Impacts on animal and human learning outcomes
Is human compositionality meta-learned?
Learning and memory are inextricable
Linking meta-learning to meta-structure
Meta-learned models as tools to test theories of cognitive development
Meta-learned models beyond and beneath the cognitive
Meta-learning and the evolution of cognition
Meta-learning as a bridge between neural networks and symbolic Bayesian models
Meta-learning goes hand-in-hand with metacognition
Meta-learning in active inference
Meta-learning modeling and the role of affective-homeostatic states in human cognition
Meta-learning: Bayesian or quantum?
Probabilistic programming versus meta-learning as models of cognition
Quantum Markov blankets for meta-learned classical inferential paradoxes with suboptimal free energy
Quo vadis, planning?
The added value of affective processes for models of human cognition and learning
The hard problem of meta-learning is what-to-learn
The meta-learning toolkit needs stronger constraints
The reinforcement metalearner as a biologically plausible meta-learning framework
Author response
Meta-learning: Data, architecture, and both