Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses

Tal Golan; JohnMark Taylor; Heiko Schütt; Benjamin Peters; Rowan P. Sommers; Katja Seeliger; Adrien Doerig; Paul Linton; Talia Konkle; Marcel van Gerven; Konrad Kording; Blake Richards; Tim C. Kietzmann; Grace W. Lindsay; Nikolaus Kriegeskorte

doi:10.1017/S0140525X23001553

Deep neural networks are not a single hypothesis but a language for expressing computational hypotheses

Published online by Cambridge University Press: 06 December 2023

Talia Konkle and

Tal Golan: Affiliation:
Department of Cognitive and Brain Sciences, Ben-Gurion University of the Negev, Be'er Sheva, Israel golan.neuro@bgu.ac.il brainsandmachines.org
JohnMark Taylor: Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/
Heiko Schütt: Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/ Center for Neural Science, New York University, New York, NY, USA
Benjamin Peters: Affiliation:
School of Psychology & Neuroscience, University of Glasgow, Glasgow, UK benjamin.peters@posteo.de
Rowan P. Sommers: Affiliation:
Department of Neurobiology of Language, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands rowan.sommers@mpi.nl
Katja Seeliger: Affiliation:
Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany katjaseeliger@posteo.de
Adrien Doerig: Affiliation:
Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany adoerig@uni-osnabrueck.de tim.kietzmann@uni-osnabrueck.de kietzmannlab.org kietzmannlab.org
Paul Linton: Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/ Presidential Scholars in Society and Neuroscience, Center for Science and Society, Columbia University, New York, NY, USA Italian Academy for Advanced Studies in America, Columbia University, New York, NY, USA
Talia Konkle: Affiliation:
Department of Psychology and Center for Brain Sciences, Harvard University, Cambridge, MA, USA talia_konkle@harvard.edu https://konklab.fas.harvard.edu/
Marcel van Gerven: Affiliation:
Donders Institute for Brain, Cognition and Behaviour, Nijmegen, The Netherlands artcogsys.com
Konrad Kording: Affiliation:
Departments of Bioengineering and Neuroscience, University of Pennsylvania, Philadelphia, PA, USA koerding@gmail.com kordinglab.com Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada blake.richards@mila.quebec linclab.org
Blake Richards: Affiliation:
Learning in Machines and Brains Program, CIFAR, Toronto, ON, Canada blake.richards@mila.quebec linclab.org Mila, Montreal, QC, Canada School of Computer Science, McGill University, Montreal, QC, Canada Department of Neurology & Neurosurgery, McGill University, Montreal, QC, Canada Montreal Neurological Institute, Montreal, QC, Canada
Tim C. Kietzmann: Affiliation:
Institute of Cognitive Science, University of Osnabrück, Osnabrück, Germany adoerig@uni-osnabrueck.de tim.kietzmann@uni-osnabrueck.de kietzmannlab.org kietzmannlab.org
Grace W. Lindsay: Affiliation:
Department of Psychology and Center for Data Science, New York University, New York, NY, USA grace.lindsay@nyu.edu lindsay-lab.github.io
Nikolaus Kriegeskorte: Affiliation:
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA jt3295@columbia.edu hs3110@columbia.edu paul.linton@columbia.edu nk2765@columbia.edu johnmarktaylor.com hebartlab.com https://linton.vision/ Departments of Psychology, Neuroscience, and Electrical Engineering, Columbia University, New York, NY, USA

Article contents

Abstract
References

Get access

Rights & Permissions

Abstract

An ideal vision model accounts for behavior and neurophysiology in both naturalistic conditions and designed lab experiments. Unlike psychological theories, artificial neural networks (ANNs) actually perform visual tasks and generate testable predictions for arbitrary inputs. These advantages enable ANNs to engage the entire spectrum of the evidence. Failures of particular models drive progress in a vibrant ANN research program of human vision.

Information

Type: Open Peer Commentary
Information: Behavioral and Brain Sciences , Volume 46 , 2023 , e392

DOI: https://doi.org/10.1017/S0140525X23001553 [Opens in a new window]
Copyright: Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Bashivan, P., Kar, K., & DiCarlo, J. J. (2019). Neural population control via deep image synthesis. Science (New York, N.Y.), 364(6439), eaav9436. https://doi.org/10.1126/science.aav9436CrossRef Google Scholar PubMed

Cichy, R. M., Roig, G., & Oliva, A. (2019). The Algonauts project. Nature Machine Intelligence, 1(12), 613–613. https://doi.org/10.1038/s42256-019-0127-zCrossRef Google Scholar

Cohen, J., Rosenfeld, E., & Kolter, Z. (2019). Certified adversarial robustness via randomized smoothing. In Chaudhuri, K. & Salakhutdinov, R. (Eds.), Proceedings of the 36th international conference on machine learning. Proceedings of Machine Learning Research, Long Beach, CA, USA (Vol. 97, pp. 1310–1320). https://proceedings.mlr.press/v97/cohen19c.html Google Scholar

Doerig, A., Sommers, R., Seeliger, K., Richards, B., Ismael, J., Lindsay, G., … Kietzmann, T. C. (2022). The neuroconnectionist research programme. Nature Reviews Neuroscience, 24, 431–450. https://doi.org/10.1038/s41583-023-00705-wCrossRef Google Scholar

Dujmović, M., Malhotra, G., & Bowers, J. S. (2020). What do adversarial images tell us about human vision?. eLife, 9, e55978. https://doi.org/10.7554/eLife.55978CrossRef Google Scholar PubMed

Dwivedi, K., Bonner, M. F., Cichy, R. M., & Roig, G. (2021). Unveiling functions of the visual cortex using task-specific deep neural networks. PLoS Computational Biology, 17(8), e1009267. https://doi.org/10.1371/journal.pcbi.1009267CrossRef Google Scholar PubMed

Feather, J., Durango, A., Gonzalez, R., & McDermott, J. (2019). Metamers of neural networks reveal divergence from human perceptual systems. In Wallach, H., Larochelle, H., Beygelzimer, A., d'Alché-Buc, F., Fox, E., & Garnett, R. (Eds.), Advances in Neural Information Processing Systems, Vancouver, BC, Canada (Vol. 32, pp. 10078–10089). Curran Associates, Inc. https://proceedings.neurips.cc/paper/2019/file/ac27b77292582bc293a51055bfc994ee-Paper.pdf Google Scholar

Funahashi, K. I., & Nakamura, Y. (1993). Approximation of dynamical systems by continuous time recurrent neural networks. Neural Networks, 6(6), 801–806. https://doi.org/10.1016/S0893-6080(05)80125-XCrossRef Google Scholar

Geirhos, R., Narayanappa, K., Mitzkus, B., Thieringer, T., Bethge, M., Wichmann, F. A., & Brendel, W. (2021). Partial success in closing the gap between human and machine vision. In Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P. S., & Wortman Vaughan, J. (Eds.), Advances in Neural Information Processing Systems (Vol. 34, pp. 23885–23899). Curran Associates, Inc. https://proceedings.neurips.cc/paper/2021/file/c8877cff22082a16395a57e97232bb6f-Paper.pdf Google Scholar

Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2019). ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. In International conference on learning representations, New Orleans, LA, USA. https://openreview.net/forum?id=Bygh9j09KX Google Scholar

Golan, T., Raju, P. C., & Kriegeskorte, N. (2020). Controversial stimuli: Pitting neural networks against each other as models of human cognition. Proceedings of the National Academy of Sciences of the United States of America, 117(47), 29330–29337. https://doi.org/10.1073/pnas.1912334117CrossRef Google Scholar PubMed

Güçlü, U., & van Gerven, M. A. J. (2015). Deep neural networks reveal a gradient in the complexity of neural representations across the ventral stream. Journal of Neuroscience, 35(27), 10005–10014. https://doi.org/10.1523/JNEUROSCI.5023-14.2015CrossRef Google Scholar PubMed

Guo, C., Lee, M., Leclerc, G., Dapello, J., Rao, Y., Madry, A., & Dicarlo, J. (2022). Adversarially trained neural representations are already as robust as biological neural representations. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., & Sabato, S. (Eds.), Proceedings of the 39th international conference on machine learning. Proceedings of Machine Learning Research, Baltimore, MD, USA (Vol. 162, pp. 8072–8081). PMLR. https://proceedings.mlr.press/v162/guo22d.html Google Scholar

Hermann, K., Chen, T., & Kornblith, S. (2020). The origins and prevalence of texture bias in convolutional neural networks. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., & Lin, H. (Eds.), Advances in Neural Information Processing Systems, Vancouver, BC, Canada (Vol. 33, pp. 19000–19015). Curran Associates, Inc. https://proceedings.neurips.cc/paper/2020/file/db5f9f42a7157abe65bb145000b5871a-Paper.pdf Google Scholar

Kriegeskorte, N., Mur, M., Ruff, D. A., Kiani, R., Bodurka, J., Esteky, H., … Bandettini, P. A. (2008). Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron, 60(6), 1126–1141. https://doi.org/10.1016/j.neuron.2008.10.043CrossRef Google Scholar PubMed

Lakatos, I. (1978). Science and pseudoscience. Philosophical Papers, 1, 1–7.Google Scholar

Lindsay, G. W. (2021). Convolutional neural networks as a model of the visual system: Past, present, and future. Journal of Cognitive Neuroscience, 33(10), 2017–2031. https://doi.org/10.1162/jocn_a_01544CrossRef Google Scholar

Madry, A., Makelov, A., Schmidt, L., Tsipras, D., & Vladu, A. (2019). Towards deep -learning models resistant to adversarial attacks. In International conference on learning representations, Vancouver, BC, Canada. https://openreview.net/forum?id=rJzIBfZAb Google Scholar

Newell, A. (1973). You can't play 20 questions with nature and win: Projective comments on the papers of this symposium. In W. G. Chase (Ed.), Visual information processing: Proceedings of the 8th annual Carnegie symposium on cognition, held at the Carnegie-Mellon University, Pittsburgh, Pennsylvania, May 19, 1972 (pp. 283–305). Academic Press.Google Scholar

Nonaka, S., Majima, K., Aoki, S. C., & Kamitani, Y. (2021). Brain hierarchy score: Which deep neural networks are hierarchically brain-like?. iScience, 24(9), 103013. https://doi.org/10.1016/j.isci.2021.103013CrossRef Google Scholar PubMed

Nuriel, O., Benaim, S., & Wolf, L. (2021). Permuted AdaIN: Reducing the bias towards global statistics in image classification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9482–9491). Online. https://openaccess.thecvf.com/content/CVPR2021/html/Nuriel_Permuted_AdaIN_Reducing_the_Bias_Towards_Global_Statistics_in_Image_CVPR_2021_paper.html CrossRef Google Scholar

Olshausen, B. A., & Field, D. J. (2005). How close are we to understanding V1?. Neural Computation, 17(8), 1665–1699. https://doi.org/10.1162/0899766054026639CrossRef Google Scholar PubMed

Peters, B., & Kriegeskorte, N. (2021). Capturing the objects of vision with neural networks. Nature Human Behaviour, 5(9), 1127–1144. https://doi.org/10.1038/s41562-021-01194-6CrossRef Google Scholar PubMed

Rajalingham, R., Issa, E. B., Bashivan, P., Kar, K., Schmidt, K., & DiCarlo, J. J. (2018). Large-scale, high-resolution comparison of the core visual object recognition behavior of humans, monkeys, and state-of-the-art deep artificial neural networks. Journal of Neuroscience, 38(33), 7255–7269. https://doi.org/10.1523/JNEUROSCI.0388-18.2018CrossRef Google Scholar PubMed

Rust, N. C., & Movshon, J. A. (2005). In praise of artifice. Nature Neuroscience, 8(12), 1647–1650. https://doi.org/10.1038/nn1606CrossRef Google Scholar PubMed

Schäfer, A. M., & Zimmermann, H. G. (2007). Recurrent neural networks are universal approximators. International Journal of Neural Systems, 17(4), 253–263. https://doi.org/10.1142/S0129065707001111CrossRef Google Scholar PubMed

Schrimpf, M., Kubilius, J., Hong, H., Majaj, N. J., Rajalingham, R., Issa, E. B., … DiCarlo, J. J. (2018). Brain-Score: Which artificial neural network for object recognition is most brain-like?. BioRxiv, 407007. https://doi.org/10.1101/407007Google Scholar

Serre, T. (2019). Deep learning: The good, the bad, and the ugly. Annual Review of Vision Science, 5, 399–426. https://doi.org/10.1146/annurev-vision-091718-014951CrossRef Google Scholar PubMed

Spoerer, C. J., McClure, P., & Kriegeskorte, N. (2017). Recurrent convolutional neural networks: A better model of biological object recognition. Frontiers in Psychology, 8, 1551. https://doi.org/10.3389/fpsyg.2017.01551CrossRef Google Scholar PubMed

Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013). Intriguing properties of neural networks. arXiv. arXiv:1312.6199. https://doi.org/10.48550/arXiv.1312.6199Google Scholar

Walker, E. Y., Sinz, F. H., Cobos, E., Muhammad, T., Froudarakis, E., Fahey, P. G., … Tolias, A. S. (2019). Inception loops discover what excites neurons most using deep predictive models. Nature Neuroscience, 22(12), 2060–2065. https://doi.org/10.1038/s41593-019-0517-xCrossRef Google Scholar PubMed