Hostname: page-component-848d4c4894-jbqgn Total loading time: 0 Render date: 2024-06-14T10:59:27.756Z Has data issue: false hasContentIssue false
  • English
  • Français

The model-resistant richness of human visual experience

Published online by Cambridge University Press:  06 December 2023

Jianghao Liu Open the ORCID record for Jianghao Liu [Opens in a new window]  and
Paolo Bartolomeo
Jianghao Liu
Affiliation:
Sorbonne Université, Inserm, CNRS, Paris Brain Institute, ICM, Hôpital de la Pitié-Salpêtrière, Paris, France jianghao.liu@icm-institute.org paolo.bartolomeo@icm-institute.org Dassault Systèmes, Vélizy-Villacoublay, France
Paolo Bartolomeo
Affiliation:
Sorbonne Université, Inserm, CNRS, Paris Brain Institute, ICM, Hôpital de la Pitié-Salpêtrière, Paris, France jianghao.liu@icm-institute.org paolo.bartolomeo@icm-institute.org
Get access Rights & Permissions [Opens in a new window]

Abstract

Current deep neural networks (DNNs) are far from being able to model the rich landscape of human visual experience. Beyond visual recognition, we explore the neural substrates of visual mental imagery and other visual experiences. Rather than shared visual representations, temporal dynamics and functional connectivity of the process are essential. Generative adversarial networks may drive future developments in simulating human visual experience.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 46 , 2023 , e401
Check for updates
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bartolomeo, P., Hajhajate, D., Liu, J., & Spagna, A. (2020). Assessing the causal role of early visual areas in visual mental imagery. Nature Reviews Neuroscience, 21(9), 517. https://doi.org/10.1038/s41583-020-0348-5CrossRefGoogle ScholarPubMed
Bartolomeo, P., Seidel Malkinson, T. (2022). Building models, testing models: Asymmetric roles of SLF III networks?: Comment on “Left and right temporal-parietal junctions (TPJs) as ‘match/mismatch’ hedonic machines: A unifying account of TPJ function” by Doricchi et al. Physics of Life Reviews, 44, 70–72. https://doi.org/10/grrsd8CrossRefGoogle ScholarPubMed
Bartolomeo, P., & Thiebaut de Schotten, M. (2016). Let thy left brain know what thy right brain doeth: Inter-hemispheric compensation of functional deficits after brain damage. Neuropsychologia, 93, 407–412. https://doi.org/10/f9g9wbCrossRefGoogle ScholarPubMed
Bergmann, J., Morgan, A. T., & Muckli, L. (2019). Two distinct feedback codes in V1 for “real” and “imaginary: Internal experiences. bioRxiv, 664870. https://doi.org/10.1101/664870Google Scholar
Cushing, C. A., Dawes, A. J., Hofmann, S. G., Lau, H., LeDoux, J. E., & Taschereau-Dumouchel, V. (2023). A generative adversarial model of intrusive imagery in the human brain. PNAS Nexus, 2(1), pgac265. https://doi.org/10.1093/pnasnexus/pgac265CrossRefGoogle ScholarPubMed
Deperrois, N., Petrovici, M. A., Senn, W., & Jordan, J. (2022). Learning cortical representations through perturbed and adversarial dreaming. eLife, 11, e76384. https://doi.org/10.7554/eLife.76384CrossRefGoogle ScholarPubMed
Dijkstra, N., Mostert, P., Lange, F. P., Bosch, S., & van Gerven, M. A. (2018). Differential temporal dynamics during visual imagery and perception. eLife, 7, e33904. doi: https://doi.org/10.7554/eLife.33904CrossRefGoogle ScholarPubMed
Gershman, S. J. (2019). The generative adversarial brain. Frontiers in Artificial Intelligence, 2, 486362. doi: https://www.frontiersin.org/articles/10.3389/frai.2019.00018CrossRefGoogle ScholarPubMed
Hahamy, A., Wilf, M., Rosin, B., Behrmann, M., & Malach, R. (2021). How do the blind “see”? The role of spontaneous brain activity in self-generated perception. Brain, 144(1), 340–353. https://doi.org/10.1093/brain/awaa384CrossRefGoogle ScholarPubMed
Keogh, R., Pearson, J., & Zeman, A. (2021). Aphantasia: The science of visual imagery extremes. Handbook of Clinical Neurology, 178, 277–296. https://doi.org/10.1016/B978-0-12-821377-3.00012-XCrossRefGoogle ScholarPubMed
Lau, H. (2019). Consciousness, metacognition, & perceptual reality monitoring. PsyArXiv. https://doi.org/10.31234/osf.io/ckbyfGoogle Scholar
Lindsay, G. W., Mrsic-Flogel, T. D., & Sahani, M. (2022). Bio-inspired neural networks implement different recurrent visual processing strategies than task-trained ones do. bioRxiv, 2022.03.07.483196. https://doi.org/10.1101/2022.03.07.483196Google Scholar
Liu, J., & Bartolomeo, P. (2023). Probing the unimaginable: The impact of aphantasia on distinct domains of visual mental imagery and visual perception. Cortex, 166, 338–347. doi: 10.1016/j.cortex.2023.06.003CrossRefGoogle ScholarPubMed
Liu, J., Bayle, D. J., Spagna, A., Sitt, J. D., Bourgeois, A., Lehongre, K., … Bartolomeo, P. (2023). Fronto-parietal networks shape human conscious report through attention gain and reorienting. Communications Biology, 6, 730. doi: 10.1038/s42003-023-05108-2CrossRefGoogle ScholarPubMed
Liu, J., Spagna, A., & Bartolomeo, P. (2022b). Hemispheric asymmetries in visual mental imagery. Brain Structure and Function, 227(2), 697–708. https://doi.org/10.1007/s00429-021-02277-wCrossRefGoogle ScholarPubMed
Liu, J., Zhan, M., Hajhajate, D., Spagna, A., Dehaene, S., Cohen, L., & Bartolomeo, P. (2023). Ultra-high field fMRI of visual mental imagery in typical imagers and aphantasic individuals. bioRxiv. https://doi.org/10.1101/2023.06.14.544909Google Scholar
Milton, F., Fulford, J., Dance, C., Gaddum, J., Heuerman-Williamson, B., Jones, K., … Zeman, A. (2021). Behavioral and neural signatures of visual imagery vividness extremes: Aphantasia versus hyperphantasia. Cerebral Cortex Communications, 2(2), tgab035. https://doi.org/10.1093/texcom/tgab035CrossRefGoogle ScholarPubMed
Muckli, L., De Martino, F., Vizioli, L., Petro, L. S., Smith, F. W., Ugurbil, K., … Yacoub, E. (2015). Contextual feedback to superficial layers of V1. Current Biology, 25(20), 2690–2695. https://doi.org/10.1016/j.cub.2015.08.057CrossRefGoogle ScholarPubMed
Spagna, A., Hajhajate, D., Liu, J., & Bartolomeo, P. (2021). Visual mental imagery engages the left fusiform gyrus, but not the early visual cortex: A meta-analysis of neuroimaging evidence. Neuroscience & Biobehavioral Reviews, 122, 201–217. https://doi.org/10.1016/j.neubiorev.2020.12.029CrossRefGoogle Scholar
van de Ven, G. M., Siegelmann, H. T., & Tolias, A. S. (2020). Brain-inspired replay for continual learning with artificial neural networks. Nature Communications, 11(1), Article 1. https://doi.org/10.1038/s41467-020-17866-2CrossRefGoogle ScholarPubMed