Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-01-09T23:43:21.009Z Has data issue: false hasContentIssue false
  • English
  • Français

Neural networks need real-world behavior

Published online by Cambridge University Press:  06 December 2023

Aedan Y. Li Open the ORCID record for Aedan Y. Li [Opens in a new window]  and
Marieke Mur Open the ORCID record for Marieke Mur [Opens in a new window]
Aedan Y. Li
Affiliation:
Department of Psychology, Western University, London, ON, Canada [email protected], www.aedanyueli.com
Marieke Mur
Affiliation:
Department of Psychology, Western University, London, ON, Canada [email protected], www.aedanyueli.com Department of Computer Science, Western University, London, ON, Canada [email protected] murlab.org
Get access Rights & Permissions [Opens in a new window]

Abstract

Bowers et al. propose to use controlled behavioral experiments when evaluating deep neural networks as models of biological vision. We agree with the sentiment and draw parallels to the notion that “neuroscience needs behavior.” As a promising path forward, we suggest complementing image recognition tasks with increasingly realistic and well-controlled task environments that engage real-world object recognition behavior.

Type
Open Peer Commentary
Information
Behavioral and Brain Sciences , Volume 46 , 2023 , e398
Check for updates
Copyright
Copyright © The Author(s), 2023. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adolph, K. E., & Franchak, J. M. (2017). The development of motor behavior. Wiley Interdisciplinary Reviews. Cognitive Science, 8(1–2), e1430. https://doi.org/10.1002/wcs.1430CrossRefGoogle ScholarPubMed
Alsbury-Nealy, K., Wang, H., Howarth, C., Gordienko, A., Schlichting, M. L., & Duncan, K. D. (2022). OpenMaze: An open-source toolbox for creating virtual navigation experiments. Behavior Research Methods, 54, 1374–1387. https://doi.org/10.3758/s13428-021-01664-9CrossRefGoogle ScholarPubMed
Barense, M. D., Rogers, T. T., Bussey, T. J., Saksida, L. M., & Graham, K. S. (2010). Influence of conceptual knowledge on visual object discrimination: Insights from semantic dementia and MTL amnesia. Cerebral Cortex, 20(11), 2568–2582. https://doi.org/10.1093/cercor/bhq004CrossRefGoogle ScholarPubMed
Behrens, T. E. J., Muller, T. H., Whittington, J. C. R., Mark, S., Baram, A. B., Stachenfeld, K. L., & Kurth-Nelson, Z. (2018). What is a cognitive map? Organizing knowledge for flexible behavior. Neuron, 100(2), 490–509. https://doi.org/10.1016/j.neuron.2018.10.002CrossRefGoogle ScholarPubMed
Brookes, J., Warburton, M., Alghadier, M., Mon-Williams, M., & Mushtaq, F. (2020). Studying human behavior with virtual reality: The unity experiment framework. Behavior Research Methods, 52, 455–463. https://doi.org/10.3758/s13428-019-01242-0CrossRefGoogle ScholarPubMed
Cowell, R. A., Barense, M. D., & Sadil, P. S. (2019). A roadmap for understanding memory: Decomposing cognitive processes into operations and representations. eNeuro, 6(4), ENEURO.0122-19.2019. https://doi.org/10.1523/ENEURO.0122-19.2019CrossRefGoogle ScholarPubMed
de Cothi, W., Nyberg, N., Griesbauer, E. M., Ghanamé, C., Zisch, F., Lefort, J. M., … Spiers, H. J. (2022). Predictive maps in rats and humans for spatial navigation. Current Biology: CB, 32(17), 3676–3689.e5. https://doi.org/10.1016/j.cub.2022.06.090CrossRefGoogle ScholarPubMed
Druckmann, S., & Rust, N. C. (2023). Unraveling the entangled brain: How do we go about it? Journal of Cognitive Neuroscience, 35, 368–371. https://doi.org/10.1162/jocn_a_01950CrossRefGoogle Scholar
Gan, C., Schwartz, J., Alter, S., Mrowca, D., Schrimpf, M., Traer, J., … Yamins, D. L. K. (2021). ThreeDWorld: A platform for interactive multi-modal physical simulation. bioRxiv. https://doi.org/10.48550/arXiv.2007.04954Google Scholar
Hodges, J. R., & Patterson, K. (2007). Semantic dementia: A unique clinicopathological syndrome. The Lancet. Neurology, 6(11), 1004–1014. https://doi.org/10.1016/S1474-4422(07)70266-1CrossRefGoogle ScholarPubMed
Jozwik, K. M., Kietzmann, T. C., Cichy, R. M., Kriegeskorte, N., & Mur, M. (2023). Deep neural networks and visuo-semantic models explain complementary components of human ventral-stream representational dynamics. The Journal of Neuroscience, 43(10), 1731–1741. https://doi.org/10.1523/JNEUROSCI.1424-22.2022CrossRefGoogle ScholarPubMed
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A., & Poeppel, D. (2017). Neuroscience needs behavior: Correcting a reductionist bias. Neuron, 93(3), 480–490. https://doi.org/10.1016/j.neuron.2016.12.041CrossRefGoogle ScholarPubMed
Lambon Ralph, M. A., Jefferies, E., Patterson, K., & Rogers, T. T. (2017). The neural and computational bases of semantic cognition. Nature Reviews Neuroscience, 18(1), 42–55. https://doi.org/10.1038/nrn.2016.150CrossRefGoogle Scholar
Li, A. Y., Ladyka-Wojcik, N., Qazilbash, H., Golestani, A., Walther, D. B., Martin, C. B., & Barense, M. D. (2022). Multimodal object representations rely on integrative coding. bioRxiv. https://doi.org/10.1101/2022.08.31.504599Google Scholar
Li, A. Y., Yuan, J. Y., Pun, C., & Barense, M. D. (2023). The effect of memory load on object reconstruction: Insights from an online mouse-tracking task. Attention, Perception & Psychophysics, 85(5), 1612–1630. https://doi.org/10.3758/s13414-022-02650-9CrossRefGoogle ScholarPubMed
Peters, B., & Kriegeskorte, N. (2021). Capturing the objects of vision with neural networks. Nature Human Behaviour, 5(9), 1127–1144. https://doi.org/10.1038/s41562-021-01194-6CrossRefGoogle ScholarPubMed
Peters, B., Retchin, M., & Kriegeskorte, N. (2022). Flying objects: Challenging humans and machines in dynamic object vision. Cognitive Computational Neuroscience. https://doi.org/10.32470/ccn.2022.1301-0CrossRefGoogle Scholar
Snow, J. C., & Culham, J. C. (2021). The treachery of images: How realism influences brain and behavior. Trends in Cognitive Sciences, 25(6), 506–519. https://doi.org/10.1016/j.tics.2021.02.008CrossRefGoogle ScholarPubMed
Spivey, M. (2007). The continuity of mind. Oxford University Press.Google Scholar
Stanford, T. R., Shankar, S., Massoglia, D. P., Costello, M. G., & Salinas, E. (2010). Perceptual decision making in less than 30 milliseconds. Nature Neuroscience, 13(3), 379–385. https://doi.org/10.1038/nn.2485CrossRefGoogle ScholarPubMed
Starrett, M. J., McAvan, A. S., Huffman, D. J., Stokes, J. D., Kyle, C. T., … Ekstrom, A. D. (2021). Landmarks: A solution for spatial navigation and memory experiments in virtual reality. Behavior Research Methods, 53, 1046–1059. https://doi.org/10.3758/s13428-020-01481-6CrossRefGoogle ScholarPubMed
Straub, D., & Rothkopf, C. A. (2022). Putting perception into action with inverse optimal control for continuous psychophysics. eLife, 11, e76635. https://doi.org/10.7554/eLife.76635CrossRefGoogle ScholarPubMed