Hostname: page-component-78c5997874-m6dg7 Total loading time: 0 Render date: 2024-11-17T18:19:12.522Z Has data issue: false hasContentIssue false

Meta-learned models as tools to test theories of cognitive development

Published online by Cambridge University Press:  23 September 2024

Kate Nussenbaum*
Affiliation:
Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA [email protected] https://www.katenuss.com/
Catherine A. Hartley*
Affiliation:
Department of Psychology, New York University, New York, NY, USA [email protected] https://www.hartleylab.org/
*
*Corresponding authors.
*Corresponding authors.

Abstract

Binz et al. argue that meta-learned models are essential tools for understanding adult cognition. Here, we propose that these models are particularly useful for testing hypotheses about why learning processes change across development. By leveraging their ability to discover optimal algorithms and account for capacity limitations, researchers can use these models to test competing theories of developmental change in learning.

Type
Open Peer Commentary
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press

Binz et al. argue compellingly for meta-learning as a tool to understand adult cognition, but their vision is incomplete: Meta-learned models are particularly apt tools for studying development. As the authors note, meta-learned models provide a natural foundation for theorizing about learning to learn (Wang, Reference Wang2021) and for understanding experience-driven changes in goal-directed behavior (Nussenbaum & Hartley, Reference Nussenbaum and Hartleyin press). Here, however, we consider the authors’ central argument and focus not on meta-learning algorithms or the process of “learning to learn,” but rather on meta-learned models as a tool for understanding developmental changes in learning. To date, research has revealed age-related shifts in the algorithms and neural circuitry that underlie learning (Bolenz, Reiter, & Eppinger, Reference Bolenz, Reiter and Eppinger2017; Gualtieri & Finn, Reference Gualtieri and Finn2022; Hartley, Nussenbaum, & Cohen, Reference Hartley, Nussenbaum and Cohen2021; Nussenbaum & Hartley, Reference Nussenbaum and Hartley2019; Raab & Hartley, Reference Raab, Hartley, Morris, Bornstein and Shenhav2018), but understanding why these shifts occur has proven more difficult.

Disentangling whether age-related changes in learning reflect adaptation to age-varying “external” ecological problems or “internal” changes in cognitive capacity has been difficult, in part because developmentalists have lacked the formalizations needed to make specific predictions about how these two factors might drive age-related changes in behavior. As an example, in many studies of reinforcement learning, participants are tasked with earning the most reward (e.g., points or money) by selecting between different options. In these tasks, children tend to make “noisier” or more exploratory choices, which typically result in poorer performance (Nussenbaum & Hartley, Reference Nussenbaum and Hartley2019) but enhanced learning of task structure (Blanco & Sloutsky, Reference Blanco and Sloutsky2021; Liquin & Gopnik, Reference Liquin and Gopnik2022; Sumner et al., Reference Sumner, Li, Perfors, Hayes, Navarro and Sarnecka2019). These findings have led researchers to theorize that children and adults have optimized their learning computations to maximize reward over different types of environments – children's learning contexts may have features (e.g., greater reward stochasticity, longer temporal horizons) that favor exploration over immediate reward gain (Gopnik, Reference Gopnik2020). An alternative account, however, is that children's “noisier” behavior reflects a more limited cognitive capacity (Craik & Bialystok, Reference Craik and Bialystok2006; Ruel, Devine, & Eppinger, Reference Ruel, Devine and Eppinger2021), which constrains their “optimization” of learning computations.

By enabling separable manipulation of external experience and internal cognitive capacity, meta-learned models may help arbitrate between these accounts of developmental change. As Binz et al. note, through meta-learning, the optimal algorithm emerges through experience, and is shaped by the distribution of environments on which the model is trained. Thus, these models can be used to test how differences in “training” experience might yield different patterns of learning at “test.” Examining how training environments influence model behavior within the same tasks that have been used to study cognitive development can provide insight into the types of environments for which children, adolescents, and adults have optimized their learning computations. As one example, Binz and Schulz (Reference Binz and Schulz2022) found that increasing the variance in the rewards that a model experienced during training led to exploration decisions at test that more closely approximated those of human learners, suggesting that people may have tuned their learning computations for environments with more stochasticity than the task context. Importantly, meta-learned models can provide insight into how optimal learning is shaped by exposure to highly complex environments in which multiple features vary and interact – environments for which analytically derived “solutions” are intractable but that are more reflective of real-world contexts than the simple tasks that have been used in most prior research.

At the same time, meta-learned models can account for how changes in capacity limitations may yield developmental changes in learning. As Binz et al. note, the complexity of meta-learned models can be manipulated easily, particularly when they are implemented as neural networks. Manipulating capacity constraints and comparing model behavior to that of learners at different ages can thus provide insight into whether developmental changes in learning are well accounted for by theories of “resource-rationality.” Binz and Schulz (Reference Binz and Schulz2022) found, for example, that increasing the complexity of the algorithms that a meta-learning network could implement led to changes in patterns of directed exploration that mirrored those that occur across adolescence (Somerville et al., Reference Somerville, Sasse, Garrad, Drysdale, Abi Akar, Insel and Wilson2017). This result exemplifies how, rather than emerging from differences in external “training” experience, age-related changes in learning can also be driven by differences in cognitive capacity.

Further, neural network implementations of meta-learning enable separable manipulations of “algorithmic complexity” via network weights and “computational complexity” via network activations. Future developmental modeling work could explore the ramifications of constraints on algorithmic versus computational complexity. Numerous studies have revealed a dissociation between children's knowledge of rules or structure and their ability to leverage them to guide behavior (Decker, Otto, Daw, & Hartley, Reference Decker, Otto, Daw and Hartley2016; Zelazo, Frye, & Rapus, Reference Zelazo, Frye and Rapus1996). The types of structural knowledge that can be acquired at different ages may depend on algorithmic complexity, while use of that knowledge may rely on processes like working memory and proactive cognitive control, which may instantiated via complex computations. Behavioral dissociations between algorithmic and computational complexity may be mirrored by neural dissociations. Age-related change in brain structure or the wiring of neural circuitry may be analogous to age-related change in network weights and relate more strongly to algorithmic complexity, whereas age-related change in patterns of neural activation during learning may be more closely related to the network activations that underlie computational complexity. Thus, predictions about how age-related change in these two forms of capacity limitations affect learning could be further tested and constrained with neural data.

While we have suggested that researchers can leverage meta-learned models to more explicitly test whether children optimize their behavior for different environments or with different constraints, this dichotomy is likely false. Experience and constraints interact throughout the lifespan, and the changing “constraints” implemented by neurobiology may themselves serve an adaptive function – it may be the case that the complexity of the learning algorithms an organism can implement and execute systematically increases through exposure to increasingly varied and complex environments. Meta-learned models of cognition thus have the potential to address questions of longstanding interest in developmental science, while empirical developmental research provides a valuable testbed for the theoretical utility of these computational tools.

Acknowledgements

We thank Rheza Budiono for helpful feedback.

Financial support

Preparation of this commentary was supported by a CV Starr Foundation Fellowship (to K. N.).

Competing interest

None.

References

Binz, M., & Schulz, E. (2022). Modeling human exploration through resource-rational reinforcement learning. Advances in Neural Information Processing Systems, 35, 3175531768.Google Scholar
Blanco, N. J., & Sloutsky, V. M. (2021). Systematic exploration and uncertainty dominate young children's choices. Developmental Science, 24(2), e13026.CrossRefGoogle ScholarPubMed
Bolenz, F., Reiter, A. M. F., & Eppinger, B. (2017). Developmental changes in learning: Computational mechanisms and social influences. Frontiers in Psychology, 8, 2048.CrossRefGoogle ScholarPubMed
Craik, F. I. M., & Bialystok, E. (2006). Cognition through the lifespan: Mechanisms of change. Trends in Cognitive Sciences, 10(3), 131138.CrossRefGoogle ScholarPubMed
Decker, J. H., Otto, A. R., Daw, N. D., & Hartley, C. A. (2016). From creatures of habit to goal-directed learners: Tracking the developmental emergence of model-based reinforcement learning. Psychological Science, 27(6), 848858.CrossRefGoogle ScholarPubMed
Gopnik, A. (2020). Childhood as a solution to explore–exploit tensions. Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences, 375(1803), 20190502.CrossRefGoogle ScholarPubMed
Gualtieri, S., & Finn, A. S. (2022). The sweet spot: When children's developing abilities, brains, and knowledge make them better learners than adults. Perspectives on Psychological Science: A Journal of the Association for Psychological Science, 17(5), 13221338.CrossRefGoogle ScholarPubMed
Hartley, C. A., Nussenbaum, K., & Cohen, A. O. (2021). Interactive development of adaptive learning and memory. Annual Review of Developmental Psychology, 3, 5985.CrossRefGoogle Scholar
Liquin, E. G., & Gopnik, A. (2022). Children are more exploratory and learn more than adults in an approach-avoid task. Cognition, 218, 104940.CrossRefGoogle Scholar
Nussenbaum, K., & Hartley, C. A. (2019). Reinforcement learning across development: What insights can we draw from a decade of research? Developmental Cognitive Neuroscience, 40, 100733.CrossRefGoogle ScholarPubMed
Nussenbaum, K., & Hartley, C.A. (in press). Understanding the development of reward learning through the lens of meta-learning. Nature Reviews Psychology.Google Scholar
Raab, H. A., & Hartley, C. A. (2018). The development of goal-directed decision-making. In Morris, R., Bornstein, A., & Shenhav, A. (Eds.), Goal-directed decision making: Computations and neural circuits (pp. 279308). Elsevier Academic Press.CrossRefGoogle Scholar
Ruel, A., Devine, S., & Eppinger, B. (2021). Resource-rational approach to meta-control problems across the lifespan. Wiley Interdisciplinary Reviews. Cognitive Science, 12(5), e1556.CrossRefGoogle ScholarPubMed
Somerville, L. H., Sasse, S. F., Garrad, M. C., Drysdale, A. T., Abi Akar, N., Insel, C., & Wilson, R. C. (2017). Charting the expansion of strategic exploratory behavior during adolescence. Journal of Experimental Psychology. General, 146(2), 155164.CrossRefGoogle ScholarPubMed
Sumner, E., Li, A. X., Perfors, A., Hayes, B., Navarro, D., & Sarnecka, B. W. (2019). The exploration advantage: Children's instinct to explore allows them to find information that adults miss. https://doi.org/10.31234/osf.io/h437vCrossRefGoogle Scholar
Wang, J. X. (2021). Meta-learning in natural and artificial intelligence. Current Opinion in Behavioral Sciences, 38, 9095.CrossRefGoogle Scholar
Zelazo, P. D., Frye, D., & Rapus, T. (1996). An age-related dissociation between knowing rules and using them. Cognitive Development, 11(1), 3763.CrossRefGoogle Scholar