1. Vision

When experimental psychologists began to probe the minds of infants in the late 1950s and 1960s, most believed that knowledge was the product of innate sensory systems, learned belief systems, and nothing more. They focused, therefore, on infants' capacities for perception and learning, and they developed the approaches and methods that have brought other cognitive systems into view. The book's first chapter introduces these approaches and methods by reviewing centuries of thinking and research on the nature, origins, and development of visual perception, from Descartes and Berkeley to Helmholtz to the present. It devotes most space to three twentieth century scientists – Eleanor Gibson, Richard Held, and David Marr – whose work provided the perspectives and methods (respectively, from experimental psychology, cognitive neuroscience, and computational cognitive science) that research on cognition in infancy builds on. Because insights into the mind tend to come first from studies of behavior, I limit my discussion in this précis to Gibson's transformative behavioral experiments probing the emergence and nature of the mechanisms by which human adults, human infants, and adult and infant animals of diverse species perceive the stable, three-dimensional (3D) visible surface layout.

To explore surface perception in adults, Gibson brought classical psychophysical experiments out of the laboratory and tasked adults, with no training in psychology or the vision sciences, with judging the absolute and relative distances and sizes of objects at diverse locations on open fields (Gibson & Bergman, Reference Gibson and Bergman1954/1991). Their judgments were strikingly accurate, and they depended on patterns of optic flow, produced as the participants moved. To extend these tests to animals and to crawling human infants, Gibson invented the “visual cliff” (Fig. 1a): A structure with a raised central platform bordering two visible surfaces. Both to protect the participants and to remove all tactile information for depth, these surfaces were viewed through a glass plate that covered the array, positioned slightly below the platform on both sides and allowing participants, with no training, to move at will in any direction. With clever lighting, the glass was invisible: When participants looked through it, from the platform, they saw one flat, textured surface, positioned far below the glass, and one flat, textured surface positioned a single safe step down. Gibson relied on the propensity of mobile animals, including crawling infants, to explore their surroundings spontaneously, and she observed their choices of visible surfaces to step on.

Figure 1. (a) The original visual cliffs used for testing dark-reared rats (left) and goats (right). (b) Multiple cues distinguished the two sides of the apparatus (including texture density, pictured on the left), but animals and human infants navigated primarily by optic flow: the depth-dependent displacements of edges projected to the eye of a walking animal or crawling infant. (c) Dark-reared cats crossed equally onto the deep and shallow sides on first exposure to light but came to avoid the deep side as their vision improved, despite the consistent and equal safety of the two sides. (d) Human infants typically refused to crawl onto the deep side, even when a parent encouraged them to do so. All figures are reprinted from Walk & Gibson (Reference Walk and Gibson1961/1991).

Gibson found that all the tested animals that walk on land avoided the apparently deep side of the cliff and crossed readily onto the apparently shallow side. For all these animals, moreover, locomotor choices were guided by patterns of optic flow, as they were in the open-air psychophysics experiments with human adults (Fig. 1b and caption). Gibson's findings provided evidence that depth perception is present in untrained animals and suggested that it depends on a common mechanism, attuned to changes in the light at the eyes as animals move over the ground.

With these tools, Gibson and her collaborators studied the origins of depth perception and found that it is innate. It was exhibited not only by newborn goats, who began to move immediately after birth when placed on the ground but not when placed on a small, elevated platform (Gibson, Reference Gibson and Lindzey1980), but also by dark-reared rats and cats whose only experience of visible surfaces occurred on the cliff device itself, where the protective glass ensured that the apparently deep and shallow sides were in fact equally close, safe, and traversable. The rats avoided the cliff on first exposure to the light, whereas the cats required a few days to adjust to a lighted environment, and initially, they walked on both sides of the platform. As their vision improved, however, the cats began to avoid the deep side, even though all their experiences implied that it was safe (Walk & Gibson, Reference Walk and Gibson1961/1991; Walk, Gibson, & Tighe, Reference Walk, Gibson and Tighe1957/1991; Fig. 1c). Crawling human infants also refused to cross on the side of the visually distant surface, even when called by a parent, who tapped on the glass surface and assured the infant that it was safe (Fig. 1d).

Gibson's studies of the development of surface perception also focused on other modes of exploration. Because young human infants cannot locomote independently, she and her students tested their perception of 3D surfaces by measuring their head movements in response to optic flow fields specifying either an approaching window or an approaching surface whose borders were matched to the window in size, shape, and movement. As the window approached, 3-month-old infants, who cannot yet reach for objects, leaned forward and to the sides to explore the scene that it progressively revealed. As the surface approached, in contrast, they moved their heads backward to keep more of the scene in view and avoid a possible collision (Gibson, Reference Gibson and Collins1982/1991).

Like others before her, Gibson also measured the duration and direction of young infants' visual exploration of events, taking advantage of the fact that infants, like their elders, tend to look longer when visible events undergo informative changes. Her experiments showed that 5-month-old infants reacted with longer looking when a surface's visible movement changed from rigid to nonrigid than when it changed from one rigid motion to another (Gibson, Owsley, & Johnston, Reference Gibson, Owsley and Johnston1978/1991). Moreover, 1-month-old infants reacted to this change in the motion of a visible object when their encounter with the original motion occurred only by touch, as they sucked on a rigid or nonrigid nipple (Gibson & Walker, Reference Gibson and Walker1984/1991).

With this research, Gibson opened a door to the modern cognitive, brain, and computational sciences. She showed that rigorous psychophysical methods that had been argued, since Helmholtz, to be applicable only to vision scientists and their highly trained graduate students, testing themselves in a laboratory, could be conducted with equal rigor on humans and other animals of diverse ages, skills, and experiences as they explored rich, natural environments. Participants in psychophysical experiments need not undergo training in systematic introspection or be capable of verbal reports, if experiments leverage their intrinsic motivation to explore the things and places they encounter.

Because visual exploration begins at the time of eye opening, it provides an especially useful window on the innate foundations of perceptual development, the course of perceptual learning, and the functioning of the developing human brain. Indeed, astute psychophysical studies, focused on the emergence of diverse visual functions including color vision and motion sensitivity, have used the simplest possible indicator of visual exploration in human infants: As each infant explores pairs of visual displays, projected side by side, on each of a series of brief trials, stimulus-blind observers judge which display the infant looks at more (Teller, Reference Teller1979). Using this method, Richard Held and collaborators presented infants with displays viewed through stereo glasses while systematically altering the sizes and orientations of the resulting binocular disparities (to dissociate effects of perceiving depth from effects of detecting disparate images). Their research generated beautiful psychophysical functions marking the onset, development, and acuity of stereopsis for each individual infant. They found that most infants began to look longer at the surfaces that they perceived to vary in depth between 10 and 20 weeks of age, marking the onset and rapid subsequent development of this cortical function (Held, Birch, & Gwiazda, Reference Held, Birch and Gwiazda1980). Research by Rachel Keen Clifton, focused on auditory perception and exploratory head turning, provided evidence for the emergence of the cortical functions underlying auditory localization over the same ages (Clifton, Morrongiello, & Dowd, Reference Clifton, Morrongiello and Dowd1984).

Recent experiments in visual neuroscience, using optical imaging to chart the activity of large populations of cells in fetal mice, have found patterns of activity, generated in retinal ganglion cells and projected to the superior colliculus, a subcortical brain structure, that simulate the optic flow patterns that guide the locomotion of the adults, infants, and animals in Gibson's experiments (Ge et al., Reference Ge, Zhang, Grizibis, Hamodi, Martinez Sabino and Crair2021). As in earlier research documenting spontaneous activity in the visual system prior to the onset of vision (e.g., Katz & Shatz, Reference Katz and Shatz1996), these activity patterns arise before the onset of visual experience. They suggest a mechanism by which innate capacities for depth perception might either emerge or be sharpened, prior to an animal's first encounters with a visible environment.

With this research, investigators of the development of visual perception provided the methods that fill this book. Throughout her work, Gibson argued that people and animals of all ages are motivated to explore their surroundings, and their exploratory behavior both reveals what they perceive and forecasts what they will learn (Gibson, Reference Gibson1969, Reference Gibson1991). Active looking and listening are likely the most important means for exploring and learning about the environment for young human infants who cannot yet grasp objects or locomote; since Gibson's seminal studies, exploratory looking has been extensively used to probe what infants see (Gredebäck, Johnson, & von Hofsten, Reference Gredebäck, Johnson and von Hofsten2010). Indeed, even unsuccessful attempts at exploration, such as newborn infants' failed attempts to contact a visibly moving object that stands just beyond their reach, reveal their perception of the object's distance, direction, and motion (von Hofsten, Reference von Hofsten1982). These behaviors also shed light on the workings of cognitive systems beyond perception.

2. Objects

Chapter 2 focuses on the research of many investigators probing infants' knowledge of objects, beginning with my personal trajectory. When my research began, I believed I was studying an intriguing aspect of visual perception: The origins of perception of the complete shapes of objects which – unless they are transparent – are never fully in view at any given time. Guided both by Gibson's work on surface perception and by the work of the neo-Gestalt psychologist, Albert Michotte (Michotte, Thines, & Crabbé, Reference Michotte, Thines and Crabbé1964), experiments in my lab probed infants' perception of objects that are partly hidden by other objects, objects that stand adjacent to or in front of other objects, and objects that move in and out of view. These studies focused on two exploratory behaviors – selective looking at and reaching for objects – that yielded converging findings. Their findings, however, did not accord either with the perceptual theories they were designed to test or with the research on surface perception that Gibson had pioneered. Here I give two examples.

First, studies led by Philip Kellman used Gibson's looking time method to probe 4-month-old infants' perception of the complete shapes of objects that are partly occluded. After infants' looking time to a center-occluded, straight rod had declined, the occluder was removed to reveal one connected rod or two short rods separated by a gap, on alternating trials (Fig. 2a). To our surprise, infants' looking rose equally for the two test displays, suggesting that they had not perceived the original rod either to end at the edge of the occluder or to be connected behind it. Further studies presenting partly occluded triangles, spheres, and cubes confirmed these findings, which were at odds both with Michotte's experiments on adults and with the Gestalt theories that inspired them.

Figure 2. Some strengths and limitations of young infants' representations of objects. (a) After a decline in looking time at an occlusion display (top figures), infants' looking times to two unoccluded displays (bottom figures) were compared. Red symbols indicate the direction of longer looking, indicative of a larger experienced change in the array. (b) While moving in a chair (i, bottom arrows), infants saw a stationary or moving rod that maintained a stable position in the infant's visual field (top dots and arrow). Their looking time (ii) was measured on the last six habituation trials and the first test trial presenting one connected rod (closed circles) and two rod fragments (open circles), undergoing the same motions as in habituation. Infants perceived a connected object when the rod itself moved and not when it was stationary, regardless of whether its image was displaced or stabilized in the infant's visual field. After Kellman and Spelke (Reference Kellman and Spelke1983) and Kellman et al. (Reference Kellman, Gleitman and Spelke1987). (c) When two familiar objects differing in shape, colors, texture, and affordances alternately appeared from behind a single screen (top four images) and then the screen was removed, infants treated the reappearance of one and of both objects (bottom two images) as equally novel or surprising. After Xu and Carey (Reference Xu and Carey1996).

In contrast, infants did perceive center-occluded objects as complete, connected bodies when their visible surfaces moved horizontally back and forth together, even as their centers remained hidden (Kellman & Spelke, Reference Kellman and Spelke1983). In these cases, however, infants' behavior still did not in accord with the Gestalt principle of “common fate,” because infants inferred a connection between the visible ends of the rod when the ends of the rod maintained a constant position in the infant's two-dimensional (2D) visible field and moved only in depth. Infants also inferred a connection between the ends of the rod when they and the rod were moved together, such that the rod's image remained centered in their visual field, while global changes in optic flow specified the changing positions of the infant and the object (Fig. 2b). These findings suggest that infants' perception of the object depended on the perceived 3D displacement of its perceived visible surfaces within the scene, not on the 2D displacement of its sensed image in a succession of static arrays (Kellman, Gleitman, & Spelke, Reference Kellman, Gleitman and Spelke1987; Fig. 2b).

These findings soon were joined by the findings of many other experiments on human infants, including newborns, performed by diverse investigators including Renee Baillargeon, Rachel Keen, Francesca Simion, Arlette Streri, and Claes von Hofsten. They were joined, as well, by experiments on newly hatched or controlled-reared chicks, conducted by Giorgio Vallortigara and his collaborators. Among other findings, young infants represented the persistence of objects that moved fully out of view, the solidity of visible or hidden objects that collided with other objects, and the continuity of object motion through space and time (see Baillargeon & Carey, Reference Baillargeon, Carey and Pauen2012, for a review).Footnote ¹ Newborn infants and newly hatched chicks also showed these abilities, in experiments suggesting a subcortical origin to their processing of object motion. For example, newborn infants perceived the complete shape of a partly occluded, moving rod only when the rod underwent rapid stroboscopic motion: A signature of processing in the superior colliculus (Regolin & Vallortigara, Reference Regolin and Vallortigara1995; Valenza, Leo, Gava, & Simion, Reference Valenza, Leo, Gava and Simion2006). I return to this finding below.

In all these studies, young infants failed to track objects under conditions in which older infants, children, and adults succeed. My second example focuses on one such situation. Fei Xu and Susan Carey presented 10-month-old infants with a large screen that hid two familiar objects – for example, a toy duck and a child's shoe – that alternately appeared and disappeared at opposite sides of the screen but were never visible at once (Fig. 2c, top four images). After viewing this event repeatedly, the screen was removed to reveal either both objects or just one object, on alternating trials (bottom images). Infants' looking times suggested no expectation that two objects would appear rather than one (Xu & Carey, Reference Xu and Carey1996). Their lack of surprise was striking, because infants showed signs of detecting and remembering the differing forms and functions of the two objects during the occlusion events: They looked longer when the two objects differed (like the duck and shoe) than when they were indistinguishable (two shoes). Moreover, when two objects appeared in alternation from behind two different screens with a visible gap between them, infants looked longer when the screens were removed to reveal just one object (Spelke, Kestenbaum, Simons, & Wein, Reference Spelke, Kestenbaum, Simons and Wein1995). Infants track objects based on their spatiotemporal properties – the continuous existence and motion of each object – but not in accord with the objects' differing forms and functions.

All these experiments suggest that infants' knowledge of objects is limited and depends on mechanisms that are distinct from those underlying perception of visible surfaces or object forms. Faced with this evidence, I once proposed, wrongly, that objects are not grasped by a perceptual system but by the only alternative of which I could conceive: A system of central cognition, like our systems of explicit reasoning about objects and their mechanical interactions (Spelke, Reference Spelke and Yonas1988). Research by Brian Scholl and others provided decisive evidence against this proposal (Scholl, Reference Scholl2001): Adults were found to share the representational system found in infants, and we use it both unconsciously and under constraints on attention and working memory that do not limit our conscious reasoning about objects. Scholl concluded that object representation depends on visual mechanisms; I inched toward the notion of core knowledge, but with little idea how core systems, interposed between perception and thought, might operate.

Recent research by the computational cognitive scientists, Tomer Ullman and Joshua Tenenbaum (Reference Ullman and Tenenbaum2020), provides a useful way of thinking about the core system of object representation in relation both to vision and to explicit thought. They propose that physical reasoning depends on a model of objects and their interactions like that of the physics engines – computer programs – used in interactive video games. At each time step, physics engines use an approximation to Newtonian mechanics to transform a representation of one 3D array of objects, each with a particular size, coarse shape, mass, position, and motion, into the 3D positions and motions of the objects at the next time step. By running such a process forward, infants may predict the future states of objects, including objects that move out of view. By inverting this process, using Bayesian inference, infants may recover object properties that cannot be directly seen, including an object's solidity, weight, and occluded location and motion (Smith et al., Reference Smith, Mei, Yao, Wu, Spelke, Tenenbaum and Ullman2021; Ullman, Reference Ullman2015; Ullman, Spelke, Battaglia, & Tenenbaum, Reference Ullman, Spelke, Battaglia and Tenenbaum2017; Ullman & Tenenbaum, Reference Ullman and Tenenbaum2020).

In contrast to core knowledge of object physics, visual perception may depend on a model of the world like that in the graphics engines used in many animated films. Graphics engines begin with a 3D representation of a scene and its light sources, and they generate 2D images of the scene from particular vantage points. Armed with a graphics engine and confronting the optic array that meets the eye, the visual system may invert this process by sampling different 3D scenes, simulating the images they would project at the eye, and using Bayesian inference to infer the most likely 3D surface arrangement that gave rise to the present 2D array, as Helmholtz once proposed. Graphics engines function together with physics engines in the production of animated video games, and cognitive systems with similar properties may function together in animals and humans to support the emergence, growth, and use of knowledge.

If Ullman and Tenenbaum are correct, core knowledge of objects takes the form of an internal model of the physical world, and it operates by simulating the sorts of interactions that occur as objects move and collide. Such a system would differ both from the visual system, whose internal models focus on the properties of light and of light-reflecting surfaces, and from the explicit systems of concepts and equations that students learn in physics classes. To my knowledge, Ullman and Tenenbaum have not discussed the processes by which the first model of objects might grow in the minds of inexperienced chicks and newborn infants, but I hazard a suggestion: Like the spontaneous, prenatal, subcortical neural activity that simulates patterns of optic flow, preparing fetal mice for their future encounters with visible surface layouts, there may be prenatal, subcortical neural activity that simulates the movements and interactions of spatiotemporally continuous, solid 3D bodies, preparing infants for their first encounters with the objects.

3. Places

As studies of infants' knowledge of objects proceeded, Barbara Landau and Henry Gleitman piqued my interest in another ability that likely builds on, and goes beyond, visual perception of the surrounding 3D layout. While studying the language learning of a young blind child, Landau observed that the child appeared to know the spatial layout of her home. To investigate the sources of this ability, she presented the child, and groups of sighted but blindfolded children, with a set of small-scale navigation tasks. First, the child was led from her mother to each of three objects occupying different locations in a single room, returning to her mother after visiting each object. Then the child was taken to one of the objects and encouraged to walk directly to another object. On arriving at the first object, she turned herself to face in the approximate direction of the next object and walked the approximately correct distance to get to it: a novel path for her (Fig. 3a). Landau concluded that the child's acts of navigation depended on a representation of the geometry of the 3D array: Like explorers, the child represented the directions and distances that she had traveled and inferred the final distance and direction to be traveled, in accord with basic theorems of Euclidean geometry (Landau, Gleitman, & Spelke, Reference Landau, Gleitman and Spelke1981; Landau, Spelke, & Gleitman, Reference Landau, Spelke and Gleitman1984).

Figure 3. Overhead view of displays from experiments testing the information used by young navigating children. (a) An oriented but blind child first walked between her mother and each of three objects (left figure, dotted arrows) and then was led to one object and encouraged to move independently to another object (left figure, solid arrows). The smaller figures show her paths of unaided (solid lines) and guided motion (dotted lines; Landau et al., Reference Landau, Gleitman and Spelke1981). (b) Children searched for a hidden object after they were turned slowly with eyes closed in a rectangular chamber, either repeatedly to become disoriented (top figures) or for one partial turn so as to remain oriented (bottom figures). In both conditions, children's eyes were closed while the two corner boxes were moved to new positions, dissociating the boxes' features from their relations to the bordering walls. Blue arrows indicate the hiding location and red stars indicate where children searched for the object. Disoriented children used room geometry to relocate the object, whereas oriented children used features of the boxes. Children showed these effects on the first trial, with no expectation that they might be disoriented, indicating that they encoded the box features in both conditions but used them to guide their search only when oriented. Drawing created by Kirsten Condry. In (c), dots of different sizes and spacing (left) influenced reorientation in a square room, though other pattern differences (e.g., dots vs. uniform gray) did not, as indicated by the figure on the right. Views of the room's corners suggest that differences in dot size and spacing produced an illusion of depth (left image), whereas other differences in patterning did not (e.g., right image). A further experiment confirmed the depth interpretation by revealing that the same differences in dot size and spacing did not elicit reorientation in a slightly elongated room with the larger dots on the more distant walls. After Lee, Winkler-Rhoades et al. (Reference Lee, Winkler-Rhoades and Spelke2012).

While we were studying the blind child, Cheng and Gallistel (Reference Cheng, Gallistel, Roitblat, Terrace and Bever1984) and Cheng (Reference Cheng1986) showed that the representations that guide navigation by animals (in their studies, rats) are limited, relative to those guiding the explicit planning of human explorers. When rats were disoriented after observing the location of food within a rectangular room containing multiple landmarks, they subsequently reoriented themselves and searched for the food in accord with its distance and direction from the walls bordering the room, but not in accord with the room's most obvious landmarks, such as a single white wall in otherwise black surroundings or an odor emanating from a source near the food's location. Faced with these findings, Cheng and Gallistel proposed that the rats' reorientation depended on a “geometric module.” Their hypothesis was stunningly corroborated by later research on mice, who use a nongeometric landmark to determine which of two rooms they are in but not to reorient themselves within that room, all within the same trial (Julian, Keinath, Muzzio, & Epstein, Reference Julian, Keinath, Muzzio and Epstein2015).

Young human infants do not navigate on their own, but infants of other species do, and experiments shed light on the origins and nature of the mechanisms that guide them. Chapter 3 discusses this research, as well as studies of young children. Linda Hermer and I adapted Cheng and Gallistel's methods for studies of 18-month-old toddlers, in hopes of discovering why humans navigate so much more flexibly than rats. To our astonishment, toddlers behaved like the rats: They navigated only by the distances and directions of the room's borders when they were disoriented, ignoring brightly colored walls, attractive toys, or distinctive decorations at the room's corners, though they used such landmarks when they were oriented (Hermer & Spelke, Reference Hermer and Spelke1994, Reference Hermer and Spelke1996; Fig. 3b). Later studies by Sang Ah Lee revealed that children's reorientation process was guided by the perceived distances and directions of the walls that bordered the floor: Children successfully reoriented in a square room whose patterning on the walls induced an illusion that the room was slightly rectangular, whereas they failed to reorient in a slightly rectangular room in which the same patterning induced an illusion that the room was square (Lee, Sovrano, & Spelke, Reference Lee, Sovrano and Spelke2012; Lee, Winkler-Rhoades, & Spelke, Reference Lee, Winkler-Rhoades and Spelke2012; Fig. 3c).

In contrast to children and rats, human adults used both geometry and landmarks when tested under similar conditions, and they mentioned the blue wall when asked why they had searched in a particular location. When the room was devoid of landmarks, however, many adults reported that they simply guessed which of the four corners contained the hidden object, even though they searched only corners at the appropriate distances and directions. Behavioral and neuroimaging experiments on navigation in virtual environments revealed that adults activate separate systems for navigating using layout geometry, and for piloting to a previously visited location using landmark objects. Only the latter system is associated with the deployment of attention to features of the room (Doeller & Burgess, Reference Doeller and Burgess2008; Doeller, King, & Burgess, Reference Doeller, King and Burgess2008), although the system for representing places is activated only when people and animals plan and carry out their own acts of navigation, not when they are passively moved (e.g., Javadi et al., Reference Javadi, Emo, Howard, Zisch, Yu, Knight and Spiers2017).

Thus, the geometric representations guiding reorientation are modular, in Fodor's (Reference Fodor1983) sense, for humans as well as rodents. Even as adults, we reorient ourselves unconsciously, guided by a tiny subset of the environmental features that we perceive and remember. Modular cognitive systems, Fodor argued, are the antithesis of belief systems: He thought they were “input systems” like vision, although navigation draws on capacities for learning, memory, and action planning. The second half of the chapter focuses on the remarkable convergence of these findings with the findings from studies of navigation and place representations in the brains of animals and people (e.g., O'Keefe & Burgess, Reference O'Keefe and Burgess1996), and from studies assessing the efficacy and limits of artificial navigation systems in autonomously moving robots with no navigation aids (e.g., Thrun, Reference Thrun, Lakemeyer and Nebel2002).

The studies from cognitive and computational neuroscience strengthen the evidence for an encapsulated navigation system that operates automatically and unconsciously, but beyond the limits of perception. Navigation depends on the hippocampus: A structure that is intimately involved in action planning, learning, and conscious episodic memory. It is fostered, moreover, by generative processes for simulating and comparing different possible routes through an environment as an aid to action planning (Foster, Reference Foster2017). As adults, we sometimes plan actions consciously, but research using functional brain imaging reveals processes of mental simulation that are far more rapid and unconscious, both in animals and in human adults. When adults rest in the middle of a real or virtual navigation task (or, indeed, a nonspatial task requiring “navigation” through a complexly structured array of task conditions), they report no awareness of the rapid simulation processes that neuroimaging reveals, but those processes are predictive of improved performance on subsequent task sessions (e.g., Javadi et al., Reference Javadi, Emo, Howard, Zisch, Yu, Knight and Spiers2017; Liu, Dolan, Kurth-Nelson, & Behrens, Reference Liu, Dolan, Kurth-Nelson and Behrens2019; Shuck & Niv, Reference Shuck and Niv2019). Research on navigation and spatial memory provides the strongest evidence for a system that combines some of the features of perceptual systems with some of the features of belief systems. It also provides evidence for unconscious processes of mental simulation, both in adults and in inexperienced animals (e.g., Farooq & Dragoi, Reference Farooq and Dragoi2019).

4. Number

Chapter 4 focuses on infants' sensitivity to number. One source of numerical information comes from the core object system: The representations that support tracking objects over occlusion and reasoning about their interactions are leveraged by older children and adults to support rapid determination of the exact number of objects in an array, up to a capacity limit of about three (Alvarez & Franconeri, Reference Alvarez and Franconeri2007). This chapter focuses primarily on an earlier emerging source of numerical information: A system for representing, imprecisely, the relative numerosities of sets of objects or events, encountered in any perceptual modality, and for combining or dividing these sets in accord with the operations of arithmetic. Because this system operates on sets with ratio-limited accuracy, it has been dubbed “the approximate number system,” or ANS.

The ANS is present and functional in newborn infants, who selectively look at visual–spatial arrays that roughly match the number of sequential sounds in a simultaneous auditory sequence (Izard, Sann, Spelke, & Streri, Reference Izard, Sann, Spelke and Streri2009; Fig. 4a). At birth, it is highly imprecise: Newborn infants match arrays of 4 or 12 visible objects to auditory sequences of the same number of syllables, but not arrays and sequences that contrast four with eight objects and syllables. Six-month-old infants succeed with the latter 2:1 ratio (Xu & Spelke, Reference Xu and Spelke2000) and can mentally transform visual arrays in accord with the operations of addition and subtraction: If an array of 10 objects is hidden behind a screen and then five of the objects move into view, infants expect approximately five objects to remain behind the screen, looking longer if the raising of the screen reveals 10 objects (McCrink & Wynn, Reference McCrink and Wynn2004). The precision of the ANS increases throughout the first year (e.g., Xu & Arriaga, Reference Xu and Arriaga2007) and beyond (Halberda, Mazzocco, & Feigenson, Reference Halberda, Mazzocco and Feigenson2008), sharpens over the course of math instruction (Piazza, Pica, Izard, Spelke, & Dehaene, Reference Piazza, Pica, Izard, Spelke and Dehaene2013), and varies with children's and adults' proficiency at mathematical learning and reasoning (Halberda et al., Reference Halberda, Mazzocco and Feigenson2008; Halberda, Ly, Wilmer, Naiman, & Germine, Reference Halberda, Ly, Wilmer, Naiman and Germine2012).

Figure 4. Displays for experiments testing sensitivity to number in human infants and adults. In (a), newborn infants were presented with sequences of syllables (top) and visual arrays that either matched or mismatched the sequences in number (middle). Infants looked longer at the visual arrays that corresponded in number to the auditory sequences, when the two numbers differed by a 3:1 ratio (bottom: left and middle bars). In (b), adults were presented with sequences of dot arrays presenting small numbers (1–3) and large numbers (8–24) on separate blocks of trials. After viewing four arrays presenting the same number (diagonally arranged images and arrows), adults saw a fifth array presenting one of three numbers (vertically arranged images). In the small-number block, adults' response to the last array was larger for larger numbers (bottom left figure) and was unaffected by changes in number (bottom right figure). In the large-number block, adults showed the opposite pattern: No effect of numerical sizes (bottom left figure) and a consistent effect of numerical changes (bottom right figure). After Izard et al. (Reference Izard, Sann, Spelke and Streri2009) and Hyde and Spelke (Reference Hyde and Spelke2009).

It is difficult to determine either the innateness of the ANS or its core function, because it applies to sets of diverse entities. Research by Fei Xu, conducted with considerably older infants, suggests that it functions, in part, to support infants' learning about the statistical properties of things and events. In one series of studies (Xu & Garcia, Reference Xu and Garcia2008), infants who viewed a transparent box containing mostly white but a few red balls looked longer if a person who blindly fished for balls in the box retrieved mostly red ones. Numerical estimation processes may foster infants' learning and reasoning in uncertain and variable environments.

Studies of animals reveal that tasks exercising the ANS elicit neural activity in homologous regions of the parietal cortex of adult monkeys and humans, suggesting that the ANS is not unique to our species (Nieder & Dehaene, Reference Nieder and Dehaene2009). This brain region also is activated in human infants who view numbers presented as dots (e.g., Hyde, Boas, Blair, & Carey, Reference Hyde, Boas, Blair and Carey2010); in children and adults who compare arrays of dots or number symbols (Cantlon et al., Reference Cantlon, Libertus, Pinel, Dehaene, Brannon and Pelphrey2009) and solve symbolic arithmetic problems (Amalric & Cantlon, Reference Amalric and Cantlon2022; Dehaene, Spelke, Pinel, Stanescu, & Tsivkin, Reference Dehaene, Spelke, Pinel, Stanescu and Tsivkin1999); and in professional mathematicians who consider the truth or meaningfulness of verbally presented statements from advanced mathematical (but not nonmathematical) fields (Amalric & Dehaene, Reference Amalric and Dehaene2016). Adults with damage to this brain region show deficits in performance of a variety of numerical tasks, including mental arithmetic (Dehaene & Cohen, Reference Dehaene and Cohen1997). All these findings suggest that the ANS contributes to mathematical reasoning.

A wealth of evidence shows that representations of exact small numbers and of approximate large numbers are represented differently and compete for attention. Evidence for this competition comes from studies of infants and adults whose brain activity was measured by electroencephalographic (EEG) recordings of neurally generated activity at their scalps. During the EEG session, the participants passively viewed blocks of trials presenting short sequences of dot arrays that varied in numerical size, some of which presented changes in number. Despite many years of experience with the natural number concepts at the center of primary school mathematics, the adults showed different EEG attentional responses when shown arrays of 1, 2, and 3 dots and arrays of 8, 16, and 24 dots (Hyde & Spelke, Reference Hyde and Spelke2009; Fig. 4b), as did the infants (Hyde & Spelke, Reference Hyde and Spelke2011). At both ages, EEG signatures of attention increased in amplitude as numbers increased from 1 to 3, suggesting that objects in the small-number arrays were represented as distinct individuals. In contrast, EEG responses did not vary in amplitude as numbers increased from 8 to 24, suggesting that the objects in each large-number array were attended to as a single group. Consistent with that suggestion, changes in number from one array to the next elicited EEG responses in the large-number condition but not the small-number condition, both in adults and in infants: Only the large numbers were represented spontaneously as a set of a certain numerical size.

Further studies by Dan Hyde and Justin Wood asked why the ANS is not activated by numerical changes in small-number arrays: Does it only apply to numbers of four or more, or does it apply to smaller numbers but face inhibition from the system for representing objects? To distinguish these possibilities, Hyde and Wood presented adults with one to three dots under conditions that prevented them from attending to each dot individually: Now the adults showed the characteristic ANS responses to changes in number for sets of all sizes (Hyde & Wood, Reference Hyde and Wood2011). Their findings provide evidence that representations of objects and sets compete for attention: One can attend to the trees or the forest but not to both at once.

Further observations by Hyde suggest that ANS representations are activated unconsciously. In his original experiments with adults, conducted in a lab focused on infants and children, adults were asked what they thought the study was about. Some participants guessed that it focused on dot patterns but few mentioned number. Although the core object and number systems compete for attention, they appear to be activated automatically and unconsciously when adults attend to small- and large-number arrays, respectively. Earlier research by Lionel Naccache and Stanislas Dehaene, presenting numbers as Arabic numerals and probing their representations in adults using behavioral experiments, functional magnetic resonance imaging (fMRI) and EEG, provided extensive evidence for representations of number that are unconscious yet demanding of attention to the entities (in their case, briefly presented number symbols; Naccache, Blandin, & Dehaene, Reference Naccache, Blandin and Dehaene2002).

Like core object and place representations, therefore, ANS representations appear to arise from a core cognitive system that is distinct from our explicit integer concepts. Does the ANS support children's learning of mathematics? To date, evidence for effects of short-term training with numerical tasks that activate the ANS has shown some positive effects on children's subsequent symbolic math performance (Hyde, Khanum, & Spelke, Reference Hyde, Khanum and Spelke2014; Khanum, Hanif, Spelke, Berteletti, & Hyde, Reference Khanum, Hanif, Spelke, Berteletti and Hyde2016; Park, Bermudez, Roberts, & Brannon, Reference Park, Bermudez, Roberts and Brannon2016). Nevertheless, preschool activities that exercise only the ANS have led to no long-term enhancement of children's subsequent learning of mathematics in school (Dillon, Kannan, Dean, Spelke, & Duflo, Reference Dillon, Kannan, Dean, Spelke and Duflo2017). The core number system therefore is not the only cognitive system needed for learning of mathematics, but it likely contributes to mathematical reasoning and learning.

5. Core knowledge

Studies of early-emerging knowledge of objects, places, and number provide the clearest evidence for the existence and properties of core cognitive systems whose abstract content supports exploration and learning. For objects, the physical properties of cohesion, continuity, and solidity govern objects' movements and interactions and support learning about objects' forms and functions. For places, the geometric properties of distance and direction support learning about the navigable paths over the ground that connect out-of-view places. For number, the properties of order and composition likely support statistical learning about predictable objects, actions, and events, as well as children's learning of primary school mathematics.

Core systems have further properties: They are ancient, emerge early in life, and are invariant over later development, as evidenced by research revealing the same domain-specific abilities, limits, and signature patterns of neural activity in each core domain, across diverse species and ages. Core systems also are impervious to our explicit beliefs and are activated automatically and unconsciously when we attend to entities in their domain. All may place demands on attentional resources, if the core systems emerge and function as generative models that simulate either the behavior of entities in their domain (for objects and numbers) or the actions that can be performed in the domain (for places, acts of navigation).

In Chapter 5, I propose that these properties are related: Any cognitive system that has some of them is likely to have them all. An ancient system that first emerged in highly distant ancestors is likely to center on abstract content, because it had to be applicable to the diverse environments that the descendants of that last common ancestor came to inhabit. Moreover, a system of abstract concepts that supports exploration and learning in a broad range of habitable environments, for animals that vary in size and behave in different ways, is likely to be useful for people of all ages, whatever their circumstances and however those circumstances change with age and experience. It is likely, therefore, to function in all human cultures. To preserve its functionality over evolutionary time scales, unimpeded by mechanisms of top-down inhibition from later-emerging brain systems, such a system should be activated automatically, independently of volitional control or conscious access. To function in the service of goal-directed behavior, it should be activated when a person or animal attends to entities in its domain.

Based on these considerations, I consider, in the next three chapters, whether the human mind contains other systems with this constellation of properties. I propose that three more core systems emerge in infancy and guide learning about entities that many animals must contend with. The system discussed in Chapter 6 has been least studied: It captures the forms and functions of objects of specific kinds, perhaps especially the living, inanimate beings (like trees) that grow in all habitable environments, whose distinctive forms allow for their recognition and categorization, and whose distinctive affordances for food and shelter bear on people's and animals' survival. For brevity, I won't discuss that system here. The other proposed systems focus, respectively, on animate beings that cause their own motion and plan efficient actions to achieve valued goal states, and on social beings who engage with one another, share experiences of attention and emotion with their social partners, and form enduring bonds.

6. Agents

Chapter 7 focuses on infants' knowledge of beings who sense their surroundings and generate their own movements. Like the movements of objects, the movements of agents are physically constrained: Agents cannot pass through walls or teleport. Unlike objects, however, agents' actions have unique and interconnected properties. Infants are sensitive to highly abstract properties of the actions of people, animals, and animated characters: They view their actions as intentional, goal-directed, perceptually guided, efficient, and causal. These properties are connected: Given evidence that an agent has caused a change in an inanimate object on contact, 3-month-old infants expect its future actions to be efficient (Liu, Brooks, & Spelke, Reference Liu, Brooks and Spelke2019; Skerry, Carey, & Spelke, Reference Skerry, Carey and Spelke2013; after Gergely, Nádasdy, Csibra, & Bíró, Reference Gergely, Nádasdy, Csibra and Bíró1995; Fig. 5a), goal-directed (Woo, Liu, & Spelke, Reference Woo, Liu and Spelke2021; after Woodward, Reference Woodward1998), and perceptually guided (Choi, Mou, & Luo, Reference Choi, Mou and Luo2018; after Luo & Johnson, Reference Luo and Johnson2009). Older infants infer that such agents value more highly the goals for which they undertake more costly actions (Liu, Ullman, Tenenbaum, & Spelke, Reference Liu, Ullman, Tenenbaum and Spelke2017; after Jara-Ettinger, Gweon, Schulz, & Tenenbaum, Reference Jara-Ettinger, Gweon, Schulz and Tenenbaum2016; Fig. 5b).

Figure 5. Displays for experiments testing infants' expectations that agents will act efficiently to achieve valued goals. (a) Three-month-old infants repeatedly viewed an agent who reached for an object over a barrier and, in one condition, caused the object to light up on contact (top right image), until their looking time to this event declined. After removal of the barrier (middle and bottom right images), infants looked longer at the same indirect motion, now inefficient. This effect was abolished when the same videos were altered so that the object changed its state with no contact with the finger (top left images and data figure, after Liu, Brooks et al. (Reference Liu, Brooks and Spelke2019). (b) Ten-month-old infants were presented with an agent who took a higher-cost action for one target than for the other (top four images). After looking time to these events had declined, they viewed a display in which the costs of reaching the two targets were equal. Infants looked longer when the agent approached the target for which it had taken a lower-cost action (bottom right image), providing evidence that they expected the agent to approach the target for whom it had taken the more costly action. After Liu et al. (Reference Liu, Ullman, Tenenbaum and Spelke2017).

As in other domains, infants' knowledge of agents is limited; I mention two limits here. First, although newborn infants expect agents to engage in biological motion, they exhibit no specific knowledge of the attributes that distinguish the bodies of their own species from those of others: Newborn human infants are equally attentive to the biological motion of a human and a hen (Simion, Regolin, & Bulf, Reference Simion, Regolin and Bulf2008; Vallortigara, Regolin, & Marconato, Reference Vallortigara, Regolin and Marconato2005). Young infants also have no knowledge of the most likely goals of the actions that human agents perform. Although older infants expect acts of reaching to be directed to objects rather than places (Woodward, Reference Woodward1998), in contrast to acts of locomotion (Hamlin, Wynn, & Bloom, Reference Hamlin, Wynn and Bloom2007), 3-month-old infants lack these expectations (Sommerville, Woodward, & Needham, Reference Sommerville, Woodward and Needham2005), though they view agents' actions as goal-directed and quickly learn an actor's specific goal when given appropriate evidence (Woo et al., Reference Woo, Liu and Spelke2021). These findings suggest that an ancient system, common to humans and other animals and centering on abstract, general properties that apply to all actions, supports young infants' learning about people's distinctive appearance, actions, and goals.

Second, young infants display little understanding of social actions, like cooperation or shared attention to objects. A wealth of evidence (to which I return below) suggests that this understanding arises at the end of the first year, but for younger infants, it is conspicuously absent. For example, if 10-month-old infants first view two people who stand side by side and turn to face each other as they converse, they, like adults, expect that if the people's lateral positions reverse during the conversation, each person will alter their movement so as to face the other again. Younger infants, in contrast, exhibit no such expectation on viewing this common, goal-directed social act: They are equally unsurprised if a person answers a friendly overture by turning toward or away from the person to whom he is speaking (Beier & Spelke, Reference Beier and Spelke2012). This finding and others suggest that core knowledge of agents applies to people's causal actions on objects but not to their social engagements with other people. Consistent with that suggestion, core knowledge of agents may take the form of a generative model that simulates the behavior of others who act efficiently to bring about high-value goal states in partially observable physical, but not social, environments (Baker, Jara-Ettinger, Saxe, & Tenenbaum, Reference Baker, Jara-Ettinger, Saxe and Tenenbaum2017; Liu et al., Reference Liu, Ullman, Tenenbaum and Spelke2017; Ullman & Tenenbaum, Reference Ullman and Tenenbaum2020). Only later in the first year might infants incorporate social motives into their analysis of agents' actions (Brune & Woodward, Reference Brune and Woodward2007).

One set of situations, however, does appear to elicit young infants' understanding of social actions: Situations in which one agent helps another agent to achieve its goal (Hamlin et al., Reference Hamlin, Wynn and Bloom2007). When a person or puppet repeatedly tries and fails to complete an action (such as climbing a hill or opening a box) in view of two other agents, one who helps the actor and one who does not, infants as young as 3 months subsequently tend to look at the agent who provided help, and 6-month-old infants tend to reach for that agent. Infants' reaching and looking have been interpreted as reflecting a social preference for the helpful character, but I believe the research leaves other interpretations open. First and foremost, looking and touching are exploratory behaviors reflecting interest or curiosity and guiding learning: Indeed, their exploratory function may be primary even in social contexts, like bars and parties, where they signal desires to get to know someone. Thus, young infants may touch or look more to a previously helpful character not because its motives were prosocial, but because it produced a more interesting and potentially informative outcome: It ended the sequence of failed actions and freed the protagonist to do something new. Future research on young infants, focused on more specific behavioral or neural signatures of exploration and social engagement, could distinguish these accounts.

7. Social cognition

Chapter 8 introduces the last system of core knowledge, focused on social beings, engagements, and relationships. Social cognitive development has long been studied, but no clear consensus has emerged regarding the origins and nature of social knowledge. In this chapter, I hypothesize that a core system like those described in previous chapters underlies infants' developing knowledge of the mental states, attributes, and social relationships that connect people to one another and to the infant.

I first consider the overt social behaviors that engage young infants. When a real or pictured person looks at them, infants tend to look back (Farroni, Csibra, Simion, & Johnson, Reference Farroni, Csibra, Simion and Johnson2002) and mirror the person's movements of attention (Field, Woodson, Greenberg, & Cohen, Reference Field, Woodson, Greenberg and Cohen1982; Hood, Willen, & Driver, Reference Hood, Willen and Driver1998). More strikingly, infants also tend to imitate the person's oral gestures and expressions of emotion (Field et al., Reference Field, Woodson, Greenberg and Cohen1982; Meltzoff & Moore, Reference Meltzoff and Moore1977; Fig. 6a). These behaviors emerge in the first months and are connected: For example, infants imitate and follow the gaze shifts of a face that begins with direct gaze, but not a face that begins with closed eyes or averted gaze. All three behaviors also are exhibited by infant apes and monkeys (e.g., Deaner & Platt, Reference Deaner and Platt2003; Ferrari, Paukner, Ionica, & Suomi, Reference Ferrari, Paukner, Ionica and Suomi2009; Mendelson, Haith, & Goldman-Rakic, Reference Mendelson, Haith and Goldman-Rakic1982; Myowa, Reference Myowa1996), who tend to affiliate with people who imitate them (Paukner, Suomi, Visalberghi, & Ferrari, Reference Paukner, Suomi, Visalberghi and Ferrari2009). Attention following, imitation, and preferences for imitators also are exhibited by human adults (e.g., Chartrand & Bargh, Reference Chartrand and Bargh1999; Driver et al., Reference Driver, Davis, Ricciardelli, Kidd, Maxwell and Baron-Cohen1999), although our imitative actions tend to be unconscious, and we only respond positively to people who imitate us when we are unaware that we are being imitated. Finally, these behaviors support learning: 6-week-old human infants, who attend to a person who looks at them and either opens his mouth or protrudes his tongue, will attempt to reproduce the behavior a day later if the same person (but not a different person) faces them without moving (Meltzoff & Moore, Reference Meltzoff and Moore1994/2002), as do rhesus macaques (Paukner, Ferrari, & Suomi, Reference Paukner, Ferrari and Suomi2011).

Figure 6. Experiments testing infants' use of social imitation to identify new members of their social world. Twelve-month-old infants, who viewed either their own parent or a stranger (the parent of another infant) imitating one of two puppets, subsequently looked to the puppet who was imitated by their parent, but not by the stranger, when a voice that was synchronized with both puppets called to them by name. After Thomas, Saxe et al. (Reference Thomas, Saxe and Spelke2022).

These findings suggest a hypothesis concerning the interconnected, abstract concepts on which a system of core social knowledge might center. Infants may respond in kind to the oral gestures, movements of attention, and emotional states of another person to signal their own engagement with the person and their motivation to share actions and experiences. Core social knowledge therefore may center on a conception of people as individuals with mental experiences of attention and emotion, who engage with one another and with the infant to share these experiences, and whose relationships persist through time.

The spontaneous behaviors of very young infants are hard to study, however, and harder to interpret. Although young infants move their attention in the direction of a potential social partner's changing gaze, this shift of attention guides their own eye movements only if the face that elicits it immediately disappears: An event that never occurs in real interactions. Young infants' own acts of imitation are subtle and sometimes appear after long and variable delays, making them difficult to leverage for further systematic study (see Meltzoff et al., Reference Meltzoff, Murray, Simpson, Heimann, Nagy, Nadel and Ferrari2018; Oostenbroeck et al., Reference Oostenbroeck, Suddendorf, Nielsen, Redshaw, Kennedy-Costantini, Davis and Slaughter2016). Although neonatal imitation has now been documented in newborn humans, apes, and monkeys, only oral movements – mouth opening, lip pursing, and tongue protrusion – have provided reproducible evidence for imitation in very young members of these species, possibly because the youngest primate infants, human and nonhuman, tend to engage primarily with close relatives, and oral movements resulting in saliva sharing are associated with close relationships at all ages (Thomas, Woo, Nettle, Spelke, & Saxe, Reference Thomas, Woo, Nettle, Spelke and Saxe2022). Finally, neonatal imitation disappears after the first few months, possibly because infants with 3 or 4 months of social experience have learned who their close social partners are, reducing their receptiveness to a new, wholly unfamiliar person who purses their lips at them.

To overcome these limits, most of the research discussed in this chapter probes the early development of social knowledge by placing infants in the role of third-party observers of social interactions, and it analyzes infants' patterns of looking at the interactions as evidence for their expectations, surprise, or interest in the events and participants. In this précis, I focus only on infants' responses to events involving two socially interactive behaviors – imitation and comforting – and their learning about the social connections between the individuals who exhibit them.

Experiments by Lindsey Powell and Heather Kosakowski presented 4-month-old infants with videos of two live people or animated characters who responded to the action of a third person or character, one by imitating that character and the other by responding with a different behavior. When the responding characters subsequently appeared without the target character, infants looked longer at the imitator, suggesting greater visual interest in characters who engage with others by sharing their actions (Kosakowski, Powell, & Spelke, Reference Kosakowski, Powell and Spelke2016; Powell & Spelke, Reference Powell and Spelke2018). Infants also expected imitators to approach those whom they had imitated (Powell & Spelke, Reference Powell and Spelke2017).

Further research provides evidence that older infants learn about the relationships connecting the social beings whose interactions they observe. In one study, Annie Spokes presented infants with animated events involving two small characters who emitted baby cries, placed side by side below three large characters with adult voices who responded to those cries with comforting. The adult characters on the left and right responded only to the baby on their side of the display, and the adult in the center responded to one of the two babies as well. At test, only the adult characters appeared, and the central character alternately approached and danced with each of the side characters. Although the adults had not previously interacted directly, infants expected interactions between the two adults who had comforted the same baby: They inferred a social connection between two characters based on their interactions with a third party (Spokes & Spelke, Reference Spokes and Spelke2017).

In recent studies, infants leveraged these abilities to learn about the members of their own social world. Ashley Thomas presented infants with videos of their own parent or an unfamiliar adult (the parent of another infant in the study) who responded to two puppets by imitating one puppet and not the other. These events were followed by videos in which the two puppets faced the infant in the parent's absence and simultaneously moved their mouths while a centrally located voice called to the baby by name (Fig. 6b). On hearing their name, infants looked to the puppet who had been imitated by their parent. Because no such effect occurred in the session with the unfamiliar adult, these findings provide evidence that infants inferred that the call to them came from the puppet whom their parent had imitated. Variations on this method showed that infants also expected a puppet who first demonstrated a relationship with them (by calling to them by name) to respond to the distress of their parent, but not to the distress of a parent of a different infant (Thomas, Saxe, & Spelke, Reference Thomas, Saxe and Spelke2022).

These findings provide evidence, I believe, for a core system of social knowledge. Like core knowledge in other domains, core social cognition is limited; I note two limits here. First, although newborn infants are acutely sensitive to people who look at them with direct gaze, they fail to distinguish people's faces from the faces of other animals, for they attend equally to the direct gaze of a human, a monkey, and a sheep (Pascalis, de Haan, & Nelson, Reference Pascalis, de Haan and Nelson2002). Second, core social knowledge is suppressed if the parties to a social interaction attend to and act on objects. At 9 months, for example, infants smile at people who imitate their otherwise purposeless gestures but not at people who imitate their actions on objects (Agnetta & Rochat, Reference Agnetta and Rochat2004; Stern, Reference Stern1985). Moreover, 10-month-old infants accept objects more readily from people whose speaking or singing suggests that they are known social partners (Kinzler, Dupoux, & Spelke, Reference Kinzler, Dupoux and Spelke2007; Mehr & Spelke, Reference Mehr and Spelke2017), but younger infants accept objects offered by anyone, regardless of their social identity or intentions, and show no understanding of gift-giving (Gordon, Reference Gordon2003). This pattern is striking, because the people whom the infant knows best both act on objects and engage with one another, and these events frequently occur together. Young infants, however, appear to view any given movement by a person either as a goal-directed action or as a social gesture, but not as simultaneously social and object-directed.

Several suggestions follow from these findings. First, core representations of people as social beings and as agents likely compete for attention in young infants, and perhaps in adults as well (Gray, Gray, & Wegner, Reference Gray, Gray and Wegner2007; Knobe & Prinz, Reference Knobe and Prinz2008; Weisman, Dweck, & Markman, Reference Weisman, Dweck and Markman2017). If so, then core social cognition may take the form of a generative model that simulates the shareable actions and experiences of known, individual people and the relationships that connect them. Infants' learning about their own social network has barely begun to be studied, however, and, to my knowledge, no detailed model of core social knowledge has been proposed. Second, infants' knowledge of people appears to undergo considerable changes toward the end of the first year. What might these changes be, and how and why might they occur? Such questions will take center stage in the successor to this book, but the last chapter of What Babies Know offers a preview of the ideas that I find most promising. To get there, however, the penultimate chapter turns to research on infants' learning of their native language.

8. Language

Infants' knowledge of language differs in critical ways from their knowledge of things, places, and people. Language is unique to humans and is learned slowly: Although infants begin to distinguish the sounds and partial meanings of highly frequent words in the first months (Bergelson & Swingley, Reference Bergelson and Swingley2012; Tincoff & Jusczyk, Reference Tincoff and Jusczyk1999), they don't confidently master the meanings of common words until about 14 months of age (Bergelson, Reference Bergelson2019; Bergelson & Aslin, Reference Bergelson and Aslin2017). At birth, infants' knowledge of objects, places, and people is the same everywhere, but even newborn infants, drawing on experiences of language in the womb, respond differently to speech in different languages (Mehler et al., Reference Mehler, Juszyk, Lambertz, Halsted, Bertoncini and Amiel-Tison1988). Core knowledge endures throughout life, but capacities for learning new languages decline with age (Hartshorne, Tenenbaum, & Pinker, Reference Hartshorne, Tenenbaum and Pinker2018; Johnson & Newport, Reference Johnson and Newport1989). Finally, infants' core knowledge is manifest in their exploration, actions, and interactions with others. During most of the first year, in contrast, infants' language learning has little effect on their overt behavior. Much of the evidence for this learning comes from experiments that test for neural signatures of attention or surprise, elicited by changes in the structures or meanings of phrases (e.g., Friederici, Friedrich, & Christophe, Reference Friederici, Friedrich and Christophe2007), or from behavioral experiments finding tiny increases in infants' looking to named objects or events (e.g., Bergelson & Swingley, Reference Bergelson and Swingley2015) or their head-turning toward speech with natural prosody and recognizable words (e.g., Jusczyk, Reference Jusczyk1997) and away from repetitive speech with no prosody or meaning (e.g., Saffran, Aslin, & Newport, Reference Saffran, Aslin and Newport1996; see Black & Bergman, Reference Black and Bergman2017).

Chapter 9 covers these and other aspects of infants' language learning, but in this précis I consider just one distinction that infants master, between content words (e.g., the nouns, verbs, and adjectives that refer to things, events, and properties) and function words (e.g., the pronouns, determiners, auxiliaries, and part-words, like the past tense -ed, that signal relations between the things or events that content words designate). Function and content words differ in frequency (function words are more frequent), number (languages have many more content words), and phonological properties (content words tend to be spoken with greater stress and function words with shorter, reduced vowels).

Based on these properties, infants distinguish function from content words from the beginning, independently of their prenatal language exposure. In one study, for example, newborn infants were played a string of English function (or content) words, followed by new function and content words that were played in alternation. Based on the phonological properties that distinguish these categories in English, infants generalized to new words in the same category, and they reacted with interest to words in the opposite category, regardless of whether they had been exposed, in utero, to English or to a different language (Shi, Werker, & Morgan, Reference Shi, Werker and Morgan1999). Infants therefore distinguish between words in these two highly abstract categories. In contrast, infants do not learn the meanings or syntactic categories of any particular content or function word in their own language until the second year: For example, French- and German-learning infants below 14–18 months fail to expect that a native-language phrase beginning with a determiner (like the French or German counterpart of the) will include a noun, whereas a phrase beginning with a pronoun (like the counterpart of he) will include a verb (de Carvalho, He, Lidz, & Christophe, Reference de Carvalho, He, Lidz and Christophe2019; Hohle, Wiessenborn, Kiefer, Schulz, & Schmitz, Reference Hohle, Wiessenborn, Kiefer, Schulz and Schmitz2004).

Although infants master the specific function words in their language slowly, they use their ability to recognize function words in any language to learn the patterns in which these words or part-words co-occur. In English, we say “The girl is dancing” and “The girl has danced,” but not “The girl is danced” or “The girl has dancing.” Remarkably, German-learning 4-month-old infants picked up on these relationships in a single session, when listening to sentences like these in an unfamiliar language (Italian): After a brief period of familiarization with the grammatical Italian phrases, EEG recordings showed a characteristic incongruity response to ungrammatical combinations of the functional morphemes, as did the EEGs of adult speakers of Italian. In contrast, adult speakers of German, tested under the same conditions as the German-learning infants, failed to respond to the relationships between the Italian function words (Friederici, Mueller, & Oberecker, Reference Friederici, Mueller and Oberecker2011). The identification and proper use of function words is a formidable task for adult language learners.

Although the distinction between function and content words is universal, the typical ordering of these words varies across languages in ways that reflect a fundamental grammatical property of the language: The ordering of heads (like the subject of a sentence) and complements (like its predicate) in phrases. In some languages, like English and French, heads precede their complements (e.g., he eats and a steak). In some languages, like Japanese, this ordering is reversed, and in others, like German, both orderings occur in different sorts of phrases; children therefore must learn which ordering(s) their own language uses. The pairing of heads and complements with function and content words is not perfect: No function words appear in some phrases, and no content words in others (e.g., John eats and another one). By 6–8 months, however, infants have learned how function and content words typically are ordered in their native language, and they quickly learn, using diverse cues, to find and order function and content words in a new, artificial language.

Evidence for this ability comes from a spectacular series of experiments, begun by Jacques Mehler and pursued by his collaborators and descendants, who have studied infants learning diverse languages (Gervain, Nespor, Mazuka, Horie, & Mehler, Reference Gervain, Nespor, Mazuka, Horie and Mehler2008). The experiments used a complex artificial language learning paradigm that I lack the space to describe here, based on the assumption that infants who have learned how heads and complements are ordered in their native language will expect a new artificial language to follow the corresponding ordering of function and content words, if no perceptible cues to their ordering are available. Indeed, infants quickly learned, in a single session, to categorize the function and content words in the artificial language based only on the differing numbers and frequencies of words in each category, and they ordered the function and content words in the artificial language in accord with the ordering of heads and complements in their native language. This finding provides evidence that infants have learned a central aspect of the syntax of their native language by about 7 months of age.

The words in the artificial language were spoken with uniform timing and intonation in the studies just described, but in further studies, additional information for the boundaries of phrases, based on subtle variations in pitch or syllable length that occur in some, but not all, of the world's languages, was added to the sequences in the artificial language. Infants used this information to determine the head–complement order of the artificial language, even when that order was opposite to the order of heads and complements in their native language, and even when the cues to phrasal boundaries in the artificial language were not present in the infant's own language (Bernard & Gervain, Reference Bernard and Gervain2012). In further studies, infants whose family members spoke two languages with opposite head–complement orders used these cues to infer the ordering of function and content words in each of their languages (Gervain & Werker, Reference Gervain and Werker2013).

In brief, long before infants have confidently mastered the sounds or meanings of any words, they have begun to learn about the fundamental abstract structure and ordering of the words and phrases in their native language or languages. Language learning does not proceed from surface properties, like specific phonemes or words, to deeper, abstract properties, like syntactic structure and morphology; indeed, it appears to proceed in the opposite direction: Younger infants are more attuned to the abstract and general properties of language – properties that adults are not aware of – and only later learn what particular sounds distinguish one word from another (Werker, Reference Werker1989). In these respects, the representations that support language learning resemble the representations of core knowledge.

Over the course of the first year, infants' language learning interacts with core knowledge in interesting ways. First, infants only learn languages that are spoken by members of their social world: They don't learn from the radio or from videos of an adult speaking directly to a different infant (Kuhl, Tsao, & Liu, Reference Kuhl, Tsao and Liu2003). Nevertheless, infants learn language even in cultures in which adults do not speak to them, likely based on the conversations they overhear (Cristia, Dupoux, Gurven, & Stieglitz, Reference Cristia, Dupoux, Gurven and Stieglitz2019). Second, infants' earliest word learning tends to focus on entities captured by core knowledge: Words that name objects (like apple; Bergelson & Swingley, Reference Bergelson and Swingley2012), accompany social gestures (like bye-bye; Bergelson & Swingley, Reference Bergelson and Swingley2013), or refer to body parts associated with actions (like hands and feet; Tincoff & Jusczyk, Reference Tincoff and Jusczyk2012), or close social engagements (like mouth and nose; Bergelson & Aslin, Reference Bergelson and Aslin2017).

Third, the function words that activate representations from a single core knowledge system can be processed rapidly and automatically, so they tend to be short and unstressed. This includes the English prepositions in and on that apply to mechanical relationships in the domain of the object system (Hespos & Spelke, Reference Hespos and Spelke2004), and the plural markers that signal entities in the domain of the number system and are spontaneously invented by isolated deaf children (Coppola, Spaepen, & Goldin-Meadow, Reference Coppola, Spaepen and Goldin-Meadow2013). In contrast, representations that require activation of two or more core knowledge systems take longer to process, and the function words that express them tend to be longer and spoken with stress, like the English prepositions that designate spatial relationships between objects (e.g., along, between, beside, and above; Landau, Reference Landau2017; Strickland, Reference Strickland2017). Nevertheless, words activate core knowledge systems automatically and seemingly effortlessly in adults and infants, suggesting that language, from the beginning, serves to represent objects and events more economically than do the core knowledge systems that simulate the behavior of these entities.

At about 9 months, infants begin to use the phrases spoken by others, containing both content words and words that are social (such as “Look!”): as invitations to share experiences with the speaker: invitations that transcend the limits of core knowledge. In one set of studies, infants viewed the events presented in Xu and Carey's experiments in which two different objects alternately moved in and out of view, but with accompanying phrases announcing the appearance of each object with one content word (e.g., “Look, a toy. Look, a toy.”), two content words (e.g., “Look, a truck. Look, a duck.”), or no content words (e.g., “Look at this. Look here.”). Infants inferred that the screen hid two objects when a different content word announced each object but not otherwise (Dewar & Xu, Reference Dewar and Xu2007; Xu, Reference Xu2002). In studies by Sandra Waxman, infants learned a new, subtle category of objects (e.g., diverse vehicles) when the same content word heralded each object (e.g., “Look, an auto.”) but not when the same function words were spoken (“Look at this one.”; Waxman & Braun, Reference Waxman and Braun2005; Waxman & Markow, Reference Waxman and Markow1995). Thus, infants infer, from a speaker's choice of content words together with words that convey social intentions, whether the speaker aims to share experiences of one object, experiences of two objects, or experiences of the commonalities among a larger set of objects.

These findings suggest that infants have come to expect that speakers will be efficient, informative, and relevant to the situation they are speaking about: They have become sensitive to the pragmatics of language and other forms of communication. Once infants have developed this expectation, language will provide them with a remarkably effective tool for learning about the world by observing what people choose to talk about and what they choose to say. How might this expectation arise? By 3 months, infants expect people's object-directed actions to be efficient and directed to things within their field of view: A possible basis for expectations that people will speak efficiently and relevantly to the current context. Infants also expect that people will share their experiences in states of social engagement: A possible basis for an expectation that they will speak informatively. If core agent and social representations compete for attention, however, how do older infants combine these notions and grasp the intentions behind a single act of speaking? The last chapter focuses on this question.

9. Beyond core knowledge

Between 9 and 14 months, changes occur in infants' social exploration and communication. At about 10 months, infants begin to understand social actions like the offering of an object (Gordon, Reference Gordon2003). Over the next months, infants begin to point to objects and to follow the points of others (Liszkowski, Carpenter, Henning, Striano, & Tomasello, Reference Liszkowski, Carpenter, Henning, Striano and Tomasello2004), to share attention to objects by looking back and forth between the object and the infant's social partner (Bruner, Reference Bruner1974; Tomasello, Reference Tomasello2008), and to view others' pointing and looking at objects as goal-directed social actions (Brune & Woodward, Reference Brune and Woodward2007). In the last chapter of this book, I hypothesize that these developments stem from the emergence of a new understanding of people's actions, engagements, and mental states. At about 10 months, infants begin to view people's actions as guided by intentions that are both social and goal-directed: People are social agents. At about 12 months, infants begin to view people's mental states as diverse, shareable experiences of a given situation: States that represent the world more finely than do perceptual systems, action systems, or systems of core knowledge (see Tomasello, Reference Tomasello2018).

Both these changes, I hypothesize, are underpinned in part by infants' developing mastery of language. By 9 months, infants likely interpret phrases like “Look, a duck,” as an invitation to share an experience of an object. This change in their understanding of speech may bring a new understanding of people, whose speech conveys intentions that are both social and object-directed. By 12 months, infants may notice that words like animal, dog, and Fido can be applied to the same object but convey different experiences of that object. This change may enrich children's understanding of their own and other speakers' mental states as simultaneously intentional (they refer to things outside themselves) and phenomenal (they convey experiences that the social agent wishes to share).

Does language learning prompt the emergence of these conceptions of other people's actions and mental states, or does it reflect changes in mental state reasoning that have other causes? The question is open, but I lean toward the first possibility for several reasons. First, representations from different core systems compete for attention, preventing young infants from understanding social actions like pointing to objects. Nevertheless, young infants appreciate that words refer in some way to things, as revealed by their early steps in learning word meanings. As symbols, words package information more economically than do systems of core knowledge, which appear to function in part through processes of mental simulation. Words also combine to express a wide array of thoughts. Language therefore provides a medium in which representations from different systems of core knowledge could be represented economically, allowing distinct representations to combine and be called on for further learning and reasoning.

Second, language carves up the world more finely than does perception, action, or core knowledge. The latter systems allow us to perceive, attend to, and act on objects, but they don't single out the experiences of an object that we might wish to share: Are we pointing to indicate to someone that this is a duck, the family pet, an intruder on the pond, or an event like the coming of spring? Our intentions to share experiences are conveyed best by language. As children's understanding of their native language grows, they become better placed to understand the diverse mental states of the people who speak to them.

With this understanding, children can begin to use language to learn new perspectives on the world – for example, that stars with the appearance of tiny points of light in the sky are distant suns, and the apparently flat ground on which we walk is a planet. These advances suggest a third reason to favor the hypothesis that language learning plays a causal role (among other factors) in children's learning of new concepts. Most of what we know as adults is learned from other people, and almost everything that children learn in school is conveyed in part by language. Neither children nor adults learn history, politics, science, or mathematics by pointing at things and exploring them in isolation: Other people, from friends and teachers to authors and public speakers, guide them through the vast conceptual space that is available to minds that are endowed with a combinatorial and symbolic natural language, informed by core knowledge.

A further reason to believe that language learning aids conceptual development stems from the tendency of languages to change with the changing lives and needs of their speakers: Languages evolve to increase their scope and efficiency as cultures change. When new tools of uncertain usefulness are invented, speakers may describe them by means of phrases, like “computing machinery,” in Turing's seminal paper (Reference Turing1950), or “self-driving car” today. If their referents prove to be widely useful and become ubiquitous, such phrases are likely to be replaced by single words, like “computer” or “laptop.” Languages thus provide efficient ways of capturing information that is useful to their speakers, increasing the economy of the phrases that convey their thoughts. Thus, the language that the child is learning provides a treasure trove of information about the culture in which he lives: It signals, by the frequency and brevity of its words and the contexts in which the words are spoken, the concepts that people find most useful and the circumstances in which they call on them.

A final reason to assign a causal role to language is that language can reverse what I call the curse of a compositional mind: Creatures who are endowed with a productively combinatorial language of thought can form a plethora of concepts, complicating the task of finding the right concept to use on any given occasion (see Piantadosi, Tenenbaum, & Goodman, Reference Piantadosi, Tenenbaum and Goodman2012, for related ideas). Because natural languages are learned from speakers who aim to be informative, however, the most frequent content words that children hear will refer to concepts that speakers consider broadly useful, and children will learn these words and corresponding concepts before the less frequent ones. Because people aim to speak economically, moreover, child learners won't be flooded with too much information. Finally, because people aim to speak relevantly to their own and their listener's current concerns, children will entertain the concepts that others express primarily in contexts in which the speakers deem them to be useful guides to thought.

Prior to the onset of formal schooling, I suggest, 1-year-old children who have come to grasp the basic syntax, semantics, and pragmatics of their language will begin to develop new perspectives on the world, guided in part by the speech of those who share experiences with them. Children's learning will vastly outpace that of other animals because the things people say, and leave unsaid, reflect not only the insights that the speaker has achieved on her own, but all that she has learned from others, directly and indirectly. The words and phrases of the language that people use to express their thoughts have made each natural language a valuable source of cultural information for child learners who are equipped with core knowledge. This, however, is a story still to be told – I hope, in part, through this book's successor.