Neural organization: Structure, function, and dynamics shows how theory and experiment can supplement each other in an integrated, evolving account of the brain's structure, function, and dynamics. (1) Structure: Studies of brain function and dynamics build on and contribute to an understanding of many brain regions, the neural circuits that constitute them, and their spatial relations. We emphasize Szentágothai's modular architectonics principle, but also stress the importance of the microcomplexes of cerebellar circuitry and the lamellae of hippocampus. (2) Function: Control of eye movements, reaching and grasping, cognitive maps, and the roles of vision receive a functional decomposition in terms of schemas. Hypotheses as to how each schema is implemented through the interaction of specific brain regions provide the basis for modeling the overall function by neural networks constrained by neural data. Synthetic PET integrates modeling of primate circuitry with data from human brain imaging. (3) Dynamics: Dynamic system theory analyzes spatiotemporal neural phenomena, such as oscillatory and chaotic activity in both single neurons and (often synchronized) neural networks, the self-organizing development and plasticity of ordered neural structures, and learning and memory phenomena associated with synaptic modification. Rhythm generation involves multiple levels of analysis, from intrinsic cellular processes to loops involving multiple brain regions. A variety of rhythms are related to memory functions. The Précis presents a multifaceted case study of the hippocampus. We conclude with the claim that language and other cognitive processes can be fruitfully studied within the framework of neural organization that the authors have charted with John Szentágothai.

A scheme for understanding the organization of human postural movements is developed in the format of a position paper. The structural characteristics of the body and the geometry of muscular actions are incorporated into a three-dimensional graphical representation of human movement mechanics in the sagittal plane. A series of neural organizational hypotheses limit a theoretically infinite number of combinations of muscle contractions and associated movement trajectories for performing postural corrections: (1) Controls are organized to use the minimum number of muscles; (2) frequently performed movements are organized to require a minimum of neural decision-making.

These hypotheses lead to the prediction that postural movements are composed of muscle contractile strategies derived from a limited set of distinct contractile patterns. The imposition of two mechanical constraints related to the configuration of support and to requirements for body stability with respect to gravity predict the conditions under which individual movement strategies will be deployed.

A complementary organizational scheme for the senses is developed. We show that organization of postural movements into combinations of distinct strategies simplifies the interpretation of sensory inputs. The fine-tuning of movement strategies can be accomplished by breaking down the complex array of feedback information into a series of scalar quantities related to the parameters of the movement strategies. For example, the magnitude, aim, and curvature of the movement trajectory generated by an individual strategy can be adjusted independently.

The second half of the report compares theoretical predictions with a series of actual experimental observations on normal subjects and patients with known sensory and motor disorders. Actual postural movements conform to theoretical predictions about the composition of individual movement strategies and the conditions under which each strategy is used. Observations on patients suggest how breakdowns in individual steps within the logical process of organization can lead to specific movement abnormalities.

Discussion focuses on the areas needing further experimentation and on the implications of the proposed organizational scheme. We conclude that although our organizational scheme is not new in demonstrating the need for simplifying the neural control of movement, it is perhaps original in imposing discrete logical control upon a continuous mechanical system. The attraction of the scheme is that it provides a framework compatible with both mechanical and physiological information and amenable to experimental testing.

Intermediate constructs are required as bridges between complex behaviors and realistic models of neural circuitry. For cognitive scientists in general, schemas are the appropriate functional units; brain theorists can work with neural layers as units intermediate between structures subserving schemas and small neural circuits.

After an account of different levels of analysis, we describe visuomotor coordination in terms of perceptual schemas and motor schemas. The interest of schemas to cognitive science in general is illustrated with the example of perceptual schemas in high-level vision and motor schemas in the control of dextrous hands.

Rana computatrix, the computational frog, is introduced to show how one constructs an evolving set of model families to mediate flexible cooperation between theory and experiment. Rana computatrix may be able to do for the study of the organizational principles of neural circuitry what Aplysia has done for the study of subcellular mechanisms of learning. Approach, avoidance, and detour behavior in frogs and toads are analyzed in terms of interacting schemas. Facilitation and prey recognition are implemented as tectal-pretectal interactions, with the tectum modeled by an array of tectal columns. We show how layered neural computation enters into models of stereopsis and how depth schemas may involve the interaction of accommodation and binocular cues in anurans.

This paper presents a general computational treatment of how mammals are able to deal with visual objects and environments. The model tries to cover the entire range from behavior and phenomenological experience to detailed neural encodings in crude but computationally plausible reductive steps. The problems addressed include perceptual constancies, eye movements and the stable visual world, object descriptions, perceptual generalizations, and the representation of extrapersonal space.

The entire development is based on an action-oriented notion of perception. The observer is assumed to be continuously sampling the ambient light for information of current value. The central problem of vision is taken to be categorizing and locating objects in the environment. The critical step in this process is the linking of visual information to symbolic object descriptions; this is called indexing, from the analogy of identifying a book from index terms. The system must also identify situations and use this knowledge to guide movement and other actions in the environment. The treatment focuses on the different representations of information used in the visual system.

The four representational frames capture information in the following forms: retinotopic, head-based, symbolic, and allocentric. The functional roles of the four frames, the communication among them, and their suggested neurophysiological realization constitute the core of the paper. The model is perforce crude, but appears to be consistent with all relevant findings.

The cerebral cortex is a rich and diverse structure that is the basis of intelligent behavior. One of the deepest mysteries of the function of cortex is that neural processing times are only about one hundred times as fast as the fastest response times for complex behavior. At the very least, this would seem to indicate that the cortex does massive amounts of parallel computation.

This paper explores the hypothesis that an important part of the cortex can be modeled as a connectionist computer that is especially suited for parallel problem solving. The connectionist computer uses a special representation, termed value unit encoding, that represents small subsets of parameters in a way that allows parallel access to many different parameter values. This computer can be thought of as computing hierarchies of sensorimotor invariants. The neural substrate can be interpreted as a commitment to data structures and algorithms that compute invariants fast enough to explain the behavioral response times. A detailed consideration of this model has several implications for the underlying anatomy and physiology.

Spatial and temporal properties related to direction selectivity of both simple and complex type visual cortex neurons were assessed by cross-correlation analysis of their responses to random ternary white noise. This stimulus consisted of multiple randomly placed bars, each colored white, black, or gray with equal probability, which were rerandomized every 5–10 ms. A first-order cross-correlation analysis of a neuron's spike train with the spatiotemporal history of the stimulus provided an estimate of the neuron's linear spatiotemporal filtering properties. A nonlinear correlation analysis measured the amount of interaction for pair-wise combinations of bars as a function of their relative spatial and temporal separations. The spatiotemporal orientation of each of these functions was quantified using a “motion energy index” (MEI), which was compared to the neurons' direction selectivity measured with drifting sinewave gratings. Both first-order and nonlinear correlation plots usually showed s–t orientation whose sign was consistent with the neuron's direction preference; however, in many cases the MEI for first-order analysis was weak compared to that seen in the nonlinear interactions. The structures of the nonlinear interaction functions were also compared with predictions from a conventional model of direction selectivity based on a simple spatiotemporally oriented linear filter, followed by an intensive nonlinearity (“LN model”). These comparisons showed that some neurons' data agreed reasonably well with such a model, while others agreed poorly or not at all. Simulations of an alternative model which combines signals from idealized lagged and nonlagged front-end linear filters produce noise correlation results more like those seen in the neurophysiological data.