The brain's operation depends on networks of nerve cells, called neurons, connected with each other by synapses. Scientists can now mimic some of the brain's behaviours with computer-based models of neural networks. One major domain of behaviour to which this approach has been applied is the question of how the brain acquires and maintains new information; that is, what we would call learning and memory. Neural networks employ various learning algorithms, which are recipes for how to change the network to store new information, and this chapter surveys learning algorithms that have been explored over the last decade. A few representative examples are presented here to illustrate the basic types of learning algorithms; the interested reader is encouraged to consult recent books listed in the section on Further reading, which present these algorithms in greater detail and provide a more complete survey.

It is important to keep in mind that the models of neural networks that are simulated in computers are far simpler than the highly complex and often messy neural systems that nature has devised. Although much simpler, neural network models capture some of the most basic features of the brain and may share some general properties that will allow us to understand better the operation of the more complex system. Neural network models are built from simple processing units that work together in parallel, each concerned with only a small piece of information.