Learning from Data

Just as our brains use learning as we develop the ability to interpret the world through patterns arriving on the auditory nerves, so can machines use learning to develop the ability to extract meaning from the sound representations extracted by auditory models.

The inputs that machines learn from are called data, and come in many forms. Sometimes we use supervised learning: training data associate sounds with answers, and the machine learns a model for that association, so that it will give good answers for novel sound data later. If we have lots of sound data, but no good answers to say what it means, we can still model the data and learn to produce compact meaningful descriptions and predictions of it, using unsupervised learning.

In this chapter we focus on supervised learning, and on artificial neural networks (ANNs or simply neural networks or neural nets) as a general class of techniques that were originally motivated by theories of how brains work. Neural nets and their descendants are widely employed in classification problems, where the answers are class decisions, and in regression problems, where the answers are continuous functions of the inputs. Our examples focus on classification.

The machine learning (ML) community discovered long ago that learning to get mostly right answers on a training set is not a safe goal for supervised learning. A system that models the training data well may get all the right answers on those, but still fail to generalize to independent testing data. So it is important to have an independent test set, and to use techniques that learn from the training data but generalize well to the independent testing data.