The earliest programs were entirely hand coded. Both the algorithm and the knowledge that the algorithm embodied were created manually. Machines that learn were always on the wish list though. One of the earliest reported programs was the checkers playing program by Arthur Samuel that went on to beat its creator, evoking the spectre of Frankenstein's monster, a fear which still echoes today among some. Since then machine learning (ML) has steadily advanced due to three factors. First, the availability of vast amounts of data that the internet has made possible. Second, the tremendous increase in computing power available. And third, a continuous evolution of algorithms. But the core of ML is to process data using first principles and incrementally build models about the domain that the data comes from. In this chapter we look at this process.

The computer is ideally suited to learning. It can never forget. The key is to incorporate a ratchet mechanism à la natural selection – a mechanism to encapsulate the lessons learnt into a usable form, a model. Robustness demands that one must build in the ability to withstand occasional mistakes. Because the outlier must not become the norm.