The data mining and machine learning literature is now flush with scenarios about predicting baseball players’ salaries from their prior year's number of hits and walks and about predicting the product sales from the prices, customer income, and level of advertising. These are amazing and noteworthy stories. They inspire data scientists to continue their cause. The classic examples feature a large two-dimensional array of cases as rows with the independent variables, also known as the stimulus variables, and the predictable variables, also known as the response variables, as columns. If we are predicting an athlete's salary, this figure is hand-tuned by the people who negotiate contracts. Better athletic production yields better salaries as athletes are constantly compared to one other. The salary is a figure updated usually one time per year at most. And only a handful of people are involved in setting the athlete's salary or the price of a consumer item. So these represent ideal cases in predictability.

Unfortunately, prediction in the case of financial analytics never turns out to be as accurate as in these sports and marketing areas. There is just more random noise in the financial markets with prices that are updated every second of every trading day. Thousands of participants are involved. Every security is affected by many other securities. For example, oil prices are affected not only by oil supply and demand and by the volume of trades at each incremental oil price level but by interest rates and various foreign exchange rates. Nevertheless, one can try these financial predictions using the same techniques in order to experiment and observe what can be predicted.

The process of attempting prediction yields at least two benefits. On the one hand, it may be possible to predict attributes from combinations of other attributes: in this case, response variables from stimulus variables. If this was the case, we can stand alongside those other successes in other areas of data science. On the other hand, prediction may not be possible or even all that useful, however; the collection exercise, getting all the data into rows and columns of the array, provides observations that can be made in an unsupervised learning sense. And discoveries can be made by applying thresholds or sorting and filtering the attributes to find maximally performing securities, as we are able to observe in Chapter 7.