In this section we will survey the technologies and methodologies used to process financial markets data. The importance of a streamlined and controlled process for sourcing, quality-proofing and channeling information steadily increased even before the crisis due to the explosion of incoming information on the one hand and the rapid rise in reporting requirements to investors, clients, regulators and for internal purposes on the other hand. Frequently, IT departments as well as business users ind themselves stuck in the middle trying to meet the resulting information processing challenge.

You could make the case that all financial services is about information processing. We can abstract the whole infrastructure of financial services as sets of information and sets of business logic. A bundle of business logic is an application, a set of rules that takes various inputs (e.g., equity prices and volumes) and that produces new information (e.g., trading signals, risk scores, client portfolio statements). But these activities can only produce meaningful results when fed with correct, timely and sufficient information. Bad information leads to bad decisions. Paraphrasing a well-known tagline we can summarize by saying “the data is the application.”

Faced with the incoming floods of data, many financial institutions found themselves singularly ill equipped to deal with the increased reporting requirements in the wake of the financial crisis.