Probabilistic time series models

A time series is an ordered collection of observations y1:T ≡ {y1, …, yT}. Typical tasks in time series analysis are the prediction of future observations (for example in weather forecasting) or the extraction of lower-dimensional information embedded in the observations (for example in automatic speech recognition). In neuroscience, common applications are related to the latter, for example the detection of epileptic events or artifacts from EEG recordings (Boashash & Mesbah 2001; Rohalova et al. 2001; Tarvainen et al. 2004; Chiappa & Barber 2005, 2006), or the detection of intention in a collection of neural recordings for the purpose of control (Wu et al. 2003). Time series models commonly make the assumptions that the recent past is more informative than the distant past and that the observations are obtained from a noisy measuring device or from an inherent stochastic system. Often, in models of physical systems, additional knowledge about the properties of the time series are built into the model, including any known physical laws or constraints; other forms of prior knowledge may relate to whether the process underlying the time series is discrete or continuous. Markov models are classical models which allow one to build in such assumptions within a probabilistic framework.

A graphical depiction

A probabilistic model of a time series y1:T is a joint distribution p(y1:T). Commonly, the structure of the model is chosen to be consistent with the causal nature of time. This is achieved with Bayes' rule, which states that the distribution of the variable x, given knowledge of the state of the variable y, is given by p(x|y) = p(x, y)/p(y), see for example Barber (2012). Here p(x, y) is the joint distribution of x and y, while p(y) = ∫ p(x, y)dx is the marginal distribution of y (i.e., the distribution of y not knowing the state of x).