In this chapter we present dynamical systems and their probabilistic description. We distinguish between system descriptions with discrete and continuous state-spaces as well as discrete and continuous time. We formulate examples of statistical models including Markov models, Markov jump processes, and stochastic differential equations. In doing so, we describe fundamental equations governing the evolution of the probability of dynamical systems. These equations include the master equation, Langevin equation, and Fokker–Plank equation. We also present sampling methods to simulate realizations of a stochastic dynamical process such as the Gillespie algorithm. We end with case studies relevant to chemistry and physics.

This chapter provides an introduction to stochastic methods for modelling spatially homogeneous systems of chemical reactions. The Gillespie stochastic simulation algorithm and the chemical master equation are presented using simple examples of chemical systems. The chemical master equation is analysed for chemical systems containing zeroth-order, first-order and second-order chemical reactions. For zeroth-order and first-order chemical reactions, the average behaviour of the stochastic chemical system is described by the ordinary differential equations (ODEs) given by the standard deterministic model. However, when we consider higher-order chemical reactions, for which the deterministic description is nonlinear, the deterministic ODE model does not provide an exact description of the average behaviour of the stochastic system.