Up to now, index set I for a stochastic process can indifferently denote either successive draws from an urn or a time step in a time evolution. In this chapter, a probabilistic dynamics for n objects divided into g categories will be defined. The simplest case is Markovian dynamics in discrete time, and so this chapter is devoted to the theory of finite Markov chains.

After studying this chapter you should be able to:

1. • define Markov chains;

2. • discuss some properties of Markov chains such as irreducibility, periodicity, stationarity and reversibility;

3. • determine the invariant distribution for Markov chains either by means of the Markov equation or by using Monte Carlo simulations;

4. • understand the meaning of statistical equilibrium for aperiodic irreducible Markov chains;

5. • use the detailed balance principle to compute the invariant distribution, if possible;

6. • exemplify the above properties using the Ehrenfest urn as a prototypical Markov chain.

From kinematics to Markovian probabilistic dynamics

6.1.1 Kinematics

Consider a population composed of n elements and g categories which partition the variate under study. In economics, the n elements may represent economic agents and the g categories can be seen as strategies, but other interpretations are possible. For instance, one can speak of n individuals working in g factories or n firms active in g economic sectors, and so on. In physics, the n elements may be particles and the g categories energy levels or, following Boltzmann, one can consider n energy elements to be divided into g particles, etc.