The structure of Markov chains is essentially probabilistic, as we have described it so far. In examining the stability properties of Markov chains, the context we shall most frequently use is also a probabilistic one: in Part II, stability properties such as recurrence or regularity will be defined as certain return to sets of positive ψ-measure, or as finite mean return times to petite sets, and so forth.

Yet for many chains, there is more structure than simply a σ-field and a probability kernel available, and the expectation is that any topological structure of the space will play a strong role in defining the behavior of the chain. In particular, we are used thinking of specific classes of sets in ℝn as having intuitively reasonable properties. When there is a topology, compact sets are thought of in some sense as manageable sets, having the same sort of properties as a finite set on a countable space; and so we could well expect “stable” chains to spend the bulk of their time in compact sets. Indeed, we would expect compact sets to have the sort of characteristics we have identified, and will identify, for small or petite sets.

Conversely, open sets are “non-negligible” in some sense, and if the chain is irreducible we might expect it at least to visit all open sets with positive probability. This indeed forms one alternative definition of “irreducibility”.

In this, the first chapter in which we explicitly introduce topological considerations, we will have, as our two main motivations, the desire to link the concept of ψ-irreducibility with that of open set irreducibility and the desire to identify compact sets as petite.