Objectives

This chapter introduces an underappreciated topic in the social sciences that lies directly in the intersection of matrix algebra (Chapters 3 and 4) and probability (Chapter 7). It is an interesting area in its own right with many applications in the social sciences, but it is also a nice reinforcement of important principles we have already covered. Essentially the idea is relevant to the things we study in the social sciences because Markov chains specifically condition on the current status of events. Researchers find that this is a nice way to describe individual human decision making and collective human determinations.

So Markov chains are very practical and useful. They model how social and physical phenomena move from one state to another. The first part of this chapter introduces the mechanics of Markov chains through the kernel. This is the defining mechanism that “moves” the Markov chain around. The second part of the chapter describes various properties of Markov chains and how such chains can differ in how they behave. The first few properties are elementary, and the last few properties are noticeably more advanced and may be skipped by the reader who only wants an introduction.

Defining Stochastic Processes and Markov Chains

Markov chains sound like an exotic mathematical idea, but actually the principle is quite simple. Suppose that your decision-making process is based only on a current state of affairs. For example, in a casino wagering decisions are usually dictated only by the current state of the gambler: the immediate confronting decision (which number to bet on, whether to take another card, etc.) and the available amount of money.