A Markov decision model is considered for the control of a truncated general immigration process, which represents a pest population, by the introduction of total catastrophes. The optimality criterion is that of minimizing the expected long-run average cost per unit time. Firstly, a necessary and sufficient condition is found under which the policy of never controlling is optimal. If this condition fails, a parametric analysis, in which a fictitious parameter is varied over the entire real line, is used to establish the optimality of a control-limit policy. Furthermore, an efficient Markov decision algorithm operating on the class of control-limit policies is developed for the computation of the optimal policy.

This paper is concerned with the problem of controlling a simple immigration–birth process, which represents a pest population, by the introduction of catastrophes which, when they occur, reduce the population size to zero. The optimality criterion is that of minimising the long-term average cost per unit time of the process. Firstly, an optimal policy is found within a restricted class of stationary policies, which introduce catastrophes if and only if the population size is greater than or equal to some critical value x. The optimality of this policy within the wider class of all stationary policies is then verified by applying the general results of Bather (1976).

This paper is concerned with the problem of selecting the transition intensities for a Markov chain in continuous time so as to minimise the long-term average cost. Sufficient conditions are established for an optimal stationary policy using unbounded solutions of the optimality equation. This is a development of recent work on Markovian decision processes in discrete time. The theory is illustrated by considering a simple birth and death process with controlled immigration.

This paper is concerned with the general problem of finding an optimal transition matrix for a finite Markov chain, where the probabilities for each transition must be chosen from a given convex family of distributions. The immediate cost is determined by this choice, but it is required to minimise the average expected cost in the long run. The problem is investigated by classifying the states according to the accessibility relations between them. If an optimal policy exists, it can be found by considering the convex subsystems associated with the states at different levels in the classification scheme.

A Markov process in discrete time with a finite state space is controlled by choosing the transition probabilities from a given convex family of distributions depending on the present state. The immediate cost is prescribed for each choice and it is required to minimise the average expected cost over an infinite future. The paper considers a special case of this general problem and provides the foundation for a general solution. The main result is that an optimal policy exists if each state of the system can be reached with positive probability from any other state by choosing a suitable policy.

A Markov process in discrete time with a finite state space is controlled by choosing the transition probabilities from a prescribed set depending on the state occupied at any time. Given the immediate cost for each choice, it is required to minimise the expected cost over an infinite future, without discounting. Various techniques are reviewed for the case when there is a finite set of possible transition matrices and an example is given to illustrate the unpredictable behaviour of policy sequences derived by backward induction. Further examples show that the existing methods may break down when there is an infinite family of transition matrices. A new approach is suggested, based on the idea of classifying the states according to their accessibility from one another.