Bisimulation is a concept that captures behavioural equivalence of states in a variety of types of transition systems. It has been widely studied in a discrete-time setting. The core of this work is to generalise the discrete-time picture to continuous time by providing a notion of behavioural equivalence for continuous-time Markov processes. In Chen et al. [(2019). Electronic Notes in Theoretical Computer Science 347 45–63.], we proposed two equivalent definitions of bisimulation for continuous-time stochastic processes where the evolution is a flow through time: the first one as an equivalence relation and the second one as a cospan of morphisms. In Chen et al. [(2020). Electronic Notes in Theoretical Computer Science.], we developed the theory further: we introduced different concepts that correspond to different behavioural equivalences and compared them to bisimulation. In particular, we studied the relation between bisimulation and symmetry groups of the dynamics. We also provided a game interpretation for two of the behavioural equivalences. The present work unifies the cited conference presentations and gives detailed proofs.

We construct a flow of continuous-time and discrete-state branching processes. Some scaling limit theorems for the flow are proved, which lead to the path-valued branching processes and nonlocal branching superprocesses, over the positive half line, studied in Li (2014).

This work is concerned with the existence of an optimal control strategy for the long-run average continuous control problem of piecewise-deterministic Markov processes (PDMPs). In Costa and Dufour (2008), sufficient conditions were derived to ensure the existence of an optimal control by using the vanishing discount approach. These conditions were mainly expressed in terms of the relative difference of the α-discount value functions. The main goal of this paper is to derive tractable conditions directly related to the primitive data of the PDMP to ensure the existence of an optimal control. The present work can be seen as a continuation of the results derived in Costa and Dufour (2008). Our main assumptions are written in terms of some integro-differential inequalities related to the so-called expected growth condition, and geometric convergence of the post-jump location kernel associated to the PDMP. An example based on the capacity expansion problem is presented, illustrating the possible applications of the results developed in the paper.

In this paper we obtain some ergodic properties and ergodic decompositions of a continuous-time, Borel right Markov process taking values in a locally compact and separable metric space. Initially, we assume that an invariant probability measure (IPM) μ exists for the process and, without making any further assumptions on the transition kernel, obtain some characterization results for the convergence of the expected occupation measure to a limit kernel. Under the same assumption, we present the so-called Yosida decomposition. Next, instead of assuming the existence of an IPM, we assume that the Markov process satisfies a certain condition, named the T'-condition. Under this condition it is shown that the Foster-Lyapunov criterion is necessary and sufficient for the existence of an IPM and that the process admits a Doeblin decomposition. Furthermore, it is shown that in this case the set of ergodic probability measures is countable and that every probability measure for the Markov process is nonsingular with respect to the transition kernel.

In this work, we find that the zero capital income tax result might not hold, even at the steady state, when the government discount rate differs from the individual one. As intuitive Pigouvian considerations would suggest, capital income should be taxed (subsidized) when the government is less (more) impatient than individuals are. However, a counterintuitive asymmetry emerges as for the steady state since, in the long run, capital income cannot be taxed because of the explosive distortionary effect of positive taxes. The asymmetry is ruled out with a logarithmic utility function because, in this case, the anticipated policy path does not affect current individual choices and thus the cumulative distortionary effect of taxes disappears.

In this paper, it is shown that the Foster-Lyapunov criterion is sufficient to ensure the existence of an invariant probability measure for both discrete- and continuous-time Markov processes without any additional hypotheses (such as irreducibility).

The variation of the state vectors p(t) = (pi(t)) of a continuous-time homogeneous Markov system with fixed size is examined. A specific time t0 after which the size order of the elements pi(t) becomes stable provides a criterion of the system's convergence rate. A method is developed to find t0 and a quickly evaluated lower bound for t0. This method is based on the geometric characteristics and the volumes of the attainable structures. Moreover, a condition concerning the selection of starting vectors p(0) is given so that the vector functions p(t) retain the same size order for every time greater than a given time t.

This paper discusses the modelling and control of pension funds.

A continuous-time stochastic pension fund model is proposed in which there are n risky assets plus the risk-free asset as well as randomness in the level of benefit outgo. We consider Markov control strategies which optimise over the contribution rate and over the range of possible asset-allocation strategies.

For a general (not necessarily quadratic) loss function it is shown that the optimal proportions of the fund invested in each of the risky assets remain constant relative to one another. Furthermore, the asset allocation strategy always lies on the capital market line familiar from modern portfolio theory.

A general quadratic loss function is proposed which provides an explicit solution for the optimal contribution and asset-allocation strategies. It is noted that these solutions are not dependent on the level of uncertainty in the level of benefit outgo, suggesting that small schemes should operate in the same way as large ones. The optimal asset-allocation strategy, however, is found to be counterintuitive leading to some discussion of the form of the loss function. Power and exponential loss functions are then investigated and related problems discussed.

The stationary distribution of the process is considered and optimal strategies compared with dynamic control strategies.

Finally there is some discussion of the effects of constraints on contribution and asset-allocation strategies.

We prove some limit theorems for contiunous time and state branching processes. The non-degenerate limit laws are obtained in critical and non-critical cases by conditioning or introducing immigration processes. The limit laws in non-critical cases are characterized in terms of the cononical measure of the cumulant semigroup. The proofs are based on estimates of the cumulant semigroup derived from the forward and backward equations, which are easier than the proffs in the classical setting.

In order to describe the evolution of the attainable structures of a continuous time homogeneous Markov system (HMS) with fixed size, we evaluate the volume of the sets of the attainable structures in Euclidean space in the course of time, and we find the value of the volume asymptotically. Then, using the concept of the volume of the attainable structures, we provide a method to evaluate the ‘age' of the system in continuous and discrete time. We also estimate the evolution of the distance of two (attainable) structures of the system as it changes following the transformations of the structures.

Characterizations of strong ergodicity for Markov chains using mean visit times have been found by several authors (Huang and Isaacson (1977), Isaacson and Arnold (1978)). In this paper a characterization of uniform strong ergodicity for a continuous-time non-homogeneous Markov chain is given. This extends the characterization, using mean visit times, that was given by Isaacson and Arnold.

This paper is concerned with the problem of selecting the transition intensities for a Markov chain in continuous time so as to minimise the long-term average cost. Sufficient conditions are established for an optimal stationary policy using unbounded solutions of the optimality equation. This is a development of recent work on Markovian decision processes in discrete time. The theory is illustrated by considering a simple birth and death process with controlled immigration.

Results on the behaviour of Markov branching processes as time goes to infinity, hitherto obtained for models which assume a discrete state-space or discrete time or both, are here generalised to a model with both state-space and time continuous. The results are similar but the methods not always so.