We study the poor-biased model for money exchange introduced in Cao & Motsch ((2023) Kinet. Relat. Models 16(5), 764–794.): agents are being randomly picked at a rate proportional to their current wealth, and then the selected agent gives a dollar to another agent picked uniformly at random. Simulations of a stochastic system of finitely many agents as well as a rigorous analysis carried out in Cao & Motsch ((2023) Kinet. Relat. Models 16(5), 764–794.), Lanchier ((2017) J. Stat. Phys. 167(1), 160–172.) suggest that, when both the number of agents and time become large enough, the distribution of money among the agents converges to a Poisson distribution. In this manuscript, we establish a uniform-in-time propagation of chaos result as the number of agents goes to infinity, which justifies the validity of the mean-field deterministic infinite system of ordinary differential equations as an approximation of the underlying stochastic agent-based dynamics.

We introduce an extension to Kermack and McKendrick’s classic susceptible–infected–recovered (SIR) model in epidemiology, whose underlying mechanism of infection consists of individuals attending randomly generated social gatherings. This gives rise to a system of ordinary differential equations (ODEs) where the force of the infection term depends non-linearly on the proportion of infected individuals. Some specific instances yield models already studied in the literature, to which the present work provides a probabilistic foundation. The basic reproduction number is seen to depend quadratically on the average size of the gatherings, which may be helpful in understanding how restrictions on social gatherings affect the spread of the disease. We rigorously justify our model by showing that the system of ODEs is the mean-field limit of the jump Markov process corresponding to the evolution of the disease in a finite population.

This paper investigates tail asymptotics of stationary distributions and quasi-stationary distributions (QSDs) of continuous-time Markov chains on subsets of the non-negative integers. Based on the so-called flux-balance equation, we establish identities for stationary measures and QSDs, which we use to derive tail asymptotics. In particular, for continuous-time Markov chains with asymptotic power law transition rates, tail asymptotics for stationary distributions and QSDs are classified into three types using three easily computable parameters: (i) super-exponential distributions, (ii) exponential-tailed distributions, and (iii) sub-exponential distributions. Our approach to establish tail asymptotics of stationary distributions is different from the classical semimartingale approach, and we do not impose ergodicity or moment bound conditions. In particular, the results also hold for explosive Markov chains, for which multiple stationary distributions may exist. Furthermore, our results on tail asymptotics of QSDs seem new. We apply our results to biochemical reaction networks, a general single-cell stochastic gene expression model, an extended class of branching processes, and stochastic population processes with bursty reproduction, none of which are birth–death processes. Our approach, together with the identities, easily extends to discrete-time Markov chains.

In this paper, we propose new Metropolis–Hastings and simulated annealing algorithms on a finite state space via modifying the energy landscape. The core idea of landscape modification rests on introducing a parameter c, such that the landscape is modified once the algorithm is above this threshold parameter to encourage exploration, while the original landscape is utilized when the algorithm is below the threshold for exploitation purposes. We illustrate the power and benefits of landscape modification by investigating its effect on the classical Curie–Weiss model with Glauber dynamics and external magnetic field in the subcritical regime. This leads to a landscape-modified mean-field equation, and with appropriate choice of c the free energy landscape can be transformed from a double-well into a single-well landscape, while the location of the global minimum is preserved on the modified landscape. Consequently, running algorithms on the modified landscape can improve the convergence to the ground state in the Curie–Weiss model. In the setting of simulated annealing, we demonstrate that landscape modification can yield improved or even subexponential mean tunnelling time between global minima in the low-temperature regime by appropriate choice of c, and we give a convergence guarantee using an improved logarithmic cooling schedule with reduced critical height. We also discuss connections between landscape modification and other acceleration techniques, such as Catoni’s energy transformation algorithm, preconditioning, importance sampling, and quantum annealing. The technique developed in this paper is not limited to simulated annealing, but is broadly applicable to any difference-based discrete optimization algorithm by a change of landscape.

This paper provides a full classification of the dynamics for continuous-time Markov chains (CTMCs) on the nonnegative integers with polynomial transition rate functions and without arbitrary large backward jumps. Such stochastic processes are abundant in applications, in particular in biology. More precisely, for CTMCs of bounded jumps, we provide necessary and sufficient conditions in terms of calculable parameters for explosivity, recurrence versus transience, positive recurrence versus null recurrence, certain absorption, and implosivity. Simple sufficient conditions for exponential ergodicity of stationary distributions and quasi-stationary distributions as well as existence and nonexistence of moments of hitting times are also obtained. Similar simple sufficient conditions for the aforementioned dynamics together with their opposite dynamics are established for CTMCs with unbounded forward jumps. Finally, we apply our results to stochastic reaction networks, an extended class of branching processes, a general bursty single-cell stochastic gene expression model, and population processes, none of which are birth–death processes. The approach is based on a mixture of Lyapunov–Foster-type results, the classical semimartingale approach, and estimates of stationary measures.

In this paper we introduce new birth-and-death processes with partial catastrophe and study some of their properties. In particular, we obtain some estimates for the mean catastrophe time, and the first and second moments of the distribution of the process at a fixed time t. This is completed by some asymptotic results.

We derive the asymptotic behavior of the total, active, and inactive branch lengths of the seed bank coalescent when the initial sample size grows to infinity. These random variables have important applications for populations evolving under some seed bank effects, such as plants and bacteria, and for some cases of structured populations like metapopulations. The proof relies on the analysis of the tree at a stopping time corresponding to the first time a deactivated lineage is reactivated. We also give conditional sampling formulas for the random partition, and we study the system at the time of the first reactivation of a lineage. All these results provide a good picture of the different regimes and behaviors of the block-counting process of the seed bank coalescent.

We revisit the forward algorithm, developed by Irle, to characterize both the value function and the stopping set for a large class of optimal stopping problems on continuous-time Markov chains. Our objective is to renew interest in this constructive method by showing its usefulness in solving some constrained optimal stopping problems that have emerged recently.

Queueing networks are stochastic systems formed by interconnected resources routing and serving jobs. They induce jump processes with distinctive properties, and find widespread use in inferential tasks. Here, service rates for jobs and potential bottlenecks in the routing mechanism must be estimated from a reduced set of observations. However, this calls for the derivation of complex conditional density representations, over both the stochastic network trajectories and the rates, which is considered an intractable problem. Numerical simulation procedures designed for this purpose do not scale, because of high computational costs; furthermore, variational approaches relying on approximating measures and full independence assumptions are unsuitable. In this paper, we offer a probabilistic interpretation of variational methods applied to inference tasks with queueing networks, and show that approximating measure choices routinely used with jump processes yield ill-defined optimization problems. Yet we demonstrate that it is still possible to enable a variational inferential task, by considering a novel space expansion treatment over an analogous counting process for job transitions. We present and compare exemplary use cases with practical queueing networks, showing that our framework offers an efficient and improved alternative where existing variational or numerically intensive solutions fail.

Detailed balance of a chemical reaction network can be defined in several different ways. Here we investigate the relationship among four types of detailed balance conditions: deterministic, stochastic, local, and zero-order local detailed balance. We show that the four types of detailed balance are equivalent when different reactions lead to different species changes and are not equivalent when some different reactions lead to the same species change. Under the condition of local detailed balance, we further show that the system has a global potential defined over the whole space, which plays a central role in the large deviation theory and the Freidlin–Wentzell-type metastability theory of chemical reaction networks. Finally, we provide a new sufficient condition for stochastic detailed balance, which is applied to construct a class of high-dimensional chemical reaction networks that both satisfies stochastic detailed balance and displays multistability.

The paper discusses the risk of ruin in insurance coverage of an epidemic in a closed population. The model studied is an extended susceptible–infective–removed (SIR) epidemic model built by Lefèvre and Simon (Methodology Comput. Appl. Prob.22, 2020) as a block-structured Markov process. A fluid component is then introduced to describe the premium amounts received and the care costs reimbursed by the insurance. Our interest is in the risk of collapse of the corresponding reserves of the company. The use of matrix-analytic methods allows us to determine the distribution of ruin time, the probability of ruin, and the final amount of reserves. The case where the reserves are subjected to a Brownian noise is also studied. Finally, some of the results obtained are illustrated for two particular standard SIR epidemic models.

Delayed production can substantially alter the qualitative behaviour of feedback systems. Motivated by stochastic mechanisms in gene expression, we consider a protein molecule which is produced in randomly timed bursts, requires an exponentially distributed time to activate and then partakes in positive regulation of its burst frequency. Asymptotically analysing the underlying master equation in the large-delay regime, we provide tractable approximations to time-dependent probability distributions of molecular copy numbers. Importantly, the presented analysis demonstrates that positive feedback systems with large production delays can constitute a stable toggle switch even if they operate with low copy numbers of active molecules.

We study the long-term behaviour of a random walker embedded in a growing sequence of graphs. We define a (generally non-Markovian) real-valued stochastic process, called the knowledge process, that represents the ratio between the number of vertices already visited by the walker and the current size of the graph. We mainly focus on the case where the underlying graph sequence is the growing sequence of complete graphs.

It has been known for nearly a decade that deterministically modeled reaction networks that are weakly reversible and consist of a single linkage class have trajectories that are bounded from both above and below by positive constants (so long as the initial condition has strictly positive components). It is conjectured that the stochastically modeled analogs of these systems are positive recurrent. We prove this conjecture in the affirmative under the following additional assumptions: (i) the system is binary, and (ii) for each species, there is a complex (vertex in the associated reaction diagram) that is a multiple of that species. To show this result, a new proof technique is developed in which we study the recurrence properties of the n-step embedded discrete-time Markov chain.

In large storage systems, files are often coded across several servers to improve reliability and retrieval speed. We study load balancing under the batch sampling routeing scheme for a network of n servers storing a set of files using the maximum distance separable (MDS) code (cf. Li (2016)). Specifically, each file is stored in equally sized pieces across L servers such that any k pieces can reconstruct the original file. When a request for a file is received, the dispatcher routes the job into the k-shortest queues among the L for which the corresponding server contains a piece of the file being requested. We establish a law of large numbers and a central limit theorem as the system becomes large (i.e. n → ∞), for the setting where all interarrival and service times are exponentially distributed. For the central limit theorem, the limit process take values in ℓ2, the space of square summable sequences. Due to the large size of such systems, a direct analysis of the n-server system is frequently intractable. The law of large numbers and diffusion approximations established in this work provide practical tools with which to perform such analysis. The power-of-d routeing scheme, also known as the supermarket model, is a special case of the model considered here.

New directions in Markov processes and research on master equations are showcased by example. The utility of Magnus expansions for handling time-varying rates is demonstrated. The useful notion in applied mathematics often turns out to be the pseudospectra and not simply the eigenvalues. We highlight that general principle with our own examples of Markov processes where exact eigenvalues are found and contrasted with the large errors produced by standard numerical methods. As a motivating application, isomerisation provides a running example and an illustration of our approaches to chemical kinetics. We also present a brief example of a totally asymmetric exclusion process.

Uniformization transforms a pseudo-Poisson process with unequal intensities (leaving rates) into one with uniform intensity. Self-transitions is the price to pay. Two intensities arise when one considers an absorbing barrier of a Markov process as a body in its own right: a pair of Markov processes intertwined by an extended Chapman–Kolmogorov equation naturally arises. We show that Sauer's two-state space empathy theory handles such intertwined processes. The price of self-transitions is also avoided.

We propose a stochastic model describing a process of awareness, evaluation, and decision making by agents on the d-dimensional integer lattice. Each agent may be in any of the three states belonging to the set {0, 1, 2. In this model 0 stands for ignorants, 1 for aware, and 2 for adopters. Aware and adopters inform its nearest ignorant neighbors about a new product innovation at rate λ. At rate α an agent in aware state becomes an adopter due to the influence of adopters' neighbors. Finally, aware and adopters forget the information about the new product, thus becoming ignorant, at rate 1. Our purpose is to analyze the influence of the parameters on the qualitative behavior of the process. We obtain sufficient conditions under which the innovation diffusion (and adoption) either becomes extinct or propagates through the population with positive probability.

We give an exponential tail approximation for the extinction time of a subcritical multitype branching process arising from the SIR epidemic model on a random graph with given degrees, where the type corresponds to the vertex degree. As a corollary we obtain a Gumbel limit law for the extinction time, when beginning with a large population. Our contribution is to allow countably many types (this corresponds to unbounded degrees in the random graph epidemic model, as the number of vertices tends to∞). We only require a second moment for the offspring-type distribution featuring in our model.

We consider long-run averages of additive functionals on infinite discrete-state Markov chains, either continuous or discrete in time. Special cases include long-run average costs or rewards, stationary moments of the components of ergodic multi-dimensional Markov chains, queueing network performance measures, and many others. By exploiting Foster-Lyapunov-type criteria involving drift conditions for the finiteness of long-run averages we determine suitable finite subsets of the state space such that the truncation error is bounded. Illustrative examples demonstrate the application of this method.