Finite-state-and-action Markov branching decision chains are studied with bounded endogenous expected population sizes and interest-rate-dependent one-period rewards that are analytic in the interest rate at zero. The existence of a stationary strong-maximum-present-value policy is established. Miller and Veinott's [1969] strong policy-improvement method is generalized to find in finite time a stationary n-present-value optimal policy and, when the one-period rewards are rational in the interest rate, a stationary strong-maximum-present-value policy. This extends previous studies of Blackwell [1962], Miller and Veinott [1969], Veinott [1974], and Rothblum [1974, 1975], in which the one-period rewards are independent of the interest rate, and Denardo [1971] in which semi-Markov decision chains with small interest rates are studied. The problem of finding a stationary n-present-value optimal policy is also formulated as a staircase linear program in which the objective function and right-hand sides, but not the constraint matrix, depend on the interest rate, and solutions for all small enough positive interest rates are sought. The optimal solutions of the primal and dual are polynomials in the reciprocal of the interest rate. A constructive rule is given for finding a stationary n-present-value optimal policy from an optimal solution of the asymptotic linear program. This generalizes the linear programming approaches for finding maximum-reward-rate and maximum-present-value policies for Markov decision chains studied by Manne [1960], d'Epenoux [1960, 1963], Balinski [1961], Derman [1962], Denardo and Fox [1968], Denardo [1970], Derman and Veinott [1972], Veinott [1973], and Hordijk and Kallenberg [1979, 1984].

This paper is concerned with how coupling can be used to enhance the efficiency of a certain class of terminating simulations, in Markov process settings in which the stationary distribution is known. We are able to theoretically establish that our coupling-based estimator is often more efficient than the naive estimator. In addition, we discuss extensions of our methodology to Markov process settings in which conventional coupling fails and show (for Doeblin chains) that knowledge of the stationary distribution is sometimes unnecessary.

Diameter measurements on successive metal hubs from a machining operation are modeled using a random walk with observation error and linear drift corresponding to tool wear. After producing and measuring a hub, the depth of the cutting tool on the lathe can be adjusted in integer multiples of 0.0001 inches. How should the tool be adjusted?

An optimal discrete adjustment strategy is derived assuming that the lathe automatically corrects for deterministic tool wear. The objective is to minimize expected run costs proportional to the sum of squared diameter deviations from a target plus fixed charges for manual tool adjustments. The optimal strategy makes no manual adjustment if an estimate of the process mean is near target. Otherwise, an adjustment is made to return the estimated mean as near to target as possible within the adjustment resolution.

The region where no adjustments are made widens near the end of the production run where adjustments have only short-term impact. The region converges as the number of remaining periods increases. Plots of expected run costs show that the extra cost of discreteness is small at high resolution but is substantial when the adjustment grid is coarse.

We consider N unreliable machines that are maintained by M repairmen. The time until failure of machine i, and its repair by repairman j, are exponentially distributed random variable with parameters λi, and μj, respectively. All failure times and repair times are independent. Machine i earns ci, per unit time, while it is working. We wish to maximize the expected total discounted reward. It is shown that when the following conditions are satisfied — c1 ≥ … ≥ cN and c1/λ1≥…≥cN/λN, the policy that assigns the fastest repairman to the machine with the lowest index is optimal. Moreover, it is shown that when the second condition is replaced by λ≤ … ≤λN, then this policy maximizes, stochastically, the number of the most reliable machines at every time t.

We study the problem of scheduling multiclass parallel server queues where the cost for each job is a general class-dependent function of the job's completion time and a second random variable. We characterize the structure of the optimal policy under fairly general conditions on the random variables.

This paper examines the preservation of several aging classes of lifetime distributions in the formation of coherent and mixed systems with independent and identically distributed (i.i.d.) or identically distributed (i.d.) component lifetimes. The increasing mean inactivity time class and the decreasing mean time to failure class are developed for the lifetime of systems with possibly dependent and i.d. component lifetimes. The increasing likelihood ratio property is also discussed for the lifetime of a coherent system with i.i.d. component lifetimes. We present sufficient conditions satisfied by the signature of a coherent system with i.i.d. components with exponential distribution, under which the decreasing mean remaining lifetime, the increasing mean inactivity time, and the decreasing mean time to failure are all satisfied by the lifetime of the system. Illustrative examples are presented to support the established results.

In this paper, we consider a generalized Pólya urn model with multiple drawings and time-dependent reinforcements. Suppose an urn initially contains w white and r red balls. At the nth action, m balls are drawn at random from the urn, say k white and m−k red balls, and then replaced in the urn along with cnk white and cn(m − k) red balls, where {cn} is a given sequence of positive integers. Repeat the above procedure ad infinitum. Let Xn be the proportion of the white balls in the urn after the nth action. We first show that Xn converges almost surely to a random variable X. Next, we give a necessary and sufficient condition for X to have a Bernoulli distribution with parameter w/(w + r). Finally, we prove that X is absolutely continuous if {cn} is bounded.

In this paper, we consider some dividend problems in the perturbed compound Poisson model under a constant barrier dividend strategy. We approximate the expected present value of dividend payments before ruin and the expected discounted penalty function based on the COS method, and construct some nonparametric estimators by using a random sample on claim number and individual claim sizes. Under a large sample size setting, we perform an error analysis of the estimators. We also provide some simulation results to verify the effectiveness of this estimation method when the sample size is finite.

In this paper, we discuss stochastic orderings of lifetimes of two heterogeneous parallel and series systems with heterogeneous dependent components having generalized Birnbaum–Saunders distributions. The comparisons presented here are based on the vector majorization of parameters. The ordering results are established in some special cases for the generalized Birnbaum–Saunders distribution based on the multivariate elliptical, normal, t, logistic, and skew-normal kernels. Further, we use these results by considering Archimedean copulas to model the dependence structure among systems with generalized Birnbaum–Saunders components. These results have been used to derive some upper and lower bounds for survival functions of lifetimes of parallel and series systems.

For the GI/GI/1 queue we show that the scaled queue size converges to reflected Brownian motion in a critical queue and converges to reflected Brownian motion with drift for a sequence of subcritical queuing models that approach a critical model. Instead of invoking the topological argument of the usual continuous-mapping approach, we give a probabilistic argument using Skorokhod embeddings in Brownian motion.

We consider the problem of optimally maintaining an offshore wind farm in which major components progressively degrade over time due to normal usage and exposure to a randomly varying environment. The turbines exhibit both economic and stochastic dependence due to shared maintenance setup costs and their common environment. Our aim is to identify optimal replacement policies that minimize the expected total discounted setup, replacement, and lost power production costs over an infinite horizon. The problem is formulated using a Markov decision process (MDP) model from which we establish monotonicity of the cost function jointly in the degradation level and environment state and characterize the structure of the optimal replacement policy. For the special case of a two-turbine farm, we prove that the replacement threshold of one turbine depends not only on its own state of degradation but also on the state of degradation of the other turbine in the farm. This result yields a complete characterization of the replacement policy of both turbines by a monotone curve. The policies characterized herein can be used to optimally prescribe timely replacements of major components and suggest when it is most beneficial to share costly maintenance resources.

We introduce a broad class of multi-hooking networks, wherein multiple copies of a seed are hooked at each step at random locations, and the number of copies follows a predetermined building sequence of numbers. We analyze the degree profile in random multi-hooking networks by tracking two kinds of node degrees—the local average degree of a specific node over time and the global overall average degree in the graph. The former experiences phases and the latter is invariant with respect to the type of building sequence and is somewhat similar to the average degree in the initial seed. We also discuss the expected number of nodes of the smallest degree. Additionally, we study distances in the network through the lens of the average total path length, the average depth of a node, the eccentricity of a node, and the diameter of the graph.

We consider a hierarchical linear regression model where the regression parameters for the units have a multivariate normal distribution whose parameters are unknown. Several replications are available for each unit. The design matrices for the units need not be the same. A complicating feature of the model is that each observation is subject to measurement error. The objective of the paper is to derive the predictive distribution of the “true” value of the response at a given design point. A Bayesian treatment is given to the problem. In addition to standard prior distributions, other prior distributions are considered. The calculations are done with the Gibbs sampler. An example is discussed in detail.

We consider an optimal stopping problem for a general discrete-time process X1, X2, …, Xn, … on a common measurable space. Stopping at time n (n = 1, 2, …) yields a reward Rn(X1, …, Xn) ≥ 0, while if we do not stop, we pay cn(X1, …, Xn) ≥ 0 and keep observing the process. The problem is to characterize all the optimal stopping times τ, i.e., such that maximize the mean net gain:

$$E(R_\tau(X_1,\dots,X_\tau)-\sum_{n=1}^{\tau-1}c_n(X_1,\dots,X_n)).$$

In the particular case of Markov sequence X1, X2, … we estimate the stability of the optimal stopping problem under perturbations of transition probabilities.

In this paper, we study the pricing of vulnerable Asian options with liquidity risk. We employ general Lévy processes to capture the changes in the liquidity discount factors and the information processes of all assets. In the proposed pricing model, we obtain the closed-form pricing formula of vulnerable Asian options using the Fourier transform methods. Finally, the derived pricing formula is used to illustrate the effects of asymmetric jump risk, and the effects are relatively stable on (vulnerable) Asian options with different moneynesses.

In this article we investigate conditions by a unified method under which the covariances of functions of two adjacent ordered random variables are nonnegative. The main structural results are applied to several kinds of ordered random variable, such as delayed record values, continuous and discrete ℓ∞⩽-spherical order statistics, epoch times of mixed Poisson processes, generalized order statistics, discrete weak record values, and epoch times of modified geometric processes. These applications extend the main results for ordinary order statistics in Qi [28] and for usual record values in Nagaraja and Nevzorov [25].

We extend the classic sequential stochastic assignment problem to include arrivals of workers. When workers are all of the same type, we show that the socially optimal policy is the same as the individually optimal policy for which workers are given priority according to last come–first served. This result also holds under several variants in the model assumptions. When workers have different types, we show that the socially optimal policy is determined by thresholds such that more valuable jobs are given to more valuable workers, but now the individually optimal policy is no longer socially optimal. We also show that the overall value increases when worker or job values become more variable.

In our article [3] we have found a gap in the middle of the proof of Theorem 3.2. Therefore, we do not know whether Theorem 3.2 is true for the reverse hazard rate order. However, we could prove the following weaker result for the stochastic order.

Expressions for the distribution, density, and percentiles of weighted sums of Rayleigh random variables are given, including the tilted Edgeworth expansion.

In quantitative finance, it is often necessary to analyze the distribution of the sum of specific functions of observed values at discrete points of an underlying process. Examples include the probability density function, the hedging error, the Asian option, and statistical hypothesis testing. We propose a method to calculate such a distribution, utilizing a recursive method, and examine it using various examples. The results of the numerical experiment show that our proposed method has high accuracy.