For discrete versions of red-and-black with a goal, for timid play to be optimal, it suffices, but is not necessary, that no win probability be less than one-half. A condition that is both necessary and sufficient is provided.

In this paper we compare the variance of functions of random variables and functionals of point processes. Specifically we give sufficient conditions on two random variables X and Y under which the variances var f(X) and var f(Y) of the function f of these random variables can be compared. For example we show that if X is smaller than Y in the shifted-up mean residual life and in the usual stochastic order, then var f(X) ≤ var f(y) for all increasing convex functions f, whenever these variances are well defined. In the context of point processes we compare the variances var Φ(M) and var Φ(N) of the functional Φ of two point processes M = {M(t), t ≥ 0} and N = {N(t), t ≥ 0}. We provide sufficient conditions under which these variances can be compared. Specifically we consider comparisons between (i) two renewal processes and between (ii) two (homogeneous or nonhomogeneous) Poisson processes. For example we show that for any nonhomogeneous Poisson process N with a rate function bounded from above by λ and from below by μ and two homogeneous Poisson processes L and M with rates λ and μ, respectively, var Φ (L) ≤ var Φ (N) ≤ var Φ (M) for any functional Φ that is increasing directionally convex in the event times, whenever these variances are well defined. This, for example, implies that if Tnis the nth event time of N, then n/λ2 ≤ var (Tn) ≤ n/μ2. Some applications of these results are given.

We show how the importance sampling identity can often be used to efficiently bound tail probabilities, illustrating with the normal, Poisson, and gamma random variables.

We give conditions on the optimality of an index policy for multiarmed bandits when arms expire independently. We also give a new simple proof of the optimality of the Gittins index policy for the classic multiarmed bandit problem.

We consider the reliability of one random variable relative to another, when the variables are continuous and take values in an interval [a, b). We give definitions and characterizations for the exponential property and the standard aging properties IFR, IFRA, and NBU. The exponential property defines an equivalence relation on the distributions, and then each of these aging properties defines a partial order on the distributions, modulo the exponential equivalence. We give a set of conditions that must be satisfied for a general aging property to define such a partial order and show that these conditions are not satisfied by the NBUE property. Several parametric families of distributions are considered.

We establish functional large deviation principles (FLDPs) for waiting and departure processes in single-server queues with unlimited waiting space and the first-in first-out service discipline. We apply the extended contraction principle to show that these processes obey FLDPs in the function space D with one of the nonuniform Skorohod topologies whenever the arrival and service processes obey FLDPs and the rate function is finite for appropriate discontinuous functions. We apply our previous FLDPs for inverse processes to obtain an FLDP for the waiting times in a queue with a superposition arrival process. We obtain FLDPs for queues within acyclic networks by showing that FLDPs are inherited by processes arising from the network operations of departure, superposition, and random splitting. For this purpose, we also obtain FLDPs for split point processes. For the special cases of deterministic arrival processes and deterministic service processes, we obtain convenient explicit expressions for the rate function of the departure process, but not more generally. In general, the rate function for the departure process evidently must be calculated numerically. We also obtain an FLDP for the departure process of completed work, which has important application to the concept of effective bandwidths for admission control and capacity planning in packet communication networks.

In this paper, we study the asymptotic equipartition property (AEP) for a nonhomogeneous Markov information source. We first give a limit theorem for the averages of the functions of two variables of this information source by using the convergence theorem for the martingale difference sequence. As corollaries, we get several limit theorems and a limit theorem of the relative entropy density, which hold for any nonhomogeneous Markov information source. Then, we get a class of strong laws of large numbers for nonhomogeneous Markov information sources. Finally, we prove the AEP for a class of nonhomogeneous Markov information sources.

A two timescale stochastic approximation scheme which uses coupled iterations is used for simulation-based parametric optimization as an alternative to traditional “infinitesimal perturbation analysis” schemes. It avoids the aggregation of data present in many other schemes. Its convergence is analyzed, and a queueing example is presented.