A single server processes jobs that can yield rewards but expire on predetermined dates. Expected immediate rewards from each job are deteriorating. The instance is formulated as a multiarmed bandit problem, and an index-based scheduling policy is shown to maximize the expected total reward.

An important problem in computer communication systems is the design of finite buffers such that a small overflow probability is achieved. For the case of batch input and general service times, we derive an asymptotically exponential expansion for the overflow probability when the batch-arrival process is Poissonian. Next, the results are extended to handle applications with periodic service opportunities and random interruptions in service. A convenient model to handle these types of applications is the versatile queueing model with exceptional first services.

A hooking network is built by stringing together components randomly chosen from a set of building blocks (graphs with hooks). The vertices are endowed with “affinities” which dictate the attachment mechanism. We study the distance from the master hook to a node in the network chosen according to its affinity after many steps of growth. Such a distance is commonly called the depth of the chosen node. We present an exact average result and a rather general central limit theorem for the depth. The affinity model covers a wide range of attachment mechanisms, such as uniform attachment and preferential attachment, among others. Naturally, the limiting normal distribution is parametrized by the structure of the building blocks and their probabilities. We also take the point of view of a visitor uninformed about the affinity mechanism by which the network is built. To explore the network, such a visitor chooses the nodes uniformly at random. We show that the distance distribution under such a uniform choice is similar to the one under random choice according to affinities.

Consider a sequence of independent and identically distributed random variables along with a specified set of k-vectors. We present an expression for E [T], the mean time until the last k observed random variables fall within this set. Not only can this expression often be used to obtain bounds on E[T], it also gives rise to an efficient way of approximating E[T] by a simulation. Specific lower and upper bounds for E[T] are also derived. These latter bounds are given in terms of a parameter, and a Markov chain Monte Carlo approach to approximate this parameter by a simulation is indicated. The results of this paper are illustrated by considering the problem of determining the mean time until a sequence of k-valued random variables has a run of size k that encompasses each value.

We show that the kth order statistic from a heterogeneous sample of n ≥ k exponential random variables is larger than that from a homogeneous exponential sample in the sense of star ordering, as conjectured by Xu and Balakrishnan [14]. As a consequence, we establish hazard rate ordering for order statistics between heterogeneous and homogeneous exponential samples, resolving an open problem of Pǎltǎnea [11]. Extensions to general spacings are also presented.

We investigate the tail distribution of the virtual waiting times in a LRD/GI/1 queue where the arrival process is long-range dependent (LRD) and the service times are independent and identically distributed (i.i.d.) random variables. We present two lower bounds on the stationary waiting time tail asymptotics, which illustrate the different dominating components that influence server performance under various conditions. In particular, we show that the tail distribution of the stationary waiting time is bounded below by that of the associated LRD/D/1 queues resulting from replacing all random service times by the mean. This shows the performance impact purely due to the long-range dependency of the arrival process. On the other hand, when the service times are subexponential, we show that the tail distribution of the stationary waiting time is bounded below by that of the corresponding D/GI/1 queue by replacing the dependent arrival process with its associated independent version. This shows the minimum performance impact due to the tail distribution of the service times. The above two lower bounds indicate that the performance of LRD/GI/1 queues will be dominated by the heavier tail of the corresponding LRD/D/1 and D/GI/1 queues. These features are further illustrated and quantified through examples and via numerous simulation experiments.

In this paper, we study the equilibrium valuation for currency options in a setting of the two-country Lucas-type economy. Different from the continuous model in Bakshi and Chen [1], we propose a discontinuous model with jump processes. Empirical findings reveal that the jump components in each country's money supply can be decomposed into the simultaneous co-jump component and the country-specific jump component. Each of the jump components is modeled with a Poisson process whose jump intensity follows a mean reversion stochastic process. By solving a partial integro-differential equation (PIDE), we get a closed-form solution to the PIDE for a European call currency option. The numerical results show that the derived option pricing formula is efficient for practical use. Importantly, we find that the co-jump has a significant impact on option price and implied volatility.

We study weighted path lengths (depths) and distances for increasing tree families. For those subclasses of increasing tree families, which can be constructed via an insertion process (e.g., recursive trees, plane-oriented recursive trees, and binary increasing trees), we can determine the limiting distribution that can be characterized as a generalized Dickman's infinitely divisible distribution.

This paper relaxes assumptions defining multivariate Brownian motion (BM) to construct processes with dependent increments as tractable models for problems in engineering and management science. We show that any Gaussian Markov process starting at zero and possessing stationary increments and a symmetric smooth kernel has a parametric kernel of a particular form, and we derive the unique unbiased, jointly sufficient, maximum-likelihood estimators of those parameters. As an application, we model a single-server queue driven by such a process and derive its transient distribution conditional on its history.

Motivated by applications to multi-antenna wireless networks, we propose a distributed and asynchronous algorithm for stochastic semidefinite programming. This algorithm is a stochastic approximation of a continuous-time matrix exponential scheme which is further regularized by the addition of an entropy-like term to the problem's objective function. We show that the resulting algorithm converges almost surely to an ɛ-approximation of the optimal solution requiring only an unbiased estimate of the gradient of the problem's stochastic objective. When applied to throughput maximization in wireless systems, the proposed algorithm retains its convergence properties under a wide array of mobility impediments such as user update asynchronicities, random delays and/or ergodically changing channels. Our theoretical analysis is complemented by extensive numerical simulations, which illustrate the robustness and scalability of the proposed method in realistic network conditions.

In this paper, we study the estimation of a scale parameter from a sample of lifetimes of coherent systems with a fixed structure. We assume that the components are independent and identically distributed having a common distribution which belongs to a scale parameter family. Some results are obtained as well for dependent (exchangeable) components. To this end, we will use the representations for the distribution function of a coherent system based on signatures. We prove that the efficiency of the estimators depends on the structure of the system and on the scale parameter family. In the dependence case, it also depends on the baseline copula function.

We consider time-inhomogeneous Markov chains on a finite state-space, whose transition probabilitiespij(t) = cijε(t)Vij are proportional to powers of a vanishing small parameter ε(t). We determine the precise relationship between this chain and the corresponding time-homogeneous chains pij= cijε(t)vij, as ε ↘ 0. Let {} be the steady-state distribution of this time-homogeneous chain. We characterize the orders {ηι} in = θ(εηι). We show that if ε(t) ↘ 0 slowly enough, then the timewise occupation measures βι := sup { q > 0 | Prob(x(t) = i) = + ∞}, called the recurrence orders, satisfy βi — βj = ηj — ηi. Moreover, : = { ηι|ηι = minj} is the set of ground states of the time-homogeneous chain, then x(t) → . in an appropriate sense, whenever η(t) is “cooled” slowly. We also show that there exists a critical ρ* such that x(t) → if and only if = + ∞. We characterize this critical rate as ρ* = max.min min max. Finally, we provide a graph algorithm for determining the orders [ηi] [βi] and the critical rate ρ*.

In the context of team recruitment, we discuss an optimal multiple stopping problem for an infinite independent and identically distributed sequence, with general reward function and constant observation cost. We establish the existence and nature of an optimal stopping rule. For the particular case where team quality is governed by the fitness of the weakest member, we show that the recruiter should be more discriminating with either a better, or a larger, group of appointees in hand.

A variety of methods for reducing the variance on Monte Carlo estimators of the expected waiting time Wn of the nth customer in a GI/G/1 queue are studied. The ideas involve Spitzer's identity, importance sampling, and sums with stratified or controlled randomized length.

This paper deals with pricing formulae for a European call option and an exchange option in the case where underlying asset price processes are represented by stochastic delay differential equations with jumps (hereafter “SDDEJ”). We introduce a new model in which Poisson jumps are added in stochastic delay differential equations to capture behaviors of an underlying asset process more precisely. We derive explicit pricing formulae for the European call option and the exchange option by proving a Lemma on the conditional expectation. Finally, we show that our “SDDEJ” model is meaningful through some numerical experiments and discussions.

We consider N unreliable machines that are maintained by M repairmen. The time until failure of machine i, and its repair by repairman j, are exponentially distributed random variable with parameters λi, and μj, respectively. All failure times and repair times are independent. Machine i earns ci, per unit time, while it is working. We wish to maximize the expected total discounted reward. It is shown that when the following conditions are satisfied — c1 ≥ … ≥ cN and c1/λ1≥…≥cN/λN, the policy that assigns the fastest repairman to the machine with the lowest index is optimal. Moreover, it is shown that when the second condition is replaced by λ≤ … ≤λN, then this policy maximizes, stochastically, the number of the most reliable machines at every time t.

This paper is concerned with how coupling can be used to enhance the efficiency of a certain class of terminating simulations, in Markov process settings in which the stationary distribution is known. We are able to theoretically establish that our coupling-based estimator is often more efficient than the naive estimator. In addition, we discuss extensions of our methodology to Markov process settings in which conventional coupling fails and show (for Doeblin chains) that knowledge of the stationary distribution is sometimes unnecessary.

Finite-state-and-action Markov branching decision chains are studied with bounded endogenous expected population sizes and interest-rate-dependent one-period rewards that are analytic in the interest rate at zero. The existence of a stationary strong-maximum-present-value policy is established. Miller and Veinott's [1969] strong policy-improvement method is generalized to find in finite time a stationary n-present-value optimal policy and, when the one-period rewards are rational in the interest rate, a stationary strong-maximum-present-value policy. This extends previous studies of Blackwell [1962], Miller and Veinott [1969], Veinott [1974], and Rothblum [1974, 1975], in which the one-period rewards are independent of the interest rate, and Denardo [1971] in which semi-Markov decision chains with small interest rates are studied. The problem of finding a stationary n-present-value optimal policy is also formulated as a staircase linear program in which the objective function and right-hand sides, but not the constraint matrix, depend on the interest rate, and solutions for all small enough positive interest rates are sought. The optimal solutions of the primal and dual are polynomials in the reciprocal of the interest rate. A constructive rule is given for finding a stationary n-present-value optimal policy from an optimal solution of the asymptotic linear program. This generalizes the linear programming approaches for finding maximum-reward-rate and maximum-present-value policies for Markov decision chains studied by Manne [1960], d'Epenoux [1960, 1963], Balinski [1961], Derman [1962], Denardo and Fox [1968], Denardo [1970], Derman and Veinott [1972], Veinott [1973], and Hordijk and Kallenberg [1979, 1984].

Diameter measurements on successive metal hubs from a machining operation are modeled using a random walk with observation error and linear drift corresponding to tool wear. After producing and measuring a hub, the depth of the cutting tool on the lathe can be adjusted in integer multiples of 0.0001 inches. How should the tool be adjusted?

An optimal discrete adjustment strategy is derived assuming that the lathe automatically corrects for deterministic tool wear. The objective is to minimize expected run costs proportional to the sum of squared diameter deviations from a target plus fixed charges for manual tool adjustments. The optimal strategy makes no manual adjustment if an estimate of the process mean is near target. Otherwise, an adjustment is made to return the estimated mean as near to target as possible within the adjustment resolution.

The region where no adjustments are made widens near the end of the production run where adjustments have only short-term impact. The region converges as the number of remaining periods increases. Plots of expected run costs show that the extra cost of discreteness is small at high resolution but is substantial when the adjustment grid is coarse.

We study the problem of scheduling multiclass parallel server queues where the cost for each job is a general class-dependent function of the job's completion time and a second random variable. We characterize the structure of the optimal policy under fairly general conditions on the random variables.