In this paper we investigate gradient estimation for a class of contracting stochastic systems on a continuous state space. We find conditions on the one-step transitions, namely differentiability and contraction in a Wasserstein distance, that guarantee differentiability of stationary costs. Then we show how to estimate the derivatives, deriving an estimator that can be seen as a generalization of the forward sensitivity analysis method used in deterministic systems. We apply the results to examples, including a neural network model.

We study the estimation of the mean function of a continuous-time stochastic process andits derivatives. The covariance function of the process is assumed to be nonparametric andto satisfy mild smoothness conditions. Assuming that n independent realizationsof the process are observed at a sampling design of size N generated by a positivedensity, we derive the asymptotic bias and variance of the local polynomial estimator asn,Nincrease to infinity. We deduce optimal sampling densities, optimal bandwidths, andpropose a new plug-in bandwidth selection method. We establish the asymptotic performanceof the plug-in bandwidth estimator and we compare, in a simulation study, its performancefor finite sizes n,N to the cross-validation and the optimalbandwidths. A software implementation of the plug-in method is available in the Renvironment.

Many applications of smoothed perturbation analysis lead to estimators with hazard rate functions of underlying distributions. A key assumption used in proving unbiasedness of the resulting estimator is that the hazard rate function be bounded, a restrictive assumption which excludes all distributions with finite support. Here, we prove through a simple example that this assumption can in fact be removed.

Given a parametric family of regenerative processes on a common probability space, we investigate when the derivatives (with respect to the parameter) are regenerative. We primarily consider sequences satisfying explicit, Lipschitz recursions, such as the waiting times in many queueing systems, and show that derivatives regenerate together with the original sequence under reasonable monotonicity or continuity assumptions. The inputs to our recursions are i.i.d. or, more generally, governed by a Harris-ergodic Markov chain. For i.i.d. input we identify explicit regeneration points; otherwise, we use coupling arguments. We give conditions for the expected steady-state derivative to be the derivative of the steady-state mean of the original sequence. Under these conditions, the derivative of the steady-state mean has a cycle-formula representation.