We consider infinitely wide multi-layer perceptrons (MLPs) which are limits of standard deep feed-forward neural networks. We assume that, for each layer, the weights of an MLP are initialized with independent and identically distributed (i.i.d.) samples from either a light-tailed (finite-variance) or a heavy-tailed distribution in the domain of attraction of a symmetric $\alpha$-stable distribution, where $\alpha\in(0,2]$ may depend on the layer. For the bias terms of the layer, we assume i.i.d. initializations with a symmetric $\alpha$-stable distribution having the same $\alpha$ parameter as that layer. Non-stable heavy-tailed weight distributions are important since they have been empirically seen to emerge in trained deep neural nets such as the ResNet and VGG series, and proven to naturally arise via stochastic gradient descent. The introduction of heavy-tailed weights broadens the class of priors in Bayesian neural networks. In this work we extend a recent result of Favaro, Fortini, and Peluchetti (2020) to show that the vector of pre-activation values at all nodes of a given hidden layer converges in the limit, under a suitable scaling, to a vector of i.i.d. random variables with symmetric $\alpha$-stable distributions, $\alpha\in(0,2]$.

We study the tail behaviour of the distribution of the area under the positive excursion of a random walk which has negative drift and heavy-tailed increments. We determine the asymptotics for tail probabilities for the area.

Assessing conditional tail risk at very high or low levels is of great interest in numerous applications. Due to data sparsity in high tails, the widely used quantile regression method can suffer from high variability at the tails, especially for heavy-tailed distributions. As an alternative to quantile regression, expectile regression, which relies on the minimization of the asymmetric l2-norm and is more sensitive to the magnitudes of extreme losses than quantile regression, is considered. In this article, we develop a new estimation method for high conditional tail risk by first estimating the intermediate conditional expectiles in regression framework, and then estimating the underlying tail index via weighted combinations of the top order conditional expectiles. The resulting conditional tail index estimators are then used as the basis for extrapolating these intermediate conditional expectiles to high tails based on reasonable assumptions on tail behaviors. Finally, we use these high conditional tail expectiles to estimate alternative risk measures such as the Value at Risk (VaR) and Expected Shortfall (ES), both in high tails. The asymptotic properties of the proposed estimators are investigated. Simulation studies and real data analysis show that the proposed method outperforms alternative approaches.

We consider the stationary solution Z of the Markov chain {Zn}n∈ℕ defined by Zn+1=ψn+1(Zn), where {ψn}n∈ℕ is a sequence of independent and identically distributed random Lipschitz functions. We estimate the probability of the event {Z>x} when x is large, and develop a state-dependent importance sampling estimator under a set of assumptions on ψn such that, for large x, the event {Z>x} is governed by a single large jump. Under natural conditions, we show that our estimator is strongly efficient. Special attention is paid to a class of perpetuities with heavy tails.

In this paper we study the number of customers in infinite-server queues with a self-exciting (Hawkes) arrival process. Initially we assume that service requirements are exponentially distributed and that the Hawkes arrival process is of a Markovian nature. We obtain a system of differential equations that characterizes the joint distribution of the arrival intensity and the number of customers. Moreover, we provide a recursive procedure that explicitly identifies (transient and stationary) moments. Subsequently, we allow for non-Markovian Hawkes arrival processes and nonexponential service times. By viewing the Hawkes process as a branching process, we find that the probability generating function of the number of customers in the system can be expressed in terms of the solution of a fixed-point equation. We also include various asymptotic results: we derive the tail of the distribution of the number of customers for the case that the intensity jumps of the Hawkes process are heavy tailed, and we consider a heavy-traffic regime. We conclude by discussing how our results can be used computationally and by verifying the numerical results via simulations.

We study the tail asymptotic of subexponential probability densities on the real line. Namely, we show that the n-fold convolution of a subexponential probability density on the real line is asymptotically equivalent to this density multiplied by n. We prove Kesten's bound, which gives a uniform in n estimate of the n-fold convolution by the tail of the density. We also introduce a class of regular subexponential functions and use it to find an analogue of Kesten's bound for functions on ℝd. The results are applied to the study of the fundamental solution to a nonlocal heat equation.

We consider the Δ(i)/G/1 queue, in which a total of n customers join a single-server queue for service. Customers join the queue independently after exponential times. We consider heavy-tailed service-time distributions with tails decaying as x-α, α ∈ (1, 2). We consider the asymptotic regime in which the population size grows to ∞ and establish that the scaled queue-length process converges to an α-stable process with a negative quadratic drift. We leverage this asymptotic result to characterize the head start that is needed to create a long period of uninterrupted activity (a busy period). The heavy-tailed service times should be contrasted with the case of light-tailed service times, for which a similar scaling limit arises (Bet et al. (2015)), but then with a Brownian motion instead of an α-stable process.

Retransmission-based failure recovery represents a primary approach in existing communication networks that guarantees data delivery in the presence of channel failures. Recent work has shown that, when data sizes have infinite support, retransmissions can cause long (-tailed) delays even if all traffic and network characteristics are light-tailed. In this paper we investigate the practically important case of bounded data units 0 ≤ Lb ≤ b under the condition that the hazard functions of the distributions of data sizes and channel statistics are proportional. To this end, we provide an explicit and uniform characterization of the entire body of the retransmission distribution ℙ[Nb > n] in both n and b. Our main discovery is that this distribution can be represented as the product of a power law and gamma distribution. This rigorous approximation clearly demonstrates the coupling of a power law distribution, dominating the main body, and the gamma distribution, determining the exponential tail. Our results are validated via simulation experiments and can be useful for designing retransmission-based systems with the required performance characteristics. From a broader perspective, this study applies to any other system, e.g. computing, where restart mechanisms are employed after a job processing failure.

The infinite source Poisson arrival model with heavy-tailed workload distributions has attracted much attention, especially in the modeling of data packet traffic in communication networks. In particular, it is well known that under suitable assumptions on the source arrival rate, the centered and scaled cumulative workload input process for the underlying processing system can be approximated by fractional Brownian motion. In many applications one is interested in the stabilization of the work inflow to the system by modifying the net input rate, using an appropriate admission control policy. In this paper we study a natural family of admission control policies which keep the associated scaled cumulative workload input asymptotically close to a prespecified linear trajectory, uniformly over time. Under such admission control policies and with natural assumptions on arrival distributions, suitably scaled and centered cumulative workload input processes are shown to converge weakly in the path space to the solution of a d-dimensional stochastic differential equation driven by a Gaussian process. It is shown that the admission control policy achieves moment stabilization in that the second moment of the solution to the stochastic differential equation (averaged over the d-stations) is bounded uniformly for all times. In one special case of control policies, as time approaches ∞, we obtain a fractional version of a stationary Ornstein-Uhlenbeck process that is driven by fractional Brownian motion with Hurst parameter H > ½.

We introduce a new approach to simulating rare events for Markov random walks with heavy-tailed increments. This approach involves sequential importance sampling and resampling, and uses a martingale representation of the corresponding estimate of the rare-event probability to show that it is unbiased and to bound its variance. By choosing the importance measures and resampling weights suitably, it is shown how this approach can yield asymptotically efficient Monte Carlo estimates.

In this paper we show that fractional Brownian motion with H < ½ can arise as a limit of a simple class of traffic processes that we call ‘scheduled traffic models’. To our knowledge, this paper provides the first simple traffic model leading to fractional Brownnian motion with H < ½. We also discuss some immediate implications of this result for queues fed by scheduled traffic, including a heavy-traffic limit theorem.

It is shown that the sample paths of Poisson shot noise with heavy-tailed semiexponential distributions satisfy a large deviation principle with a rate function that is insensitive to the shot shape. This demonstrates that, on the scale of large deviations, paths to rare events do not depend on the shot shape.

We study asymptotically optimal simulation algorithms for approximating the tail probability of P(eX1+⋯+ eXd>u) as u→∞. The first algorithm proposed is based on conditional Monte Carlo and assumes that (X1,…,Xd) has an elliptical distribution with very mild assumptions on the radial component. This algorithm is applicable to a large class of models in finance, as we demonstrate with examples. In addition, we propose an importance sampling algorithm for an arbitrary dependence structure that is shown to be asymptotically optimal under mild assumptions on the marginal distributions and, basically, that we can simulate efficiently (X1,…,Xd|Xj >b) for large b. Extensions that allow us to handle portfolios of financial options are also discussed.

In this paper we consider a risk model where claims arrive according to a Markovian arrival process (MAP). When the surplus becomes negative or the insurer is in deficit, the insurer could borrow money at a constant debit interest rate to repay the claims. We derive the integro-differential equations satisfied by the discounted penalty functions and discuss the solutions. A matrix renewal equation is obtained for the discounted penalty function provided that the initial surplus is nonnegative. Based on this matrix renewal equation, we present some asymptotic formulae for the discounted penalty functions when the claim size distributions are heavy tailed.

In this paper we consider a discrete-time process which grows according to a random walk with nonnegative increments between crash times at which it collapses to 0. We assume that the probability of crashing depends on the level of the process. We study the stochastic stability of this growth-collapse process. Special emphasis is given to the case in which the probability of crashing tends to 0 as the level of the process increases. In particular, we show that the process may exhibit long-range dependence and that the crash sizes may have a power law distribution.

Let ξ, ξ1, ξ2,… be a sequence of independent and identically distributed random variables, and let Sn=ξ1+⋯+ξn and Mn=maxk≤nSk. Let τ=min{n≥1: Sn≤0}. We assume that ξ has a heavy-tailed distribution and negative, finite mean E(ξ)<0. We find the asymptotics of P{Mτ ∈ (x, x+T]} as x→∞, for a fixed, positive constant T.

Nonexponential asymptotics for solutions of two specific defective renewal equations are obtained. These include the special cases of asymptotics for a compound geometric distribution and the convolution of a compound geometric distribution with a distribution function. As applications of these results, we study the asymptotic behavior of the demographic birth rate of females, the perpetual put option in mathematics of finance, and the renewal function for terminating renewal processes.

Consider a random walk S=(Sn: n≥0) that is ‘perturbed’ by a stationary sequence (ξn: n≥0) to produce the process S=(Sn+ξn: n≥0). In this paper, we are concerned with developing limit theorems and approximations for the distribution of Mn=max{Sk+ξk: 0≤k≤n} when the random walk has a drift close to 0. Such maxima are of interest in several modeling contexts, including operations management and insurance risk theory. The associated limits combine features of both conventional diffusion approximations for random walks and extreme-value limit theory.

Let (Yn, Nn)n≥1 be independent and identically distributed bivariate random variables such that the Nn are positive with finite mean ν and the Yn have a common heavy-tailed distribution F. We consider the process (Zn)n≥1 defined by Zn = Yn - Σn-1, where It is shown that the probability that the maximum M = maxn≥1Zn exceeds x is approximately as x → ∞, where F' := 1 - F. Then we study the integrated tail of the maximum of a random walk with long-tailed increments and negative drift over the interval [0, σ], defined by some stopping time σ, in the case in which the randomly stopped sum is negative. Finally, an application to risk theory is considered.

In this paper, we discuss max-sum equivalence and convolution closure of heavy-tailed distributions. We generalize the well-known max-sum equivalence and convolution closure in the class of regular variation to two larger classes of heavy-tailed distributions. As applications of these results, we study asymptotic behaviour of the tails of compound geometric convolutions, the ruin probability in the compound Poisson risk process perturbed by an α-stable Lévy motion, and the equilibrium waiting-time distribution of the M/G/k queue.