In this paper, we study the problem of non parametric estimationof an unknown regression function from dependent data withsub-Gaussian errors. As a particular case, we handle theautoregressive framework. For this purpose, we consider acollection of finite dimensional linear spaces (e.g. linear spacesspanned by wavelets or piecewise polynomials on a possiblyirregular grid) and we estimate the regression function by aleast-squares estimator built on a data driven selected linearspace among the collection. This data driven choice is performedvia the minimization of a penalized criterion akin to the Mallows'C p . We state non asymptotic risk bounds for our estimator insome ${\mathbb{L}}_2$ -norm and we show that it is adaptive in the minimaxsense over a large class of Besov balls of the form Bα,p,∞(R) with p ≥ 1.

We establish the uniform almost sure convergence of the partitioning estimate, which is a histogram-like mean regression function estimate, under ergodic conditions for a stationary and unbounded process. The main application of our results concerns time series analysis and prediction in the Markov processes case.

A number of stationary stochastic processes are presented with properties pertinent to modelling time series from turbulence and finance. Specifically, the one-dimensional marginal distributions have log-linear tails and the autocorrelation may have two or more time scales. Discrete time models with a given marginal distribution are constructed as sums of independent autoregressions. A similar construction is made in continuous time by considering sums of Ornstein-Uhlenbeck-type processes. To prepare for this, a new property of self-decomposable distributions is presented. Also another, rather different, construction of stationary processes with generalized logistic marginal distributions as an infinite sum of Gaussian processes is proposed. In this way processes with continuous sample paths can be constructed. Multivariate versions of the various constructions are also given.

The Cauchy problem in the form of (1.11) with linear and constant coefficients is discussed. The solution (1.10) can be given in explicit form when the stochastic process is a multidimensional autoregression (AR) type, or Ornstein–Uhlenbeck process. Functionals of (1.10) form were studied by Kac in the Brownian motion case. The solutions are obtained with the help of the Radon–Nikodym transformation, proposed by Novikov [12].

The problem of estimating the transfer function of a linear system, together with the spectral density of an additive disturbance, is considered. The set of models used consists of linear rational transfer functions and the spectral densities are estimated from a finite-order autoregressive disturbance description. The true system and disturbance spectrum are, however, not necessarily of finite order. We investigate the properties of the estimates obtained as the number of observations tends to ∞ at the same time as the model order employed tends to ∞. It is shown that the estimates are strongly consistent and asymptotically normal, and an expression for the asymptotic variances is also given. The variance of the transfer function estimate at a certain frequency is related to the signal/noise ratio at that frequency and the model orders used, as well as the number of observations. The variance of the noise spectral estimate relates in a similar way to the squared value of the true spectrum.

We consider a multiple regression model in which the regressors are Fourier cosine vectors. These regressors are intended as approximations to ‘slowly changing' regressors of the kind often found in time series regression applications. The errors in the model are assumed to be generated by a special type of autoregressive model defined so that the regressors are eigenvectors of the quadratic forms occurring in the exponent of the probability density of the errors. This autoregression is intended as an approximation to the usual stationary autoregression. Both approximations are adopted for the sake of mathematical convenience.

Student's t-statistics are constructed for the autoregressive coefficients in a manner analogous to ordinary regression. It is shown that these statistics are distributed as Student's t to the first order of approximation, that is with errors in the density of order T−1/2, where T is the sample size, while the squares of the statistics are distributed as the square of Student's t to the second order of approximation, that is with errors in the density of order Τ–1.

Many population models which are far from stationarity can nevertheless be written in autoregressive format, perhaps with random coefficient. It is the thesis of this paper that procedures developed for stationary time series models are a useful guide to inferential results for population processes and may indeed be directly applicable. The illustrations concentrate on estimation of the matrix of mean vital rates in an age-structured population.

This paper is in three parts. The first deals with the algebraic and topological structure of spaces of rational transfer function linear systems—ARMAX systems, as they have been called. This structure theory is dominated by the concept of a space of systems of order, or McMillan degree, n, because of the fact that this space, M(n), can be realised as a kind of high-dimensional algebraic surface of dimension n(2s + m) where s and m are the numbers of outputs and inputs. In principle, therefore, the fitting of a rational transfer model to data can be considered as the problem of determining n and then the appropriate element of M(n). However, the fact that M(n) appears to need a large number of coordinate neighbourhoods to cover it complicates the task. The problems associated with this program, as well as theory necessary for the analysis of algorithms to carry out aspects of the program, are also discussed in this first part of the paper, Sections 1 and 2.

The second part, Sections 3 and 4, deals with algorithms to carry out the fitting of a model and exhibits these algorithms through simulations and the analysis of real data.

The third part of the paper discusses the asymptotic properties of the algorithm. These properties depend on uniform rates of convergence being established for covariances up to some lag increasing indefinitely with the length of record, T. The necessary limit theorems and the analysis of the algorithms are given in Section 5. Many of these results are of interest independent of the algorithms being studied.

The stationary autoregressive model but with a long-tailed error distribution is analysed using the method of functional least squares. A family of estimators indexed by a real parameter is obtained and uniform consistency and weak convergence established. The optimum member of the family is chosen to have minimum variance with respect to the parameter, and the parameter value chosen detects and adjusts for long-tailed error distributions. Results of a simulation are given.

We consider an L1 analogue of the least squares estimator for the parameters of stationary, finite-order autoregressions. This estimator, the least absolute deviation (LAD), is shown to be strongly consistent via a result that may have independent interest. The striking feature is that the conditions are so mild as to include processes with infinite variance, notably the stationary, finite autoregressions driven by stable increments in Lα, α > 1. Finally, sampling properties of LAD are compared to those of least squares. Together with a known convergence rate result for least squares, the Monte Carlo study provides evidence for a conjecture on the convergence rate of LAD.

The least squares estimators βi(N), j = 1, …, p, from N data points, of the autoregressive constants for a stationary autoregressive model are considered when the disturbances have a distribution attracted to a stable law of index α < 2. It is shown that N1/δ(βi(N) – β) converges almost surely to zero for any δ > α. Some comments are made on alternative definitions of the βi(N).

Given a ‘black box’ of logical elements, a sequence of independent random pulses is fed in and the output from the black box is observed. A procedure is given for estimating the parameters of the black box, in the case when the black box is constructed in a manner analogous to the classical autoregressive scheme.