Abstract

The Normal linear regression model with natural conjugate prior offers an attractive framework for carrying out Bayesian analysis of non- or semiparametric regression models. The points on the unknown nonparametric regression line can be treated as unobserved components. Prior information on the degree of smoothness of the nonparametric regression line can be combined with the data to yield a proper posterior, despite the fact that the number of parameters in the model is greater than the number of data points. In this paper, we investigate how much prior information is required in order to allow for empirical Bayesian inference about the nonparametric regression line. To be precise, if η is the parameter controlling the degree of smoothness in the nonparametric regression line, we investigate what the minimal amount of nondata information is such that η can be estimated in an empirical Bayes fashion. We show how this problem relates to the issue of estimation of the error variance in the state equation for a simple state space model. Our theoretical results are illustrated empirically using artificial data.

Introduction

The Normal linear regression model with natural conjugate prior provides an attractive framework for conducting Bayesian inference in non- or semi-parametric regression models. Not only does this well-understood model provide simple analytical results, it can easily be used as a component of a more complicated model (e.g. a nonparametric probit or tobit model or nonparametric regression with non-Normal errors). Since Bayesian analysis of more complicated models often requires posterior simulation (e.g. Markov chain Monte Carlo, MCMC, algorithms), having analytical results for the basic nonparametric regression component of the MCMC algorithm offers great computational simplifications.