Different Markov chains can be used for approximate sampling of a distribution given by an unnormalized density function with respect to the Lebesgue measure. The hit-and-run, (hybrid) slice sampler, and random walk Metropolis algorithm are popular tools to simulate such Markov chains. We develop a general approach to compare the efficiency of these sampling procedures by the use of a partial ordering of their Markov operators, the covariance ordering. In particular, we show that the hit-and-run and the simple slice sampler are more efficient than a hybrid slice sampler based on hit-and-run, which, itself, is more efficient than a (lazy) random walk Metropolis algorithm.

In this paper we consider the optimal scaling of high-dimensional random walk Metropolis algorithms for densities differentiable in the L p mean but which may be irregular at some points (such as the Laplace density, for example) and/or supported on an interval. Our main result is the weak convergence of the Markov chain (appropriately rescaled in time and space) to a Langevin diffusion process as the dimension d goes to ∞. As the log-density might be nondifferentiable, the limiting diffusion could be singular. The scaling limit is established under assumptions which are much weaker than the one used in the original derivation of Roberts et al. (1997). This result has important practical implications for the use of random walk Metropolis algorithms in Bayesian frameworks based on sparsity inducing priors.

Scaling of proposals for Metropolis algorithms is an important practical problem in Markov chain Monte Carlo implementation. Analyses of the random walk Metropolis for high-dimensional targets with specific functional forms have shown that in many cases the optimal scaling is achieved when the acceptance rate is approximately 0.234, but that there are exceptions. We present a general set of sufficient conditions which are invariant to orthonormal transformation of the coordinate axes and which ensure that the limiting optimal acceptance rate is 0.234. The criteria are shown to hold for the joint distribution of successive elements of a stationary pth-order multivariate Markov process.