Organizers:

Xiao-Li Meng (Chair), meng@galton.uchicago.edu David van Dyk, vandyk@stat.harvard.edu

Speakers

10:30 a.m.
The Art of Data Augmentation: From EM to MCMC
David van Dyk, Department of Statistics, Harvard University
Xiao-Li Meng, Department of Statistics, University of Chicago
Keywords: Auxiliary variables; ; EM algorithm; Gibbs sampler; Hierarchical model; Markov Chain Monte Carlo; Mixed-effects models; Posterior sampling; Rate of convergence

In recent years, the variety of methods for Markov Chain Monte Carlo has grown at an astounding rate. We describe a number of recent extension to Tanner and Wong's (1987) seminal paper on Data Augmentation (DA), which was popularized earlier as the deterministic EM algorithm in statistics and as auxiliary variables (e.g.,Swendsen and Wang's (1987) algorithm) in physics. Tanner and Wong used DA to simplify simulation, while auxiliary variables aim to improve speed. In general, however, constructing DA schemes that result in both simple and fast algorithms is a matter of art since strategies vary greatly with the models being considered. We introduce an effective general strategy which combines the ideas of marginal DA and conditional DA with regards to a working parameter, together with a deterministic approximation method for selecting prior distributions for the working parameter that result in good DA schemes. We then apply this strategy to mixed-effects models, to obtain efficient Markov chain Monte Carlo algorithms for posterior sampling. We conclude with a comparison of the deterministic approximation with the use of Haar measure as a prior on the working parameter.

11:00 a.m.
Transformation Group and MCMC
Jun S. Liu, Department of Statistics, Stanford University
Ying Nian Wu, Department of Statistics, University of Michigan

The Gibbs sampler (Geman and Geman, 1984; Gelfand and Smith, 1990) is a popular Markov China Monte Carlo (MCMC) scheme for simulating high dimensional distributions, where the random variable to be simulated is decomposed into one (or low) dimensional components, and the transition of the underlying Markov chain consists of a sequence of conditional moves, with each one updating a component conditional on the current values of other components. We propose a generalization of the Gibbs sampler, where the conditional moves along the coordinate axes (or the subspaces of the components) are generalized to the conditional moves along the orbits of transformation groups. We give an explicit formula for the conditional distribution along the orbits, where an important component is the Haar measure on the group of transformations. With more flexible conditional moves, this scheme can lead to more efficient MCMC algorithms. In particular, we study two methods for accelerating MCMC using this scheme. One is the parameter expanded data augmentation (PX-DA) algorithm and the other is the windowed Gibbs sampler in image simultation.

11:30 a.m.
The Use of Multiple-Try Method and Local Optimization in Metropolis Sampling
Jun S. Liu, Department of Statistics, Stanford University
Faming Liang, National University of Singapore
Wing Hung Wong, UCLA

We propose two novel techniques for general Metropolis sampling.The first one is a new Metropolis-type transition rule, which we call the Multiple-try Metropolis (MTM). The new rule is a nontrivial twist of the original Metropolis-Hastings algorithms; and can be used to accommodate moves with very large stepsize. Our second finding is that the Adaptive Direction Sampling framework of Gilks, Roberts, and George (1994) can be extended, together with the MTM, to rigorously incorporate local deterministic moves into a Markov chain Monte Carlo sampler for simulating random variables in continuous state-space. Numerical studies show that the new method performs significantly better than the traditional Metropolis-Hastings sampler. With minor tailoring in using the rule, the multiple-try method can be used to explore a fixed or random direction along the sample space without having to know how to draw from the required conditional distribution. These techniques, together with the method of exchange Monte Carlo and genetic-type of moves, can be used to accelerate Monte Carlo simulations of large molecules.