Skip to main content
SpringerLink
Log in

Interacting multiple try algorithms with different proposal distributions

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

We introduce a new class of interacting Markov chain Monte Carlo (MCMC) algorithms which is designed to increase the efficiency of a modified multiple-try Metropolis (MTM) sampler. The extension with respect to the existing MCMC literature is twofold. First, the sampler proposed extends the basic MTM algorithm by allowing for different proposal distributions in the multiple-try generation step. Second, we exploit the different proposal distributions to naturally introduce an interacting MTM mechanism (IMTM) that expands the class of population Monte Carlo methods and builds connections with the rapidly expanding world of adaptive MCMC. We show the validity of the algorithm and discuss the choice of the selection weights and of the different proposals. The numerical studies show that the interaction mechanism allows the IMTM to efficiently explore the state space leading to higher efficiency than other competing algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Atchadé, Y., Roberts, G.O., Rosenthal, J.S.: Towards optimal scaling of Metropolis-coupled Markov chain Monte Carlo. Stat. Comput. 21, 555–568 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Barrett, M., Galipeau, P., Sanchez, C., Emond, M., Reid, B.: Determination of the frequency of loss of heterozygosity in esophageal adenocarcinoma nu cell sorting, whole genome amplification and microsatellite polymorphisms. Oncogene 12 (1996)

  • Bédard, M., Douc, R., Moulines, E.: Scaling analysis of multiple-try MCMC methods. Technical report, Université de Montréal (2010)

  • Campillo, F., Rakotozafy, R., Rossi, V.: Parallel and interacting Markov chain Monte Carlo algorithm. Math. Comput. Simul. 79, 3424–3433 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Cappé, O., Gullin, A., Marin, J., Robert, C.P.: Population Monte Carlo. J. Comput. Graph. Stat. 13, 907–927 (2004)

    Article  Google Scholar 

  • Casarin, R., Marin, J.-M.: Online data processing: Comparison of Bayesian regularized particle filters. Electron. J. Stat. 3, 239–258 (2009)

    Article  MathSciNet  Google Scholar 

  • Casarin, R., Marin, J.-M., Robert, C.: A discussion on: approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations by Rue, H., Martino, S. and Chopin, N. J. R. Stat. Soc. B 71, 360–362 (2009)

    Google Scholar 

  • Celeux, G., Marin, J.-M., Robert, C.: Iterated importance sampling in missing data problems. Comput. Stat. Data Anal. 50, 3386–3404 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Chauveau, D., Vandekerkhove, P.: Improving convergence of the Hastings-Metropolis algorithm with an adaptive proposal. Scand. J. Stat. 29, 13 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Craiu, R.V., Lemieux, C.: Acceleration of the multiple-try Metropolis algorithm using antithetic and stratified sampling. Stat. Comput. 17, 109–120 (2007)

    Article  MathSciNet  Google Scholar 

  • Craiu, R.V., Meng, X.L.: Multi-process parallel antithetic coupling for forward and backward MCMC. Ann. Stat. 33, 661–697 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Craiu, R.V., Rosenthal, J.S., Yang, C.: Learn from thy neighbor: parallel-chain adaptive and regional MCMC. J. Am. Stat. Assoc. 104, 1454–1466 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Del Moral, P.: Feynman-Kac Formulae. Genealogical and Interacting Particle Systems with Applications. Springer, Berlin (2004)

    MATH  Google Scholar 

  • Del Moral, P., Doucet, A., Jasra, A.: Sequential Monte Carlo samplers. J. R. Stat. Soc. B 68, 411–436 (2006)

    Article  MATH  Google Scholar 

  • Desai, M.: Mixture models for genetic changes in cancer cells. Ph.D. thesis, University of Washington (2000)

  • Früwirth-Schnatter, S.: Finite Mixture and Markov Switching Models. Springer, Berlin (2006)

    Google Scholar 

  • Gelman, A., Rubin, D.B.: Inference from iterative simulation using multiple sequences (with discussion). Stat. Sci. 457–511 (1992)

  • Geyer, C.J., Thompson, E.A.: Annealing Markov chain Monte Carlo with applications to ancestral inference. Tech. rep. 589, University of Minnesota (1994)

  • Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109 (1970)

    Article  MATH  Google Scholar 

  • Heard, N.A., Holmes, C., Stephens, D.: A quantitative study of gene regulation involved in the immune response od anophelinemosquitoes: an application of Bayesian hierarchical clustering of curves. J. Am. Stat. Assoc. 101, 18–29 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Jasra, A., Stephens, D.A., Holmes, C.: Markov chain Monte Carlo methods and the label switching problem in Bayesian mixture modelling. Stat. Sci. 20, 50–67 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  • Jasra, A., Stephens, D., Holmes, C.: On population-based simulation for static inference. Stat. Comput. 17, 263–279 (2007)

    Article  MathSciNet  Google Scholar 

  • Jennison, C.: Discussion of Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods, by A.F.M. Smith and G.O. Roberts. J. R. Stat. Soc. B 55, 54–56 (1993)

    Google Scholar 

  • Liu, J., Liang, F., Wong, W.: The multiple-try method and local optimization in Metropolis sampling. J. Am. Stat. Assoc. 95, 121–134 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  • Liang, F., Wong, W.: Real parameter evolutionary Monte Carlo with applications to Bayesian mixture models. J. Am. Stat. Assoc. 96, 653–666 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Marinari, E., Parisi, G.: Simulated tempering: a new Monte Carlo scheme. Europhys. Lett. 19, 451–458 (1992)

    Article  Google Scholar 

  • Mengersen, K., Robert, C.: The pinball sampler. In: Bernardo, J., Dawid, A., Berger, J., West, M. (eds.) Bayesian Statistics 7. Springer, Berlin (2003)

    Google Scholar 

  • Metropolis, N., Rosenbluth, A., Rosenbluth, M., Teller, A., Teller, E.: Equations of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092 (1953)

    Article  Google Scholar 

  • Neal, R.M.: Sampling from multimodal distributions using tempered transitions. Tech. rep. 9421, University of Toronto (1994)

  • Pandolfi, S., Bartolucci, F., Friel, N.: A generalization of the multiple-try Metropolis algorithm for Bayesian estimation and model selection. In: Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (AISTATS), Chia Laguna Resort, Sardinia, Italy, pp. 581–588 (2010a)

    Google Scholar 

  • Pandolfi, S., Bartolucci, F., Friel, N.: A generalized Multiple-try Metropolis version of the Reversible Jump algorithm. Tech. rep. (2010b). http://arxiv.org/pdf/1006.0621

  • Pritchard, J.K., Stephens, M., Donnelly, P.: Inference of population structure using multilocus genotype data. Genetics 155, 945–959 (2000)

    Google Scholar 

  • Richardson, S., Green, P.J.: On Bayesian analysis of mixtures with an unknown number of components (with discussion). J. R. Stat. Soc. B 4(59), 731–792 (1997)

    Article  MathSciNet  Google Scholar 

  • Shephard, N., Pitt, M.: Likelihood analysis of non-Gaussian measurement time series. Biometrika 84, 653–667 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  • So, M.K.P.: Bayesian analysis of nonlinear and non-Gaussian state space models via multiple-try sampling methods. Stat. Comput. 16, 125–141 (2006)

    Article  MathSciNet  Google Scholar 

  • Taylor, S.: Modelling stochastic volatility. Math. Finance 4, 183–204 (1994)

    Article  MATH  Google Scholar 

  • Warnes, G.: The Normal kernel coupler: an adaptive Markov chain Monte Carlo method for efficiently sampling from multi-modal distributions. Technical report, George Washington University (2001)

Download references

Author information

Authors and Affiliations

  1. Advanced School of Economics, Venice, Italy

    Roberto Casarin

  2. University Ca’ Foscari of Venice, Venice, Italy

    Roberto Casarin

  3. Universidad Carlos III de Madrid, Madrid, Spain

    Fabrizio Leisen

  4. University of Toronto, Toronto, Canada

    Radu Craiu

Authors
  1. Roberto Casarin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Radu Craiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabrizio Leisen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Radu Craiu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Casarin, R., Craiu, R. & Leisen, F. Interacting multiple try algorithms with different proposal distributions. Stat Comput 23, 185–200 (2013). https://doi.org/10.1007/s11222-011-9301-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-011-9301-9

Keywords

Search

Navigation