Skip to main content
×
×
Home

Random Construction of Interpolating Sets for High-Dimensional Integration

  • Mark Huber (a1) and Sarah Schott (a2)
Abstract

Computing the value of a high-dimensional integral can often be reduced to the problem of finding the ratio between the measures of two sets. Monte Carlo methods are often used to approximate this ratio, but often one set will be exponentially larger than the other, which leads to an exponentially large variance. A standard method of dealing with this problem is to interpolate between the sets with a sequence of nested sets where neighboring sets have relative measures bounded above by a constant. Choosing such a well-balanced sequence can rarely be done without extensive study of a problem. Here a new approach that automatically obtains such sets is presented. These well-balanced sets allow for faster approximation algorithms for integrals and sums using fewer samples, and better tempering and annealing Markov chains for generating random samples. Applications, such as finding the partition function of the Ising model and normalizing constants for posterior distributions in Bayesian methods, are discussed.

Copyright
Corresponding author
Postal address: Department of Mathematical Sciences, Claremont McKenna College, 850 Columbia Avenue, Claremont, CA 91711, USA. Email address: mhuber@cmc.edu
∗∗ Postal address: Department of Mathematics, Duke University, 117 Physics Building, Science Drive, Durham, NC 27708, USA. Email address: schott@math.duke.edu
References
Hide All
[1] Besag, J. (1974). Spatial interaction and the statistical analysis of lattice systems. J. R. Statist. Soc. B 36, 192236.
[2] Chernoff, H. (1952). A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Statist. 23, 493507.
[3] Dyer, M. and Frieze, A. (1991). Computing the volume of convex bodies: a case where randomness provably helps. In Probabilistic Combinatorics and Its Applications (Proc. Symp. Appl. Math. 44), American Mathematical Society, Providence, RI, pp. 123169.
[4] Fishman, G. S. (1994). Choosing sample path length and number of sample paths when starting in steady state. Operat. Res. Lett. 16, 209219.
[5] Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface, pp. 156163.
[6] Huber, M. L. (2012). Approximating algorithms for the normalizing constant of Gibbs distributions. Preprint. Available at http:/uk.arxiv.org/abs/1206.2689.
[7] Huber, M. and Schott, S. (2011). Using T{P}{A} for Bayesian inference. In Bayesian Statistics 9 (Proc. 9th Valencia Internat. Meeting), Oxford University Press, pp. 257282.
[8] Huber, M. L. and Wolpert, R. L. (2009). Likelihood-based inference for Matérn type-{III} repulsive point processes. Adv. Appl. Prob. 41, 958977.
[9] Jerrum, M. R., Valiant, L. G. and Vazirani, V. V. (1986). Random generation of combinatorial structures from a uniform distribution. Theoret. Comput. Sci. 43, 169188.
[10] Karatzas, I. and Shreve, S. E. (1991). Brownian Motion and Stochastic Calculus, 2nd edn. Springer, New York.
[11] Kirkpatrick, S., Gelatt, C. D. Jr. and Vecchi, M. P. (1983). Optimization by simulated annealing. Science 220, 671680.
[12] Marinari, E. and Parisi, G. (1992). Simulated tempering: a new Monte Carlo scheme. Europhys. Lett. 19, 451458.
[13] Motwani, R. and Raghavan, P. (1995). Randomized Algorithms. Cambridge University Press.
[14] Murray, I., Ghahramani, Z. and MacKay, D. J. C. (2006). MCMC for doubly-intractable distributions. In Proc. 22nd Annual Conf. Uncertainty Artificial Intelligence, AUAI Press, pp. 359366.
[15] Propp, J. G. and Wilson, D. B. (1996). Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures Algorithms 9, 223252.
[16] Robert, C. P. and Casella, G. (2004). Monte Carlo Statistical Methods, 2nd edn. Springer, New York.
[17] Shonkwiler, R. W. and Mendivil, F. (2009). Expolorations in Monte Carlo Methods. Springer, New York.
[18] Skilling, J. (2006). Nested sampling for general Bayesian computation. Bayesian Analysis 1, 833859.
[19] ŠtefankoviČ, D., Vempala, S. and Vigoda, E. (2009). Adaptive simulated annealing: a near-optimal connection between sampling and counting. J. ACM 56, 36pp.
[20] Swendsen, R. H. and Wang, J.-S. (1986). Replica Monte Carlo simulation of spin-glasses. Phys. Rev. Lett. 57, 26072609.
[21] Valleau, J. P. and Card, D. N. (1972). Monte Carlo estimation of the free energy by multistage sampling. J. Chem. Phys. 57, 54575462.
[22] Woodward, D. B., Schmidler, S. C. and Huber, M. (2009). Conditions for rapid mixing of parallel and simulated tempering on multimodal distributions. Ann. Appl. Prob. 19, 617640.
[23] Woodward, D. B., Schmidler, S. C. and Huber, M. (2009). Sufficient conditions for torpid mixing of parallel and simulated tempering. Electron. J. Prob. 14, 780804.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Journal of Applied Probability
  • ISSN: 0021-9002
  • EISSN: 1475-6072
  • URL: /core/journals/journal-of-applied-probability
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords

MSC classification

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed