Skip to main content

Estimating effective population size from samples of sequences: a bootstrap Monte Carlo integration method

  • Joseph Felsenstein (a1)

We would like to use maximum likelihood to estimate parameters such as the effective population size Ne, or, if we do not know mutation rates, the product 4Neμof mutation rate per site and effective population size. To compute the likelihood for a sample of unrecombined nucleotide sequences taken from a random-mating population it is necessary to sum over all genealogies that could have led to the sequences, computing for each one the probability that it would have yielded the sequences, and weighting each one by its prior probability. The genealogies vary in tree topology and in branch lengths. Although the likelihood and the prior are straightforward to compute, the summation over all genealogies seems at first sight hopelessly difficult. This paper reports that it is possible to carry out a Monte Carlo integration to evaluate the likelihoods pproximately. The method uses bootstrap sampling of sites to create data sets for each of which a maximum likelihood tree is estimated. The resulting trees are assumed to be sampled from a distribution whose height is proportional to the likelihood surface for the full data. That it will be so is dependent on a theorem which is not proven, but seems likely to be true if the sequences are not short. One can use the resulting estimated likelihood curve to make a maximum likelihood estimate of the parameter of interest, Ne or of 4Neμ. The method requires at least 100 times the computational effort required for estimation of a phylogeny by maximum likelihood, but is practical on today's work stations. The method does not at present have any way of dealing with recombination.

Hide All
Avise, J. C. (1989). Gene trees and organismal histories: a phylogenetic approach to population biology. Evolution 43, 11921208
Cann, R. L., Stoneking, M. & Wilson, A. C. (1987). Mitochondrial DNA and human evolution. Nature 325, 3136.
Edwards, A. W. F. (1970). Estimation of the branch points of a branching diffusion process. Journal of the Royal Statistical Society, B32, 155174.
Efron, B. (1979). Bootstrap methods: another look at the jackknife. Annals of Statistics 7, 126.
Efron, B. (1982). The Jackknife, the Bootstrap and Other Resampling Plans. Philadelphia: Society for Industrial and Applied Mathematics.
Felsenstein, J. (1981). Evolutionary trees from gene frequencies and quantitative characters: finding maximum likelihood estimates. Evolution 35, 12291242.
Felsenstein, J. (1985). Confidence limits on phylogenies with a molecular clock. Systematic Zoology 34, 152161.
Felsenstein, J. (1988). Phylogenies from molecular sequences: inference and reliability. Annual Review of Genetics 22, 521565.
Felsenstein, J. (1992). Estimating effective population size from samples of sequences: inefficiency of pairwise and segregating sites as compared to phylogenetic estimates. Genetical Research 59, 139147.
Griffiths, R. C. (1989). Genealogical tree probabilities in the infinitely-many-site model. Journal of Mathematical Biology 27, 667680.
Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97109.
Hammersley, J. M. & Handscomb, D. C. (1964). Monte Carlo Methods. London: Methuen.
Jukes, T. H. & Cantor, C. (1969). Evolution of protein molecules. In Mammalian Protein Metabolism (ed. Munro, M. N.), pp. 21132. New York: Academic Press.
Kahn, H. (1950). Random sampling (Monte Carlo) techniques in neutron attenuation problems. I. Nucleonics 6 (5), 2737.
Kendall, M. G. & Stewart, A. (1973). The Advanced Theory of Statistics, Vol. 2, 3rd Edn.New York: Hafner.
Kimura, M. & Ohta, T. (1972). On the stochastic model for estimation of mutational distance between homologous proteins. Journal of Molecular Evolution 2, 8790.
Kingman, J. F. C. (1982 a). The coalescent. Stochastic Processes and Their Applications 13, 235248.
Kingman, J. F. C. (1982 b). On the genealogy of large populations. Journal of Applied Probability 19 A, 2743.
Künsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. Annals of Statistics 17, 12171241.
Margush, T. & McMorris, F. R. (1981). Consensus n-trees. Bulletin of Mathematical Biology 43, 239244.
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H. & Teller, E. (1953). Equation of state calculations by fast computing machines. Journal of Chemical Physics 21, 10871092.
Nei, M. & Tajima, F. (1981) DNA polymorphism detectable by restriction endonucleases. Genetics 97, 145163.
Strobeck, C. (1983). Estimation of the neutral mutation rate in a finite population from DNA sequence data. Theoretical Population Biology 24, 160172.
Watterson, G. A. (1975) On the number of segregating sites in genetical models without recombination. Theoretical Population Biology 7, 256276.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Genetics Research
  • ISSN: 0016-6723
  • EISSN: 1469-5073
  • URL: /core/journals/genetics-research
Please enter your name
Please enter a valid email address
Who would you like to send this to? *