Book contents
- Frontmatter
- Contents
- Preface to the Second Edition
- Preface to the First Edition
- 1 Algorithms and Computers
- 2 Computer Arithmetic
- 3 Matrices and Linear Equations
- 4 More Methods for Solving Linear Equations
- 5 Regression Computations
- 6 Eigenproblems
- 7 Functions: Interpolation, Smoothing, and Approximation
- 8 Introduction to Optimization and Nonlinear Equations
- 9 Maximum Likelihood and Nonlinear Regression
- 10 Numerical Integration and Monte Carlo Methods
- 11 Generating Random Variables from Other Distributions
- 12 Statistical Methods for Integration and Monte Carlo
- 13 Markov Chain Monte Carlo Methods
- 14 Sorting and Fast Algorithms
- Author Index
- Subject Index
- References
14 - Sorting and Fast Algorithms
Published online by Cambridge University Press: 01 June 2011
Book contents
- Frontmatter
- Contents
- Preface to the Second Edition
- Preface to the First Edition
- 1 Algorithms and Computers
- 2 Computer Arithmetic
- 3 Matrices and Linear Equations
- 4 More Methods for Solving Linear Equations
- 5 Regression Computations
- 6 Eigenproblems
- 7 Functions: Interpolation, Smoothing, and Approximation
- 8 Introduction to Optimization and Nonlinear Equations
- 9 Maximum Likelihood and Nonlinear Regression
- 10 Numerical Integration and Monte Carlo Methods
- 11 Generating Random Variables from Other Distributions
- 12 Statistical Methods for Integration and Monte Carlo
- 13 Markov Chain Monte Carlo Methods
- 14 Sorting and Fast Algorithms
- Author Index
- Subject Index
- References
Summary
Introduction
The theme of this chapter is a simple one: there may be better, faster ways of computing something than you may ever have thought of. One of the maxims of computer science is that just a few programs use most of the resources. Early in the history of computing, people recognized that much of the computing resources went into the common task of sorting a list of numbers. If this task could be done faster, then everything could be done better. A concentrated effort on improving sorting algorithms led to several breakthroughs, all following the principle known as “divide and conquer.” In simple terms, to solve a large task, break it into smaller tasks of the same kind. Cleverly done, the resulting algorithm can be more efficient than anyone would have expected, earning the jargon adjective “fast.” The principle of divide and conquer will be discussed in the next section, followed by a discussion of fast algorithms for sorting. Section 14.4 comprises statistical applications of divide and conquer. Another great breakthrough, the fast Fourier transform (FFT), will be discussed in Section 14.5. Using the FFT to compute convolutions will be discussed in Section 14.6, followed by some interesting applications of the FFT to statistics in Section 14.7. This chapter will close with some topics that are important but don't really fit elsewhere: algorithms for constructing permutations and combinations.
- Type
- Chapter
- Information
- Numerical Methods of Statistics , pp. 403 - 438Publisher: Cambridge University PressPrint publication year: 2011
References
, , and (1974), The Design and Analysis of Computer Algorithms. Reading, MA: Addison-Wesley.Google Scholar
, , and (1992), “Methods for Exact Goodness-of-Fit Tests,” Journal of the American Statistical Association 87: 464–9.CrossRefGoogle Scholar
(1970), “A Linear Filtering Approach to the Computation of the Discrete Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 18: 451–5.CrossRefGoogle Scholar
(1975), “Numerical Inversions of Characteristic Functions,” Scandinavian Actuarial Journal, pp. 121–4.CrossRefGoogle Scholar
(1967), “Algorithm 29: Permutation of the Elements of a Vector,” Computer Journal 10: 311.Google Scholar
, , and (1967), “Historical Notes on the Fast Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 15: 76–9.CrossRefGoogle Scholar
and (1965), “An Algorithm for the Machine Calculation of Complex Fourier Series,” Mathematics of Computation 19: 297–301.CrossRefGoogle Scholar
(1975), “Numerical Inversion of a Characteristic Function,” Biometrika 60: 415–17.CrossRefGoogle Scholar
(1980), “ASS 155: The Distribution of a Linear Combination of Chi-Squared Random Variables,” Applied Statistics 29: 323–33.CrossRefGoogle Scholar
, , and (1970), “Exact Distribution for χ2 and for the Likelihood-Ratio Statistic for the Equiprobable Multinomial Distribution,” Journal of the American Statistical Association 65: 267–83.Google Scholar
(1961), “Algorithm 63 (PARTITION) and Algorithm 65 (FIND),” Communications of the ACM 4: 321–2.CrossRefGoogle Scholar
and (1963), “Estimates of Location Based on Rank Tests,” Annals of Mathematical Statistics 34: 598–611.CrossRefGoogle Scholar
(1961), “Computing the Distribution of Quadratic Forms in Normal Variables,” Biometrika 48: 419–26.CrossRefGoogle Scholar
and (1978), “Lower Bounds for Selection in X + Y and Other Multisets,” Journal of the ACM 25: 556–70.CrossRefGoogle Scholar
(1974), “A Numbering System for Combinations,” Communications of the ACM 17: 45–6.CrossRefGoogle Scholar
(1997), The Art of Computer Programming (vol. 2: Seminumerical Algorithms), 3rd ed. Reading, MA: Addison-Wesley.Google Scholar
(1998), The Art of Computer Programming (vol. 3: Sorting and Searching), 2nd ed. Reading, MA: Addison-Wesley.Google Scholar
and (2002), “Improving the Numerical Technique for Computing the Accumulated Distribution of a Quadratic Form in Normal Variables,” Econometric Reviews 21: 149–65.Google Scholar
and (1977), “Algorithm 516: An Algorithm for Obtaining Confidence Intervals and Point Estimates Based on Rank in a Two-Sample Location Problem,” ACM Transactions on Mathematical Software 3: 183–5.CrossRefGoogle Scholar
and (1986), “FEXACT: A Fortran Subroutine for Fisher's Exact Test on Unordered r × c Contingency Tables,” ACM Transactions on Mathematical Software 12: 154–61.CrossRefGoogle Scholar
(1984), “Fast Computation of the Hodges–Lehmann Location Estimator,” ACM Transactions on Mathematical Software 10: 265–70.CrossRefGoogle Scholar
(1987), “An Alternative Method for Computing Overflow Probabilities,” Communications in Statistics A 16: 3355–7.CrossRefGoogle Scholar
and (1983), “On Obtaining Permutation Distributions in Polynomial Time,” Journal of the American Statistical Association 78: 435–40.CrossRefGoogle Scholar
, , and (1969), “The Chirp-z Transform Algorithm and Its Applications,” Bell System Technical Journal 48: 1249–92.CrossRefGoogle Scholar
(1987), “Computing the Probability of Hash Table/Urn Overflow,” Communications in Statistics A 16: 3343–53.CrossRefGoogle Scholar
and (1988), “Fast Computation of the Hodges–Lehmann Estimator and Its Associated Confidence Limits,” American Statistical Association Proceedings of the Statistical Computing Section, pp. 187–91.Google Scholar
(1963), “Algorithm 202: Generation of Permutations in Lexicographical Order,” Communications of the ACM 6: 517.CrossRefGoogle Scholar
(1982), “Kernel Density Estimation Using the Fast Fourier Transform,” Applied Statistics 31: 93–7.CrossRefGoogle Scholar
(1969), “An Algorithm for Computing the Mixed-Radix Fast Fourier Transform,” IEEE Transactions on Audio and Electroacoustics 17: 93–103.CrossRefGoogle Scholar
and (1991), “Efficient Calculation of the Permutation Distribution of Trimmed Means,” Journal of the American Statistical Association 86: 729–37.CrossRefGoogle Scholar
(1984a), “On Inverting Permutation Tests,” Journal of the American Statistical Association 79: 200–7.CrossRefGoogle Scholar
(1984b), “An Algorithm for Exact Logistic Regression,” Journal of the American Statistical Association 79: 709–11.CrossRefGoogle Scholar
(1961), “Generation of Permutations by Transposition,” Mathematics of Computation 15: 192–5.CrossRefGoogle Scholar
- 1
- Cited by