Hostname: page-component-89b8bd64d-ksp62 Total loading time: 0 Render date: 2026-05-06T23:00:29.040Z Has data issue: false hasContentIssue false
  • English
  • Français

Numerical algorithms for constrained maximum likelihood estimation

Published online by Cambridge University Press:  17 February 2009

Z. F. Li ,
M. R. Osborne  and
T. Prvan
Z. F. Li
Affiliation:
National Centre for Epidemiology and Population Health, Australian National University, Canberra, ACT 0200, Australia; e-mail: zhengfeng.li@anu.edu.au.
M. R. Osborne
Affiliation:
Centre for Mathematics and its Applications, School of Mathematical Sciences, Australian National University, Canberra, ACT 0200, Australia; e-mail: Mike.Osborne@maths.anu.edu.au.
T. Prvan
Affiliation:
School of Mathematics and Statistics, The University of Canberra, Canberra, ACT 2617, Australia; e-mail: TaniaP@ise.canberra.edu.au.
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

This paper describes a SQP-type algorithm for solving a constrained maximum likelihood estimation problem that incorporates a number of novel features. We call it MLESOL. MLESOL maintains the use of an estimate of the Fisher information matrix to the Hessian of the negative log-likelihood but also encompasses a secant approximation S to the second-order part of the augmented Lagrangian function along with tests for when to use this information. The local quadratic model used has a form something like that of Tapia's SQP augmented scale BFGS secant method but explores the additional structure of the objective function. The step choice algorithm is based on minimising a local quadratic model subject to the linearised constraints and an elliptical trust region centred at the current approximate minimiser. This is accomplished using the Byrd and Omojokun trust region approach, together with a special module for assessing the quality of the step thus computed. The numerical performance of MLESOL is studied by means of an example involving the estimation of a mixture density.

Information

Type
Research Article
Information
The ANZIAM Journal , Volume 45 , Issue 1 , July 2003 , pp. 91 - 114
Copyright
Copyright © Australian Mathematical Society 2003

References

[1]Bunch, D., “Maximum likelihood estimation of probabilistic choice models”, SIAM J. Sci. Stat. Comput. 8 (1987) 56–70.CrossRefGoogle Scholar
[2]Byrd, R. H., “Robust trust region methods for constrained optimization”, in Third SIAM Conference on Optimization,Houston,TX,May, 1987, (SIAM, 1987).Google Scholar
[3]Dennis, J. E., Gay, D. M. and Welsch, R. E., “An adaptive nonlinear least-squares algorithm”, Trans. Math. Software 7 (1981) 348–368.CrossRefGoogle Scholar
[4]Dennis, J. E. and Schnabel, R. B., Numerical methods for unconstrained optimization and nonlinear equations (Prentice-Hall, Englewood Cliffs, NJ, 1983).Google Scholar
[5]El-Alem, M., “A global convergence theory for the Celis-Dennis and Tapia trust region algorithm for constrained optimization”, SIAM J. Numer. Anal. 28 (1991) 266–290.CrossRefGoogle Scholar
[6]Engels, J. R. and Martinez, H. J., “Local and superlinear convergence for partially known quasi-Newton methods”, SIAM J. Optim. 1 (1991) 42–56.CrossRefGoogle Scholar
[7]Gay, D. M. and Welsch, R. E., “Maximum likelihood and quasi-likelihood for nonlinear exponential family regression models”, J. Amer. Assoc. 83 (1988) 990–998.CrossRefGoogle Scholar
[8]Gonglewski, J. D., “Quasi-Newton methods for maximum-likelihood estimation”, Ph. D. Thesis, University of Houston, TX, 1986.Google Scholar
[9]Huschens, J., “Exploiting additional structure in equality constrained optimization by structured SQP secant algorithms”, J. Optim. Theory Appl. 77 (1993) 382–359.CrossRefGoogle Scholar
[10]Lalee, M., Nocedal, J. and Plantenga, T., “On the implementation of an algorithm for large-scale equality constrained optimization”, SIAM J. Optim. 8 (1998) 682–706.CrossRefGoogle Scholar
[11]Martinez, H. J., Péerez, Y. R. and Método, El, “Bfgs estructurado para el problema de maxima verosimilitud”, Reporte Tecnico, Depto. de Matmáticas, Universidad del Valle, Cali, Colombia, 1989.Google Scholar
[12]Nocedal, J. and Overton, M., “Projected Hessian updating algorithms for nonlinear constrained optimization calculation”, SIAM J. Numer. Anal. 22 (1985) 821–850.CrossRefGoogle Scholar
[13]Omojokun, E. O., “Trust region algorithms for optimization with nonlinear equality and inequality constraints”, Ph. D. Thesis, Department of Computer Science, University of Colorado, Boulder, 1989.Google Scholar
[14]Osborne, M. R., “Estimating nonlinear models by maximum likelihood for the exponential family”, SIAM J. Sci. Stat. Comput. 8 (1987) 446–456.CrossRefGoogle Scholar
[15]Osborne, M. R., “Scoring with constraints”, ANZIAM J. 42 (2000) 9–25.CrossRefGoogle Scholar
[16]Ripley, B. D., “Computer generation of random variables: A tutorial”, Internat. Statist. Review. 53 (1983) 301–319.CrossRefGoogle Scholar
[17]Schittkowski, K., “NLPQL: A FORTRAN subroutine solving constrained nonlinear programming problems”, Ann. Oper. Res. 5 (1985/1986) 485–500.CrossRefGoogle Scholar
[18]Sen, P. K. and Singer, J. M., Large sample methods in statistics (Chapman, 1993).CrossRefGoogle Scholar
[19]Tapia, R. A., “On secant updates for use in general constrained optimization”, Math. Comput. 51 (1988) 181–203.CrossRefGoogle Scholar
[20]Walker, H. and Gonglewski, J. D., “Quasi-Newton methods for maximum-likelihood estimation”, Technical report, University of Houston, TX, 1986.Google Scholar