Hostname: page-component-cb9f654ff-k7rjm Total loading time: 0 Render date: 2025-08-30T01:22:35.471Z Has data issue: false hasContentIssue false
  • English
  • Français

A DIRECT SEARCH QUASI-NEWTON METHOD FOR NONSMOOTH UNCONSTRAINED OPTIMIZATION

Published online by Cambridge University Press:  23 October 2017

C. J. PRICE Open the ORCID record for C. J. PRICE [Opens in a new window]
C. J. PRICE*
Affiliation:
Department of Mathematics and Statistics, University of Canterbury, Christchurch, New Zealand email C.Price@math.canterbury.ac.nz
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

A direct search quasi-Newton algorithm is presented for local minimization of Lipschitz continuous black-box functions. The method estimates the gradient via central differences using a maximal frame around each iterate. When nonsmoothness prevents progress, a global direction search is used to locate a descent direction. Almost sure convergence to Clarke stationary point(s) is shown, where convergence is independent of the accuracy of the gradient estimates. Numerical results show that the method is effective in practice.

Keywords

derivative freenonconvexClarke generalized derivative

MSC classification

Primary: 65K05: Mathematical programming methods

Information

Type
Research Article
Information
The ANZIAM Journal , Volume 59 , Issue 2 , October 2017 , pp. 215 - 231
Copyright
© 2017 Australian Mathematical Society 

References

Akbari, Z., Yousefpour, R. and Reza Peyghami, M., “A new nonsmooth trust region algorithm for locally Lipschitz unconstrained optimization problems”, J. Optim. Theory Appl. 164 (2015) 733–754; doi:10.1007/s10957-014-0534-6.Google Scholar
Appel, M. J., Labarre, R. and Radulović, D., “On accelerated random search”, SIAM J. Optim. 14 (2003) 708–731; doi:10.1137/S105262340240063X.Google Scholar
Audet, C. and Dennis, J. E. Jr., “Analysis of generalized pattern searches”, SIAM J. Optim. 13 (2003) 889–903; doi:10.1137/S1052623400378742.Google Scholar
Audet, C. and Dennis, J. E. Jr., “Mesh adaptive direct search algorithms for constrained optimization”, SIAM J. Optim. 17 (2006) 188–217; doi:10.1137/040603371.Google Scholar
Audet, C. and Dennis, J. E. Jr., “OrthoMADS: a deterministic MADS instance with orthogonal directions”, SIAM J. Optim. 20 (2009) 948–966; doi:10.1137/080716980.Google Scholar
Bagirov, A. M., Karasözen, B. and Sezer, M., “Discrete gradient method: derivative free method for nonsmooth optimization”, J. Optim. Theory Appl. 137 (2008) 317–334 ; doi:10.1007/s10957-007-9335-5.Google Scholar
Bagirov, A. M. and Ugon, J., “Piecewise partially separable functions and a derivative-free algorithm for large scale nonsmooth optimization”, J. Global Optim. 35 (2006) 163–195 ; doi:10.1007/s10898-005-3834-4.Google Scholar
Burke, J. V., Lewis, A. S. and Overton, M. L., “A robust gradient sampling algorithm for nonsmooth nonconvex optimization”, SIAM J. Optim. 15 (2005) 751–779; doi:10.1137/030601296.Google Scholar
Clarke, F. H., Optimization and nonsmooth analysis, Volume 5 of SIAM Classics in Applied Mathematics (SIAM, Philadelphia, 1990).CrossRefGoogle Scholar
Coope, I. D. and Price, C. J., “Frame based methods for unconstrained optimization”, J. Optim. Theory Appl. 107 (2000) 261–274; doi:10.1023/A:1026429319405.Google Scholar
Custódio, A. L., Dennis, J. E. Jr. and Vicente, L. N., “Using simplex gradients of nonsmooth functions in direct search methods”, IMA J. Numer. Anal. 28 (2008) 770–784 ; doi:10.1093/imanum/drn045.Google Scholar
Davis, C., “Theory of positive linear dependence”, Amer. J. Math. 76 (1954) 733–746 ;doi:10.2307/2372648.Google Scholar
Dennis, J. E. Jr., Private communication (University of Canterbury, NZ, 2004).Google Scholar
Gill, P. E., Murray, W. and Wright, M. H., Practical optimization (Academic Press, London, 1981).Google Scholar
Hooke, R. and Jeeves, T. A., “Direct search solution of numerical and statistical problems”, Assoc. Computing Machinery J. 8 (1960) 212–229; doi:10.1145/321062.321069.Google Scholar
Lukšan, L. and Vlček, J., “Test problems for nonsmooth unconstrained and linearly constrained optimization”, Technical Report 798, Prague: Institute of Computer Science, Academy of Sciences of the Czech Republic, 2000,http://www.apmath.spbu.ru/cnsa/pdf/obzor/TestProblemsforNonsmoothOptimization.pdf.Google Scholar
Makela, M., “Survey of bundle methods for nonsmooth optimization”, Optim. Methods Softw. 17 (2002) 1–29; doi:10.1080/10556780290027828.Google Scholar
Moré, J. J., Garbow, B. S. and Hillstrom, K.E., “Testing unconstrained optimization software”, ACM Trans. Math. Software 7 (1981) 17–41; doi:10.1145/355934.355936.Google Scholar
Nelder, J. A. and Mead, R., “A simplex method for function minimization”, Comput. J. 7 (1965) 308–313; doi:10.1093/comjnl/7.4.308.Google Scholar
Polak, E. and Roysett, J. O., “Algorithms for finite and semi-infinite min-max-min problems using adaptive smoothing techniques”, J. Optim. Theory Appl. 119 (2003) 421–457 ;https://link.springer.com/article/10.1023/B:JOTA.0000006684.67437.c3.Google Scholar
Price, C. J. and Coope, I. D., “Frame based ray search algorithms in unconstrained optimization”, J. Optim. Theory Appl. 116 (2003) 359–377; https://link.springer.com/article/10.1023/A:1022414105888.Google Scholar
Price, C. J., Reale, M. and Robertson, B. L., “A direct search method for smooth and nonsmooth unconstrained optimization”, ANZIAM J. 48 (2008) C927–C948; http://www.math.canterbury.ac.nz/∼m.reale/pub/priceetal08.pdf.Google Scholar
Price, C. J., Robertson, B. L. and Reale, M., “A hybrid Hooke and Jeeves – Direct method for non-smooth optimization”, Adv. Model. Optim. 11 (2009) 43–61 ;http://www.math.canterbury.ac.nz/∼m.reale/wp/hjdirect.pdf.Google Scholar
Robertson, B. L., Price, C. J. and Reale, M., “CARTopt: a random search method for nonsmooth unconstrained optimization”, Comput. Optim. Appl. 56 (2013) 291–315 ; doi:10.1007/s10589-013-9560-9.Google Scholar
Sun, L.-P., “A quasi-Newton algorithm without calculating derivatives for unconstrained optimization”, J. Comput. Math. 12 (1994) 380–386; http://www.jstor.org/stable/43692595.Google Scholar
Tang, C. M., Liu, S., Jian, J. B. and Li, J. L., “A feasible SQP-GS algorithm for nonconvex, nonsmooth constrained optimization”, Numer. Algorithms 65 (2014) 1–22 ; doi:10.1007/s11075-012-9692-5.Google Scholar
Torczon, V., “On the convergence of pattern search algorithms”, SIAM J. Optim. 7 (1997) 1–25; doi:10.1137/S1052623493250780.Google Scholar
Vicente, L. N. and Custodio, A. L., “Analysis of direct searches for discontinuous functions”, Math. Program. Ser. A 133 (2012) 299–325; doi:10.1007/s10107-010-0429-8.Google Scholar
Wright, M. H., “Direct search methods: once scorned, now respectable”, in: Numerical analysis 1995 (Dundee, 1995), Volume 344 of Pitman Res. Notes Math. Ser. (Longman, Harlow, 1996) 191–208.Google Scholar
Wu, T. and Sun, L.-P., “A new quasi-Newton pattern search method based on symmetric rank-one update for unconstrained optimization”, Comput. Math. Appl. 55 (2008) 1201–1214; doi:10.1016/j.camwa.2007.06.012.Google Scholar