Introduction

Iterative methods for optimization are categorized into two classes. One class is called line search methods and the other class as trust region algorithms. Trust-Region methods are iterative method in which a model (mk) approximates the objective function (f) and this model is minimized in a neighborhood of the current iterate (the trust region). In case of a line-search method, the iterations are performed toward some particular directions; for example, the gradient directions are used to find the successive iterates in steepest descent [Liu and Chen, 2004]. However, in a Trust-Region algorithm, its iterates are derived by solving the corresponding optimization problem iteratively within an enclosed region. Therefore, we have more options to choose the iterates. Indeed, we can consider line-search methods as special cases of trust region methods [A. R. Conn et al., (2000)]. Trust-Region methods first introduced by M. J. D. Powel in 1970 [M. J. D. Powell, (1970)]. Powell [M. J. D. Powell, (1975)] also established the convergence result of unconstrained Trust-Region method optimization. Fletcher [R. Fletcher, (1972)] first recommended Trust-Region algorithms to solve linearly constrained optimization problems and non-smooth optimization problems [R. Fletcher, (1982)]. Trust-Region methods are very essential and effective methods in the area of nonlinear optimization. These methods are also useful for non-convex optimization problems and non-smooth optimization problems [Sun (2004)].

As most of the research works on trust region algorithms are mostly started in the 80s, trust region algorithms are less mature compare to line search algorithms, and the applications of trust region algorithms are limited as compared to line search algorithms. However, trust region methods have two major advantages. One is that they are reliable and robust; another is that they have very strong convergence properties. The key contents of any trust region algorithm are how to calculate the trust region trial step and how a decision can be made if a trial step should be accepted or not. An iteration of a trust region algorithm has the following form; a trust region is available at the beginning of the iteration. This is possible by considering an initial guess value X0 ∈ ℝn and trust region radius Δ0 > 0.