Machine Learning for Engineers

In the examples studied in Chapter 4, the exact optimization of the (regularized) training loss was feasible through simple numerical procedures or via closed-form analytical solutions. In practice, exact optimization is often computationally intractable, and scalable implementations must rely on approximate optimization methods that perform local, iterative updates in search of an optimized solution. This chapter provides an introduction to local optimization methods for machine learning.