This chapter introduces gradient descent and Newton's method, two of the most popular techniques in optimization. They are simple, effective, and versatile in a wide range of scenarios. For example, stochastic gradient descent is the backbone of machine learning training. Gradient-based approaches use the first derivative of the objective function to iteratively search for local minima where the gradient is zero. Newton's method, and its related quasi-Newton methods, use the second derivative of the objective to converge in fewer, more expensive iterations. We will develop several first-order and second-order algorithms and analyze their convergence rate, stability, computational complexity, and scaling to higher-dimensional problems. In all iterative methods, the resulting solution is only locally optimal, unless the problem is convex. We stress the importance of the initial guess for nonconvex problems, as this determines which local minima the algorithm converges to and how fast it converges. Here we mostly consider unconstrained problems, although gradient-based approaches are also useful for constrained problems.