Introduction
We have studied in the preceeding chapters various methods for fitting a polynomial to a given set of points, viz., Finite Difference methods, Lagrange's method, Divided Difference method and cubic splines. In all these methods (except Bezier/B-Splines) the polynomial passes through specified points. We say that the polynomial interpolates the given function (known or unknown) at the tabular points. In the method of Least Squares we fit a polynomial or some other function which may or may not pass through any of the data points.
Let us suppose we are given n pairs of values (xi, yi) where for some value xi of the variable x, the value of the variable y is given as yi, i = 1(1)n. In the Least Squares method, sequencing/ordering of xi's is not essential. Further, a pair of values may also repeat or there may be two or more values of y corresponding to the same value of x. If the data is derived from a single-valued function, then of course each pair will be unique. However, if an experiment is conducted several times, then the values may repeat. In the context of statistical survey, for the same value of variate x, there may be different outcomes of y; e.g., in the height versus weight study, there may be many individuals with different weights having same height and vice-versa.
Least Squares Method
Without loss of generality we shall assume in the following discussion that the values of x are not repeated and that they are exact while the corresponding values of y only, are subject to error. In the Least Squares method, we can approximate the given function (known or unknown) by a polynomial (or some other standard functions). If n data points (xi, yi), i = 1(1)n are given, then by least squares method, we can fit a polynomial of degree m, given by y = a0+a1x+a2x2 + … + amxm, m ≤ n − 1. When m = n − 1, the polynomial will coincide with the Lagrange's polynomial.