Problem: For a given class of functions and a set of data find a (the?) function of class which minimizes the sum of squares of errors.
Linear regression is the solution to this problem where is the class of linear functions.
A quadratic function is a function of the form for fixed constants , , and .
Note that taking we see that a linear function is a
special case of a quadratic function.
Find a quadratic function minimizing the sum of squares of errors for the data .
Setting these equal to zero and performing some algebra, we find , , and .
Problem: Given a set of data find a function of two variables (of some specified class) best fitting the data.
When
is taken to be a linear function, then this is
the problem of multiple linear regression.