next up previous [pdf]

Next: Differentiation by a complex Up: MULTIVARIATE LEAST SQUARES Previous: Inside an abstract vector

Normal equations

An important concept is that when energy is minimum, the residual is orthogonal to the fitting functions. The fitting functions are the column vectors $\bold f_1$, $\bold f_2$, and $\bold f_3$. Let us verify only that the dot product $ \bold r \cdot \bold f_2 $ vanishes; to do so, we show that those two vectors are orthogonal. Energy minimum is found by:

\begin{displaymath}
0 \quad = \quad {\partial\over \partial m_2}\ \bold r \cdot ...
...\over \partial m_2}
\quad = \quad 2\; \bold r \cdot \bold f_2
\end{displaymath} (42)

(To compute the derivative, refer to equation (23).) Equation (42) shows that the residual is orthogonal to a fitting function. The fitting functions are the column vectors in the fitting matrix.

The basic least-squares equations are often called the ``normal'' equations. The word ``normal'' means perpendicular. We can rewrite equation (39) to emphasize the perpendicularity. Bring both terms to the right, and recall the definition of the residual $\bold r$ from equation (23):

$\displaystyle \bold 0$ $\textstyle =$ $\displaystyle \bold F\T\,( \bold F \bold m - {\bf d})$ (43)
$\displaystyle \bold 0$ $\textstyle =$ $\displaystyle \bold F\T\,\bold r$ (44)

Equation (44) says that the residual vector $\bold r$ is perpendicular to each row in the $\bold F\T$ matrix. These rows are the fitting functions. Therefore, the residual, after it has been minimized, is perpendicular to all the fitting functions.


next up previous [pdf]

Next: Differentiation by a complex Up: MULTIVARIATE LEAST SQUARES Previous: Inside an abstract vector

2014-12-01