Model fitting by least squares

Sign convention

On the last day of the survey, a storm blew up, the sea got rough, and the receivers drifted further downwind. The data recorded that day had a larger than usual difference from that predicted by the final model. We could call $(\bold d-\bold F\bold m)$ the experimental error. (Here, $\bold d$ is data, $\bold m$ is model parameters, and $\bold F$ is their linear relation.)

The alternate view is that our theory was too simple. It lacked model parameters for the waves and the drifting cables. Because of this model oversimplification, we had a modeling error of the opposite polarity $(\bold F\bold m-\bold d)$.

Strong experimentalists prefer to think of the error as experimental error, something for them to work out. Likewise, a strong analyst likes to think of the error as a theoretical problem. (Weaker investigators might be inclined to take the opposite view.)

Opposite to common practice, I define the sign convention for the error (or residual) as $(\bold F\bold m-\bold d)$. Here is why. Minus signs are a source of confusion and errors. Putting the minus sign on the field data limits it to one location, while putting it in model space would spread it into as many parts as model space has parts.

Beginners often feel disappointment when the data does not fit the model very well. They see it as a defect in the data instead of an opportunity to discover what our data contains that our theory does not.

Model fitting by least squares