Model fitting by least squares

Unknown input: deconvolution with a known filter

For solving the unknown-input problem, we put the known filter $f_t$ in a matrix of downshifted columns $\bold F$. Our statement of wishes is now to find $x_t$ so that $\bold y \approx \bold F \bold x$. We can expect to have trouble finding unknown inputs $x_t$ when we are dealing with certain kinds of filters, such as bandpass filters. If the output is zero in a frequency band, we are never able to find the input in that band and need to prevent $x_t$ from diverging there. We prevent divergence by the statement that we wish $\bold 0\approx\epsilon\,\bold x$, where $\epsilon$ is a parameter that is small with exact size chosen by experimentation. Putting both wishes into a single, partitioned matrix equation gives:

$$\left[ \begin{array}{c} \bold 0 \\ \bold 0 \end{array... ... \begin{array}{c} \bold y \\ \bold 0 \end{array} \right]$$ (48)

To minimize the residuals $\bold r_1$ and $\bold r_2$, we can minimize the scalar $\bold r\T\, \bold r = \bold r_1\T\, \bold r_1 + \bold r_2\T\, \bold r_2$. Expanding:

$$Q(\bold x\T, \bold x) = (\bold F \bold x - \bold y)\T\,(\bold F\bold x-\bold y) + \epsilon^2 \bold x\T\,\bold x$$

$$= (\bold x\T\,\bold F\T - \bold y\T) (\bold F\bold x-\bold y) + \epsilon^2 \bold x\T\,\bold x$$ (49)

We solved this minimization in the frequency domain (beginning from equation (4)).

Formally the solution is found just as with equation (46), but this solution looks unappealing in practice because there are so many unknowns and the problem can be solved much more quickly in the Fourier domain. To motivate ourselves to solve this problem in the time domain, we need either to find an approximate solution method that is much faster, or find ourselves with an application that needs boundaries, or needs time-variable weighting functions.

Model fitting by least squares