From: William Knight Newsgroups: sci.math.num-analysis Subject: Re: Why SQUARE in least squares method ? Date: Sun, 21 Jun 1998 11:16:31 -0700 Sam Sirlin wrote: > > pausch@merope.saaf.se (Paul Schlyter) writes: > > Gerald Gutierrez wrote: > > > I'm in search of the reason why people use squaring when they do line > > approximation for a set of points using the least squares method. Is > > there a particular reason why people don't use a power of 1, or 3, or > > 4, or 2.5 ? Is there anything optimal about using 2 ? __________________________________________________________________________ > > Mostly because it's harder to do other things, but sometimes others > are used. ......[cut].... Sam Sirlin Email: sam@kalessin.jpl.nasa.gov __________________________________________________________________________ That`s the REAL reason and it`s seldom mentioned. The "reasons" usually offered are superficial. L2 is compatible with linear algebra. This puts the extensive machinery of linear algebra at our disposal. With least squares we get theorems where otherwise we don't. SOME ASPECTS OF THIS: L2 is an inner product; the others aren't L2 is invariant under rotation of coordinate system; the others aren't. L2 decouples over orthogonal subspaces; the others don't. In probability, variances of independent random variables add; mean absolute deviations, don't. In statistics, univariate variance extends to multivariate covariance matrix; mean absolute deviation doesn't extend. PARTICULAR MANIFESTATIONS: In statistics, error estimates are simple for coefficients of lines obtained by least squares; usually not so for lines otherwise obtained. The correlation coefficient in statistics, and beyond that, partial correlation, derive from least squares fits. The whole theory of Fourier series is based on sums of squares. Harmonic analysis of statistical time series, an instance of Fourier series. Orthogonal polynomials where you can raise the degree of the polynomial without changing earlier coefficients. Other systems of orthogonal functions, Bessel, Spherical harmonics, etc. The analysis of variance table in statistics bill knight university of new brunswick canada