From: rusin@vesuvius.math.niu.edu (Dave Rusin)
Subject: Re: matrix concept
Date: 22 Feb 1999 02:32:08 GMT
Newsgroups: sci.math
Keywords: What is the Singular Value Decomposition
Jian Wang wrote:
>What is the singular value of a matrix, such as [1 2;3 4] ?
I'm guessing you're asking about the _singular value decomposition_ (SVD) of
a matrix. This is one of the helpful factorizations of a matrix, that is
a presentation of a matrix A as a product M1 * M2 * M3 * ... where
each M_i is supposed to be of a certain type. (The Jordan Canonical Form
is another familiar factorization, for example).
The SVD is a presentation A = U D V' where U and V are orthogonal
and D is diagonal (of the same dimensions as A) with, usually,
non-negative decreasing diagonal terms. For a real matrix A one can show
that such U, V, and D exist; except in degenerate cases (repeated singular
values) they are unique. These terms are known as the
'singular values' of A. In the special case that A is symmetric
positive definite, U=V and the SVD is just the diagonalization of A, so
the singular values are the same as the eigenvalues. In the general case,
the terms in the SVD may be derived from the diagonalization of AA' or A'A.
For the matrix A = [1,2; 3,4] the singular values are about 5.46498, .3659662
and the matrices U, V are approximately
[-.4045535848 -.9145142957 ] [-.5760484368 .8174155605 ]
[ ], [ ]
[-.9145142957 .4045535848 ] [-.8174155605 -.5760484368 ]
The singular values may be interpreted thus: if we view A as representing
a linear transformation on some Euclidean space, then it carries the unit
ball in that space to an ellipsoid in another Euclidean space. The
singular values are the lengths of the semi-axes of that ellipsoid.
(The matrices U and V carry information about the locations of those
axes and the vectors in the first space which map into them.)
Arguably the most important applications of the SVD are those which
enable the selection of "dominant terms" in a linear map or other
matrix problems; for example this can be applied to the problem of finding
optimal low-dimensional representations of high-dimensional datasets.
For example, see
http://www.math.niu.edu/~rusin/known-math/98/multidim.scal
or several other topics on pages 15-XX and 62-XX of
http://www.math.niu.edu/~rusin/known-math/welcome.html
(Incidentally, a SVD decomposition was used to help derive good coordinates
for the "bubbles" which appear on those pages!)
dave
==============================================================================
[Remark: uniqueness of U and V is only up to signs of the columns --djr]