From: jpf@hydra.cfm.brown.edu (Jim P. Ferry)
Newsgroups: sci.math
Subject: Re: Karhunen-Loeve transformation
Date: 27 Mar 1998 20:32:14 GMT
The Karhunen-Loeve procedure has many aliases: Proper Orthogonal
Decomposition, Principal Component Analysis, the Singular Value
Decomposition, analysis by Empirical Eigenfunctions. These different
names connote different things, such as whether we are working in R^n
or some Hilbert space, but the ideas are essentially the same.
I think of K-L as a slimmed-down version of SVD. The SVD of a matrix A
(of rank r) is
H
A = U D V,
where U and V are unitary, and D is a matrix that is zero outside its
principal r x r submatrix, this submatrix being diagonal with non-
increasing, positive entries (these are the non-zero singular values
of A). Call this submatrix L.
The K-L decompostion is
1/2 H
A = S L T,
where S and T are the first r columns of U and V, respectively.
K-L is traditionally defined in terms of the correlation matrices
H H H H
K_S = A A = S L S or K_T = A A = T L T,
where the decompostions given are essentially eigensystem decompostions,
except that the zero eigenspaces are left out.
If you want to apply this to a Hilbert space other than R^n, you just
need to change your notion of what inner-product (i.e., matrix
multiplication) you're using.
In my doctoral thesis I used K-L as follows. A is a matrix whose
columns are snapshots of a flow field at different times. I formed the
correlation matrix K_T, computed T and L, then S = A T L^(-1/2). The
columns of S are the so-called "eigenpictures" (in this context). I like
to think of them as the principal directions of the ellipsoid of best fit
to the attractor of my flow.
-Jim Ferry