From: jpf@hydra.cfm.brown.edu (Jim P. Ferry) Newsgroups: sci.math Subject: Re: Karhunen-Loeve transformation Date: 27 Mar 1998 20:32:14 GMT The Karhunen-Loeve procedure has many aliases: Proper Orthogonal Decomposition, Principal Component Analysis, the Singular Value Decomposition, analysis by Empirical Eigenfunctions. These different names connote different things, such as whether we are working in R^n or some Hilbert space, but the ideas are essentially the same. I think of K-L as a slimmed-down version of SVD. The SVD of a matrix A (of rank r) is H A = U D V, where U and V are unitary, and D is a matrix that is zero outside its principal r x r submatrix, this submatrix being diagonal with non- increasing, positive entries (these are the non-zero singular values of A). Call this submatrix L. The K-L decompostion is 1/2 H A = S L T, where S and T are the first r columns of U and V, respectively. K-L is traditionally defined in terms of the correlation matrices H H H H K_S = A A = S L S or K_T = A A = T L T, where the decompostions given are essentially eigensystem decompostions, except that the zero eigenspaces are left out. If you want to apply this to a Hilbert space other than R^n, you just need to change your notion of what inner-product (i.e., matrix multiplication) you're using. In my doctoral thesis I used K-L as follows. A is a matrix whose columns are snapshots of a flow field at different times. I formed the correlation matrix K_T, computed T and L, then S = A T L^(-1/2). The columns of S are the so-called "eigenpictures" (in this context). I like to think of them as the principal directions of the ellipsoid of best fit to the attractor of my flow. -Jim Ferry