Talk:Covariance matrix

This formula looks at first sight very complicated. Actually its derivation is quite simple (for simplicity we assume &mu to be 0,(just replace everywhere X by X - &mu if you want)):
1) fix a direction in n dimensions (unit vector), let's call it u
2) project your data onto this direction (you get a number for each of your data vectors, or taking all together a set of scalar samples); you perform just a scalar product i.e.: $S(i)=(X(i),u)$
3) compute ordinary variance for this new set of scalar numbers

We are almost finished. Of course for every unit vector you get (in general) different values, so you do not have just one number like in the scalar case but a whole bunch of numbers (a continuum) parametrised by the unit vectors in n dimensions (actually only the direction counts, u and -u give the same value) Now comes the big trick. We do not have to keep this infinity of numbers, as you can see below all the information is contained in the covariance matrix (wow!)

${\rm {var}}{((u,X))}=E((u,X)^{2})=E((u,X)(u,X))=E(u^{\top }XX^{\top }u)$

Now because u is a constant we have:

$E(u^{\top }XX^{\top }u)=u^{\top }E(XX^{\top })u$

or

${\rm {var}}{((u,X)}=u^{\top }E(XX^{\top })u=u^{\top }\Sigma u$

and we are done... (easy, isn't it :)

I have moved the comments above to this discussion page for several reasons. The assertion that this very simple formula looks "very complicated" seems exceedingly silly and definitely not NPOV. Then whoever wrote it refers to "its derivation". That makes no sense. It is a definition, not an identity or any sort of proposition. What proposition that author was trying to derive is never stated. The writing is a model of unclarity. Michael Hardy 22:52 Mar 12, 2003 (UTC)