Let's talk a little bit about the normal distribution. So the standard normal density, a random variable follows a standard normal, if it's associated population density is 1 / square root 2 pi and then e to the -z squared / 2. For z between minus infinity and plus infinity. And then the non-standard normal distribution is any random variable x that has the distribution of mu plus sigma z, where z is a standard normal and the density of that random variable works out to be phi of x- mu / sigma divided by sigma. So we can write out the non-standard normal density as a simple function of the standard normal density. And the non-standard normal density looks to be like 1 over square root 2 pi sigma squared e to the negative ( x- mu ) squared over 2 sigma squared. And again, from minus infinity less than x less than plus infinity. For the standard normal distribution, all of the odd moments are zeroes. So expected value of z, expected value of z cubed, and so on, are all 0. The normal distribution, the non-standard normal distribution is characterized by the mean and the variance. So we might write if something is non standard normal distribution just, a normal distribution. We might write that x is normal mu sigma squared. Okay, so hopefully all of this is review for you. And we can use this to build up the multivariate normal distribution. So we're going to say that a vector, z, let's say is a standard multivariate normal if its density satisfies, 2 pi to the -n/2, the multivariate density, e to the negative norm of z squared over 2. So this is just by the way equal to the product i = 1 to n of 2 pi to the minus 1/2, e to the negative zi squared over 2. Okay, so, we get the multivariate normal distribution just as the product of a collection, the multivariate standard normal distribution just is the product of a collection of IID standard normals. So that's for an n by 1 vector z. And then we might define a multivariate non-standard normal, say x. So it would not necessarily mean zero in variance/covariance matrix is I. We might define that as mu plus sigma to the 1/2 times z. Where sigma is a variance/covariance matrix, sigma to the 1/2 matrix, that decomposition is called the Cholesky decomposition. So notice if we write x this way, the expected value of x is equal to mu, because the expected value of z is 0, and the variance of x is equal to sigma to the 1/2 variance of z, variance of z, sigma to the 1/2 transpose. Which variance of z is I, so we just use this fact and it equals sigma. Okay, so using that we can define this non-standard normal, multivariate normal distribution. Which people would then just call then the normal distribution or the multivariate normal distribution maybe. And we would write that x is normal, mu, sigma. Now, if the density is associated with it, we could use the transformation to figure out the density associated with it. And it is 2 pi, to the -n/2, determinant of sigma to the -1/2, e to negative x minus mu transpose sigma inverse x minus mu divided by 2. Okay, so that's what the non-standard normal, what we would just call the normal distribution works out to be. The normal distribution, the multivariate normal distribution has many convenient properties. First of all, any subvector of x, any subvector of x also follows a multivariate normal distribution with of course the relevant mean and variant. So for example if I take x and break it into two column vectors x1 and x2 and my mu has the same corresponding column vectors, mu1, mu2 and my sigma is now a matrix with, That I've decomposed with the same dimensions, then for example x1 is going to be now multivariate normal with mean mu1 and variance sigma1. And similarly, x2 is going to be normal with mean mu and variance of sigma22. Should have been sigma11. Okay, so any subvector of a multivariate normal vector is multivariate normal. The second thing that is true about the multivariate normal is any full rank linear combination of multivariate normals is multivariate normal. So let's suppose I take Ax + b. Then we know what the expected value of that is. The expected value of Ax + b has to be equal to A mu + b. And we know what the variance of Ax + b has to be, That has to be A variance of x which is sigma, A transpose. However, we also know when x is multivariate normal that then the variable Ax + b, let's call that w. Then we know that w has to be normally distributed also with mean A mu + b and variance A sigma A transpose. So the normal distribution has this property that all linear, the multivariate normal distribution has this property that all full ranked linear combinations of multivariate normals are also multivariate normal. In addition, all conditional distributions, so if I take, for example, x1 given x2 where x1 and x2 are defined as above, that will also be normally distributed. So all linear combinations, all marginals, and all conditional distributions of the multivariate normal distribution are also multivariate normal. Another property of the multivariate normal is that absence of covariance implies independence. So take for example my x1 and x2 up here and I look at its covariance matrix, I have it written out right here. It's sigma12, which is I wrote sigma21 there, but that's just sigma12 transpose because the matrix has to be symmetric. If that's 0, so in other words if the block off diagonal of the sigma matrix, sigma matrix is 0 then that would imply that x1 is not just uncorrelated with x2, but it's actually independent of x2, okay? So the normal distribution is one of the strange distributions where this is true. Where not only does independence imply absence of covariance which is true for all distributions, but the reverse direction also applies. Absence of covariance implies independence. So in many ways, the multivariate normal distribution is probably the most convenient distribution to work with.
