[MUSIC] Hi everyone, this is Professor Yongjin Yoon from KAIST, this is 4th session of the week five, of a class of Basic Mathematics for the beginning of AI, Part 1 Linear Algebra. Now we study about relationship between the determinant of the matrix, and then especially for the homogeneous linear algebraic equation. And now, we are now ready to look at two related problems, one is matrix eigen-problem, the other is a diagonalisation of a square matrix. These two problems have many applications such as, we can use this for vibration problems in complicated spring-mass system. And we can use this for finding principal strain and stresses axes, and also modeling for population growth. And also in Financial Engineering, we also use this one for the understanding Queue theory. And we also use this one for the Machine Learning Algorithm, especially deep learning and the enforcement learning too. So let me review the previous part, the homogeneous system ax equals 0, has infinitely many solutions, if and only if, determinant of a equals 0. For the homogeneous system, determinant of a is 0, then the solution has infinitely many solutions. How about determinant of a is not 0? Then this homogeneous system has a unique solution which is x equals 0. So matrix eigen value problem, maybe stated as follows, given on n by n matrix a, can you find n by 1 matrix x, such that ax equals Lambda x, where Lambda is a real or complex number. So from here, as you see, we try to simplify matrix a, to the number Lambda here. But this is a very interesting phenomena because the complicated system ax, can be expressed with a very simple system Lambda x. If we can find Lambda, then we can reduce a lot of like complications of calculation, and a lot of things you cant understand. Obviously, x equals 0, satisfied is ax equals Lambda x, for any Lambda. However, we are not interested in that this case, we are more interested in finding x which is not 0. So a is n by matrix, x is n by 1 matrix, Lambda is number, x is n by 1 matrix. So we should match each other, so we put n by n, identity matrix here, so we can make matrix like ax equals Lambda ix. And once we put the right hand side, Lambda ix, to the left side, then it becomes a minus Lambda ix equals 0. So here, this is the homogeneous system of linear algebraic equations, we can solve that. And we are interested in finding the non-0 solution for this, a minus Lambda i times x equals 0. So to have more than just one trivial solution, x equals 0, the condition is what? Determinant of a minus Lambda i equals 0. In this case, this homogeneous linear algebraic equation, a minus Lambda i times x equals 0, has an infinitely many solutions. We can find non-trivial solution x, such that ax equals Lambda x, only for Lambda which is satisfied, determinant of a, minus Lambda i equals 0. To solve this Eigen value problem, we find Lambda first, from the determinant of a minus Lambda i equals 0. This one we say characteristic equation of a, and this characteristic equation become the polynomial equation of a. And then from this determinant of a minus Lambda i equals 0, this polynomial equation. By solving this polynomial equation, the value of Lambda satisfying the characteristic equation are called, eigenvalues of a. Eigen means one, only single from this German word, so Eigen value means, it be present, the characteristic of a, with that number. So if a, is an n by n matrix, the characteristic equation is a polynomial equation of order of n, and the matrix a can then have up to the end distinct eigen values. So if Lambda is unknown, then we may solve the homogenious system which is a minus Lambda i, times x equals 0, and to find non-trivial x, for the Eigen value problem. So we differ or the solution of x, of the above homogeneous system, record the x is Eigen vector away, corresponding to the Eigen value Lambda. To write this maybe, you may have some difficulties to understand it, so again, let's study with example. So find the Eigen Values and Eigen vector over a, for the 2 by 2 matrix, 3, 2, 1, 2. For the first step, we need to formulate the characteristic equation, a minus Lambda i, which is 3 minus 3, 2, 1, 2, to minus Lambda 1, 0, 0, 1, and determinant of that equals 0. So determinant of 3 minus Lambda 2, 1, 2 minus Lambda, and then this one is ad minus bc, so 3 minus Lambda, times 2 minus Lambda, minus 3 equals 0. So that we can find Lambda equals 1 and 4, so I eigen value of matrix a, is 1 and 4 here. So to find eigen vectors, we have to solve the homogeneous system. So for the 3 minus Lambda 2, 1 and 2 minus Lambda, which is a minus Lambda i and x equals 0, for each eigen value. So we have 2, eigenvalue here, Lambda equals 1, for the Lambda equals 1, we just put down there, because Lambda equals 1 on the a minus Lambda i, so become 2, 2, 1, 1, and 0, 0. We can easy to see that, if we just let the y equals t, then x equals minus t, so the Eigen vectors for the Lambda equals 1, which eigenvalue cause 1 is minus t and t, this is again vector. For the other, eigenvalue Lambda equals 4, we just put the Lambda for 4 in the aim for Lambda i here, and we can find easily that y equals s, and y equal 2s. So eigenvector for the eigenvalue 4, is what? 2ss. So this is the eigenvalue for matrix a, 1 and 4, and corresponding eigenvector for the eigenvalue 1 and 4. Now let's find eigenvalue and eigenvector for this matrix a, 3 by 3 matrix, 1, 0, 1, 0, 1, 1, and 1, 1, 0. So first step we need to solve the characteristic equation, which is a minus Lambda i, the determinant of a minus lambda i is what? 1 minus Lambda 0, 1, 0, 1 minus Lambda 1, and 1, 1, 0, minus Lambda equals 0, so this is third order of the polynomial equation for Lambda. So there is a three different distinct Lambda here, Lambda equals 1, 2 and minus 1. So we have three different eigenvalues, so let's find the eigenvectors. So you can formulate the homogeneous linear algebraic equations, a minus Lambda i, x equals 0, like this. For the first eigenvalue Lambda 1, we can solve this homogeneous linear algebraic equation, first row say, g equals 0, right? And third row say x plus y minus g equals 0, so instead we can say, y equals t, and the x equals minus y, so x equals minus t. So in this case, the eigenvector for Lambda equals 1, is minus t, t, 0. And for the second eigenvalue, Lambda equals minus 1, we can formulate tabular form like this, and we can easily solve this with let g equals p. And from the second row, 2 y plus g equals 0, so y equals minus p over 2, and from the first row, we can find the x equals minus p over 2. So in this case, the eigenvector is what? Minus p over 2, minus p over 2p. So this is eigenvector for the eigenvalue Lambda equals minus 1, we have last eigenvalue which is Lambda equals 2. The process is same, we put the Lambda equals 2, in the a minus Lambda i, and then we can find g equals s, and from the 2nd row, y equal g, so y equals s too. From the first row, we also know that x equals s, so in this case, the eigenvector for the eigenvalue Lambda equal 2 is what? S, s, s. So we have three different eigenvector with corresponding eigenvalues. The last one, to find eigenvalues, and eigenvectors for this 4 by 4 matrix, so characteristic equation for this 4 by 4 matrix, is very easy because as you see this is lower triangular matrix. So we can find easily the determinant of this a minus Lambda i, is what? 3 minus Lambda to the 4 equals 0, so we have only 1 eigenvalue here, which is Lambda equals 3. So this matrix has only 1 eigenvalue, that is 3, so we say that eigenvalue 3 has multiple multiplicity of 4, and from the characteristic equation we may think of the matrix a, has a full eigenvalues with the same value 3. So let's find eigenvector, and with Lambda equals 3, then you become like very simple, like a linear algebraic equation, and from the first row, he keeps x plus y plus g equals 0. And from the third row, x plus y equals 0, so from here we know that g equals 0. If we let x equal t and y equals minus t, x equals t, then y equals minus t. What about w? We just put whatever number s, so in this case, the eigenvector for the Lambda equals 3 is t minus t, 0, s. So in summary, can you find x which is in a 0, such that such satisfy the ax equals Lambda x? Yes, if we can find determinant a minus Lambda i equals 0, and you can find Lambda, which is satisfied, this determinant of a minus Lambda i is 0, then we can find Lambda, which can make x, is 90. So this value of Lambda satisfy determinant a minus Lambda i equals 0, are called eigenvalue of a. And for each eigenvalue of a, we can solve the homogeneous system, and then we can find the eigenvector of a, which is corresponding to the each eigenvalue. So up to here, this is the end of the week five, and then we study about the eigenvalue problem, and then for the next week we are going to study about diagonalization problem. Thank you very much.
