> For the complete documentation index, see [llms.txt](https://theshank.gitbook.io/ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://theshank.gitbook.io/ai/linear-algebra/matrix-decomposition/eigen-value-decomposition.md).

# Eigen Value Decomposition

Solution of the below equation gives the eigenvalues and eigenvectors.

$$
Ax = \lambda x
$$

**Eigenvectors are basically the vectors which only scale under the transformation** $$A$$**where the scale is given by** $$\lambda$$**.** 

Let $$A$$be matrix of dimension $$n \times n$$. And let it have all the real eigenvalues and eigenvectors i.e $$n$$eigenvalues and eigenvectors. Then we will following equations. 

$$
Ax\_i = \lambda\_i x\_i  \forall i
$$

Basically we get $$n$$ equations with $$x\_i$$being eigenvectors, now we should we able to represent all the $$n$$individual equations using one matrix equation. You can see that all the equations can be combined as 

$$
A\[x\_1, x\_2, ..., x\_n] = \[\lambda\_1 x\_1, \lambda\_2 x\_2, .... \lambda\_n x\_n] \\
\= \[x\_1, x\_2, ..., x\_n] \text{diag}(\lambda\_1, \lambda\_2, ...,  \lambda\_n)
$$

where $$\[x\_1, x\_2, ..., x\_n]$$is essentially a matrix with vector $$x\_1, x\_2, ..., x\_n$$as its columns. 

Let's represent $$\[x\_1, x\_2, ..., x\_n]$$as $$V$$i.e $$V = \[x\_1, x\_2, ..., x\_n]$$. Then we can write 

$$
AV = V\text{diag}(\lambda\_1, \lambda\_2, \lambda\_n) \\
A = V\text{diag}(\lambda\_1, \lambda\_2, ...,  \lambda\_n)V^{-1}
$$

Where $$V$$is the matrix made by eigenvectors as columns of $$V$$and $${diag}(\lambda\_1,  \lambda\_2, ..., \lambda\_n)$$is the diagonal matrix with eigenvalues as diagonal elements. 

**This is eigenvalue decomposition.** 

### **Real-Symmetric Matrix**

Now if the matrix $$A$$is real-symmetric matrix, the eigenvectors are actually orthogonal. Then we represent the decomposition as 

$$
A = Q\Lambda Q^T
$$

Where $$Q$$is an orthogonal matrix formed by orthogonal eigenvectors. 

### Use of Eigenvalues

* Matrix is singular if and only if any of it's eigenvalues are zero. 
* If $$A$$is real-symmetrix then solution of $$f(x)= x^T Ax$$ subject to $$||x||\_2 = 1$$. Here is $$x$$is an eigenvector then $$f(x)$$is corresponding eigenvalue. The minimum and maximum value of $$f(x)$$is simply minimum and maximum eigenvalue. 
* If all eigenvalues are positive then the matrix is positive-definite. Similary about positive-semidefinite, etc. 
* $$A^2 = AA=X\Lambda X^{-1}X\Lambda X^{-1} = X\Lambda^2 X^{-1}$$

{% embed url="<https://youtu.be/HWnCv4iHCDc>" %}

{% embed url="<https://youtu.be/PFDu9oVAE-g>" %}