# Positive (Semi)Definite Matrices
## Symmetric Matrix
When the matrix entries are the same across the diagonal.
$$
A = A^T
$$
## Positive Definite or Positive Semi-definite
It's defined for symmetric matrix $$M$$.
An $$n\times n$$ symmetric real matrix $$M$$ is said to be positive-definite if $$x^TMx > 0$$for all non-zero $$\mathbf {x}$$in $$\mathbb {R} ^{n}$$. Formally
**Positive Semi-Definite**
An $$n\times n$$ symmetric real matrix $$M$$ is said to be positive semi-definite if $$x^TMX \geq 0$$ for all non-zero $$\mathbf {x}$$in $$\mathbb {R} ^{n}$$.
**What does positive definite mean?** It's similar to saying for example in one dimensional that $$kx^2$$should be positive $$\forall x$$which means $$k>0$$. Similarly, for higher dimensions, to get the higher dimensional parabola always positive we need a positive definite matrix.
### PSD from the lens of dot-product
Let's see
We can consider $$x^TAx$$ as dot product $$(x^T)Ax$$. Now PSD means $$(x^T)(Ax) \geq 0$$ which means $$x^T$$ and $$Ax$$ form an acute angle between them i.e point in the same directoin. Hence, it means that $$A$$ is a transformation that keeps $$Ax$$ in the same direction as $$x$$.
### Properties of PD or PSD matrices
**All the eigenvalues of a positive definite matrix are positive. A matrix is positive definite if it’s symmetric and all its eigenvalues are positive.**
As established all the transformed vectors Ax are in same direction as x for positive dot product. Let's consider some eigen vector $$v\_i$$with it's eigen value $$\lambda\_i$$. Now we know that $$Av\_i = \lambda\_iv\_i$$ , so for $$(v\_i^T)(Av\_i) \geq 0 \implies v\_i^T\lambda\_uv\_i \geq 0 \implies \lambda\_i ||v\_i|| \geq 0 \implies \lambda\_i \geq 0$$
**The determinant of a positive definite matrix is always positive, so a positive definite matrix is always** [**nonsingular**](https://mathworld.wolfram.com/NonsingularMatrix.html)**.**
$$
\text{det}(A) = \prod\_{n=1}^N \lambda\_n
$$
So if all the $$\lambda\_n$$are positive, determinant will also be positive.
**Decomposition**
If $$A$$ is **Positive semi-definite if and only if it can be decomposed as follows:**
$$
A = B^TB
$$
Also
$$
A = LL^T
$$
Where $$L$$ is a lower triangular matrix. And this is called Cholesky decomposition.
## PSD Matrices defining Ellipsoid
Let's say we have $$Q(x) = x^TAx$$, be a positive definite form. Then the set
$$
{x \in \mathbf R^n: Q(x)=c}
$$
where $$c$$ is some constant. This set of points forms an ellipsoid.
### Resrouces
{% embed url="" %}
Very comprehensive study of PSD Matrices. Relating it to dot product, quadratic programming, and visualization.
{% endembed %}
{% embed url="" %}
{% embed url="" %}
#### Why does PSD matrices have to be symmetric?
{% embed url="" %}
### Full Rank Matrix
In [linear algebra](https://en.wikipedia.org/wiki/Linear_algebra), the **rank** of a [matrix](https://en.wikipedia.org/wiki/Matrix_\(mathematics\)) A is the [dimension](https://en.wikipedia.org/wiki/Dimension_\(vector_space\)) of the [vector space](https://en.wikipedia.org/wiki/Vector_space) generated (or [spanned](https://en.wikipedia.org/wiki/Linear_span)) by its columns.This corresponds to the maximal number of [linearly independent](https://en.wikipedia.org/wiki/Linearly_independent) columns of A. This, in turn, is identical to the dimension of the vector space spanned by its rows. Rank is thus a measure of the "[nondegenerateness](https://en.wikipedia.org/wiki/Degenerate_form)" of the [system of linear equations](https://en.wikipedia.org/wiki/System_of_linear_equations) and [linear transformation](https://en.wikipedia.org/wiki/Linear_transformation) encoded by A.
A matrix is said to have **full rank** if its rank equals the largest possible for a matrix of the same dimensions, which is the lesser of the number of rows and columns.
## Resources
* Talks about multiple properties of PSD and also gives insight about Hessians of functions as they are PSD
{% embed url="" %}
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://theshank.gitbook.io/ai/linear-algebra/positive-semi-definite-matrices.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.