> For the complete documentation index, see [llms.txt](https://theshank.gitbook.io/ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://theshank.gitbook.io/ai/optimization/gauss-newton.md).

# Gauss-Newton

It is an extension of Newton's method used **non-linear least square problems** where you **don't need to calculate the Hessian**.  

 The formulation is something like $$m$$functions $$r = (r\_1, ......, r\_m)$$ which are called residuals, with $$n$$ variables/parameters to be optimized $$\beta = (\beta\_1, ......., \beta\_n)$$. 

And we want to minimize the sum of sqaures

$$
S(\beta) = \sum\_i^m r\_i(\beta)^2
$$

Update equation for this comes out to be:

![](/files/4sojdXXIApUFhyXCogk4)

and most of the time we have residual $$r\_i(\beta)$$as $$r\_i(\beta) = y\_i - f(x\_i, \beta)$$ *.* 

​**Now just apply Newton's method in this and in the calculation of Hessian of the function, just ignore the 2nd order term, the hessian will basically be some function of jaobian.**