Levenberg–Marquardt

Levenberg-Marquardt is also an optimization used for non-linear least square problems.

The LMA interpolates between the Gauss–Newton algorithm (GNA) and the method of gradient descent. LMA can also be viewed as Gauss–Newton using a trust region approach.

Adaptive Lambda λ

The (non-negative) damping factor is adjusted at each iteration. If reduction of cost is rapid, a smaller value can be used, bringing the algorithm closer to the Gauss–Newton algorithm, whereas if an iteration gives insufficient reduction in the residual, can be increased, giving a step closer to the gradient-descent direction.