Regression

Ridge Regression

This is used for preventing the overfitting in data. They simply use the regularization with the regression cost function. So new cost function for ridge regression:-

$$L = \parallel wX-y\; \parallel^2 +\; \lambda\parallel w\;\parallel^2$$

Lasso Regression

This is used for preventing the overfitting in data. They simply use the regularization with the regression cost function. The difference they have with Ridge regression is just in the way they apply regularisation. In the lasso, they use $L_1$ regularisation:

$$L = \parallel wX-y\; \parallel^2 +\; \lambda\parallel w\;\parallel$$