Overfitting of the machine learning model is a problem in which the model learns all the noises from the training data and performs well in training data but while coming to validating it doesn't work well, that is how we check if a model is overfitted or not. This may happen due to the high complexity level of the ML model to learn all the parameters and during training, it learns noisy data also. Because of this model is unable to recognize the appropriate pattern in the dataset.
 |
| Overfitting |
Methods to avoid overfitting in ML
Let us talk about various ways to prevent our machine-learning model from getting overfitted.
(1) Early Stopping
This method seeks to stop the training before the model starts learning the noise within the model but this lead to other problems such as underfitting also.
(3) Data Augmentation
While it is better to introduce clean relevant data in your training data sometimes noisy data is added to make the model more stable.
(2) Train with more data
Expanding the training set to include more data can increase the accuracy of our model.
(4) Feature Selection
Allowing only important features and rejecting irrelevant and redundant features.
(5) Regularization
In regularization, we limit the amount of variance in the model. we apply a penalty to input parameters. It reduces the overfitting nature of the Machine learning model even if the model works well and ensures the problems do not occur in the future. In regularization, coefficients shrink.
Type of Regularisation
- Ridge
- Lasso
- Elastic Net
- Gradient Descent
(6) Ensemble Learning
Ensemble learning methods are made up of a set of machine learning models. e.g. decision tree. The most common ensemble method is bagging and boosting.
Cases where Overfitting is not a problem
Here is a few examples where overfitting is not a problem:
- when the training data is large enough for the model to effectively learn the underlying patterns.
- when the model itself is simple and therefore not prone to learning random fluctuations in the data.
- when the model is only used for exploratory data analysis rather than prediction. While it is still important to consider overfitting and take steps to prevent it, in some situations it may not be a major issue.
