Skip to main content
Learning curves help understand the evolution of a model’s performance with the amount of data used to train that model. They can be used for instance to know whether more data would help the model be better or not. They also enable you to know whether the model is a good fit, underfitting, or overfitting.

​
What is a learning curve

To produce a learning curve, many models are trained using a different proportion of the data available for training. For each proportion, the model performance metric (mean square error in our case) will be calculated. In some cases, to avoid having too much variability in the results, it is possible to train multiple models for the same proportion of data and get an average (see curve on test data on the left). On the Figure shown below for instance, you can see that training a model with more data will improve the model’s performance as the mean square error (MSE) keeps going down for the test data. Point 1 shows the MSE when the model is trained on 50% of the available data, and point 2 shows the MSE when the model is trained on 100% of the available data. Learning curve with test data MSE decreasing as the proportion of training data grows from 50% to 100%

​
How to read a learning curve?

In the examples below, we trained 3 models (linear regression, polynomial regression and decision tree regression) on the same data and we plotted the learning curve for each model:
Learning curve for an underfitting linear regression with high train and test error and a small gap

Example for an underfitting model: Linear regression

  • Trend in data neither covered in train nor test set. Error for both sets very high (small gap between the two curves).

  • Bringing more data won’t help. You need to use a more complex model to capture the trends in the data.

Learning curve for a suitable polynomial regression with low train and test error converging together

Example for a suitable model: Polynomial regression

  • Trend in data captured well for both data sets. Low train set error and similar test set error (small gap between the two curves, or gap that becomes smaller).

  • Bringing more data can help if the curves are not fully converged.

Learning curve for an overfitting decision tree regression with a large gap between train and test error

Example for an overfitting model: Decision Tree Regression

  • Train set captured very well, test set very bad. Low train set error, high test set error (large gap between the two curves).

  • Use a less complex model to reduce overfitting. If you are sure you’re using the right model bringing more data also helps to reduce overfitting.

Last modified on July 31, 2026