Free tools Windows power users keep installed
One-click scans. No signup required.
Underfitting means a model is too limited to learn the useful pattern; overfitting means it has learned the training examples too closely, including quirks that do not carry over. Compare performance on training data with performance on validation data to tell them apart: weak results on both suggest underfitting, while strong training results paired with materially weaker validation results suggest overfitting.
What overfitting and underfitting mean
Machine-learning models learn patterns from examples. An underfit model is too constrained or simple to capture the useful signal, so it performs poorly on the data it learned from and generally on new data as well. An overfit model tracks its training examples so closely that it can learn accidental noise or peculiarities; it may score well on those examples but perform worse on unseen ones.
As an Amazon Associate I earn from qualifying purchases.
The student analogy is useful as a shorthand: an overfitting model “memorizes,” while an underfitting model “doesn’t study enough.” It describes the difference in outcomes, not a literal account of how a model learns.
Bias and variance offer another way to understand the trade-off. A model that is too simple can have high bias, missing real structure. A highly flexible model can have high variance, responding strongly to which particular examples it saw. The aim is not to maximize training performance, but to capture patterns that generalize. Noise and evaluation design also affect generalization, so bias and variance do not explain every failure.
#1 Best Overall
How to diagnose the problem from scores
Choose a metric that fits the task, then compare training performance with validation performance using a suitable split. Scikit-learn’s underfitting and overfitting example summarizes the basic pattern: low training and validation scores indicate underfitting; a high training score and low validation score indicate overfitting.
| Training score | Validation score | Likely interpretation |
|---|---|---|
| Low | Low | Underfitting is a possibility: the model may be too limited, or its constraints too strong. |
| High | Materially lower | Overfitting is a possibility: the model may be fitting training-specific noise or quirks. |
| High | High and reasonably close | The model appears to generalize well on this validation design; confirm that the data and metric reflect intended use. |
These patterns are clues, not a diagnosis by themselves. Before changing the model, check whether the metric matches the task, the data are sound, the split is appropriate, and information from validation examples has not leaked into training. A validation set that does not resemble the situations the model will face in use can give a misleadingly reassuring—or discouraging—score.
Rank #2
- Use scikit-learn to track an example ML project end to end
- Explore several models, including support vector machines, decision trees, random forests, and ensemble methods
- Exploit unsupervised learning techniques such as dimensionality reduction, clustering, and anomaly detection
- Dive into neural net architectures, including convolutional nets, recurrent nets, generative adversarial networks, autoencoders, diffusion models, and transformers
- Use TensorFlow and Keras to build and train neural nets for computer vision, natural language processing, generative models, and deep reinforcement learning
Use curves to distinguish model limits from data limits
Validation curves: test one hyperparameter
A validation curve plots training and validation scores across values of one hyperparameter. It can show whether the model is too constrained at one end of the range or too flexible at the other. For regularized models, stronger regularization constrains coefficients and may reduce overfitting; too much can suppress useful signal and lead to underfitting. Scikit-learn explains this trade-off in its regularization example.
PC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchLearning curves: test the effect of more examples
A learning curve plots training and validation performance as the training-set size changes. If validation performance improves as more representative examples are added and the gap narrows, additional data may help. If both scores settle at a poor level, more data alone may not solve the problem: the model may lack the capacity to represent the signal, or another issue may be limiting performance. Scikit-learn describes both curve types in its learning-curve and validation-curve guide.
Rank #3
What to change—and how to test it
Treat each adjustment as an experiment rather than a guaranteed fix. Keep the validation design and metric consistent, and change one major factor at a time so you can see what affected the result.
If the model appears to overfit
- Try a less flexible model or strengthen regularization.
- Use a validation curve to explore a range of regularization settings rather than assuming that more is always better.
- If feasible, add representative training data. More examples can reduce sensitivity to the particular training sample, but they help only if the model can represent the underlying relationship.
If the model appears to underfit
- Try a more expressive model.
- Relax regularization if it is constraining the model too strongly.
- Use validation performance to check that added flexibility improves generalization, not just the training score.
Scikit-learn’s polynomial example illustrates how a simple model can miss a pattern while a more flexible one can become sensitive to training samples. The right level of complexity depends on the data and task, not on a universal score threshold.
Rank #4
Evaluate models without contaminating the test set
- Fit on training data. Use it to estimate the model’s parameters.
- Compare choices with validation data or cross-validation. Use a consistent, task-appropriate metric and split strategy to assess candidate models and settings.
- Reserve a final test set. Evaluate on it after choices are made, as a check on performance beyond the data used for fitting and selection.
If you repeatedly adjust a model in response to its test score, your choices begin adapting to that set; it is no longer an untouched estimate of generalization. Likewise, once you have optimized hyperparameters against a validation score, that score is no longer an untouched estimate. Scikit-learn’s cross-validation guide covers the role of evaluation splits.
Recommended Free Tools
Choose splits that preserve the structure relevant to the model’s intended use. For time-dependent or grouped observations, a random split may put related examples on both sides and make evaluation easier than real deployment. The appropriate splitting strategy depends on the data; no single splitter is right for every structured dataset.
Best Value
Compare candidates on more than one number
Use the same metric and split strategy to compare candidate models or settings. Consider the full pattern rather than selecting whichever model has the highest training score.
- Training score and validation or cross-validation score.
- The size of the train–validation gap.
- How much performance varies across folds or repeated splits.
- How validation performance changes as training-set size grows.
- The model’s complexity and the strength of its regularization.
A small gap is not automatically proof of success: both scores can be poor, or the validation design can fail to represent future use. The useful choice is the one that performs well on appropriately held-out data, not the one that best remembers its training examples.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problems
