Overfitting
When a model learns the training data too well — including its noise and random fluctuations — and performs poorly on new, unseen data. The model essentially memorizes rather than generalizes.
Why It Matters
Overfitting is one of the most common pitfalls in ML. A model that overfits is useless in production because it cannot handle real-world data it has not seen before.
Example
A model that scores 99% accuracy on training data but only 60% on test data — it memorized the training examples instead of learning general patterns.
Think of it like...
Like a student who memorizes every answer in a practice exam but fails the real exam because the questions are slightly different — they learned the answers, not the concepts.
Related Terms
Regularization
Techniques used to prevent overfitting by adding constraints or penalties to the model during training. Regularization discourages the model from becoming too complex or fitting noise in the training data.
Dropout
A regularization technique where random neurons are temporarily disabled (dropped out) during each training step. This forces the network to not rely too heavily on any single neuron and builds redundancy.
Cross-Validation
A model evaluation technique that splits data into multiple folds, trains on some folds and tests on the held-out fold, repeating so every fold serves as the test set. It provides a robust estimate of model performance.
Underfitting
When a model is too simple to capture the underlying patterns in the data, resulting in poor performance on both training and new data. The model has not learned enough from the training data.
Generalization
A model's ability to perform well on new, unseen data that was not part of its training set. Generalization is the ultimate goal of machine learning — learning patterns, not memorizing examples.