Convergence
The point where a model stops improving significantly – the loss stabilizes and further epochs bring no progress.
Convergence = the loss no longer decreases significantly. Shows that the model has learned what it can – the right moment for early stopping.
Explanation
Convergence is monitored through loss curves. A converged model has reached a minimum (local or global) of the loss function.
Marketing Relevance
Convergence determines when training can be stopped. Non-convergence indicates problems like wrong learning rate or faulty data.
Common Pitfalls
Convergence ≠ good solution (local minima). False convergence due to too low learning rate. Slow training confused with good solution.
Origin & History
Convergence theory for optimization goes back to Cauchy (1847). For neural networks, Robbins & Monro (1951) proved SGD convergence under certain conditions. Modern research studies convergence rates of different optimizers.
Comparisons & Differences
Convergence vs. Early Stopping
Convergence is the natural endpoint; early stopping stops earlier based on validation loss – often the better choice.
Convergence vs. Overfitting
Training loss can converge while validation loss rises again – that is overfitting, not true convergence.