K-Fold Cross-Validation
Cross-validation variant that splits the dataset into k equal parts and trains k models.
K-Fold splits data into k parts, trains k models with rotating test set, and averages results – the gold standard for robust model evaluation.
Explanation
Each fold serves as test set once, the remaining k-1 as training. The result is the average over all k evaluations.
Marketing Relevance
K-Fold with k=5 or k=10 is the standard for model evaluation and hyperparameter tuning in ML practice.
Common Pitfalls
K too small (high variance) or too large (high compute cost). Not suitable for time series without special splits.
Origin & History
K-Fold CV was formalized in the 1970s by Stone and Geisser. k=10 became the compromise between bias and variance. Leave-one-out (k=n) is the special case.
Comparisons & Differences
K-Fold Cross-Validation vs. Hold-Out Validation
Hold-out makes a single split; K-Fold uses k different splits and is much more robust, but k times slower.
K-Fold Cross-Validation vs. Stratified K-Fold
Standard K-Fold splits randomly; Stratified K-Fold preserves class distribution in each fold – important with class imbalance.