Grid Search
Hyperparameter tuning method that systematically tries all combinations of a predefined parameter space.
Grid search systematically tries all hyperparameter combinations – simple to implement but exponentially expensive and usually less efficient than random search.
Explanation
Grid search tests every combination (e.g., LR=[0.001, 0.01, 0.1] × Batch=[32, 64] = 6 runs). Simple but exponentially expensive.
Marketing Relevance
Grid search is the entry point for hyperparameter tuning. Sufficient for small search spaces, for many parameters use random search or Bayesian optimization.
Common Pitfalls
Exponentially growing cost. Wastes budget on unimportant parameters. Often less efficient than random search.
Origin & History
Grid search was standard in ML for decades. Bergstra & Bengio (2012) showed that random search usually delivers better results with the same budget, ending grid search's dominance.
Comparisons & Differences
Grid Search vs. Random Search
Grid search tests all combinations systematically; random search picks random points – more efficient because it covers more of the search space.
Grid Search vs. Bayesian Optimization
Grid search is uninformed (blindly tries all points); Bayesian optimization uses past results for smarter point selection.