Random Search
Hyperparameter tuning by randomly sampling from the parameter space – more efficient than grid search with the same compute budget.
Random search picks hyperparameters randomly instead of systematically – almost always better than grid search with the same budget because unimportant parameters waste less budget.
Explanation
Random search tries random combinations, covering more of the search space, especially when parameters have different importance.
Marketing Relevance
Random search is the recommended starting point for hyperparameter tuning: simple, parallelizable, and surprisingly effective.
Common Pitfalls
No guarantee of finding the optimum. With very small budget, Bayesian optimization may be better. Reproducibility requires seed management.
Origin & History
Bergstra & Bengio (2012) proved mathematically and empirically that random search outperforms grid search. The paper "Random Search for Hyper-Parameter Optimization" became one of the most influential ML papers.
Comparisons & Differences
Random Search vs. Grid Search
Grid search wastes budget on unimportant parameter dimensions; random search distributes budget evenly across the entire search space.
Random Search vs. Bayesian Optimization
Random search is uninformed; Bayesian optimization learns from past runs – better with small budget but more complex.