Energy-Based Model (EBM)
Energy-based models assign energy values to data points – low energy for likely data, high for unlikely – and generate by energy minimization.
EBMs define an energy landscape over data – generation by "going downhill" to low energies, the physical concept behind score matching and diffusion.
Explanation
The model learns a scalar energy function E(x). Sampling uses MCMC or Langevin dynamics to find low-energy points. EBMs are more flexible than likelihood models but harder to train. Score matching and contrastive divergence are typical training methods.
Marketing Relevance
EBMs are the conceptual bridge between classical physics and modern generative AI – they explain why diffusion models work.
Example
An EBM for image generation learns an energy landscape: natural images have low energy, noise has high. Generation = "going downhill."
Common Pitfalls
Partition function intractable. MCMC sampling slow. Training unstable with contrastive divergence. In practice often superseded by diffusion models.
Origin & History
LeCun (2006) formalized the EBM framework. Boltzmann Machines (Hinton) and Restricted Boltzmann Machines were early EBMs. Du & Mordatch (2019) showed modern EBMs for image generation. The concept lives on in score-based generative models.
Comparisons & Differences
Energy-Based Model (EBM) vs. Diffusion Model
Diffusion models are a special type of EBM with tractable training; general EBMs need difficult MCMC sampling.
Energy-Based Model (EBM) vs. GAN
EBMs model an explicit energy function; GANs train implicitly through adversarial game without energy concept.