Quantization-Aware Training (QAT)
A training method that simulates quantization errors during training so the model learns to handle lower precision – higher quality than post-training quantization.
Quantization-Aware Training simulates quantization errors during training – the model learns to handle lower precision and retains more quality than post-training quantization.
Explanation
QAT inserts "fake quantization" nodes into the compute graph: Forward pass simulates INT8/INT4 rounding, backpropagation uses Straight-Through Estimator for gradients. The model compensates for quantization errors during training.
Marketing Relevance
QAT delivers significantly better quality than post-training quantization at extreme quantization (4-bit, 2-bit). Important for edge deployment where every bit counts.
Example
Google uses QAT for on-device models: An INT4-QAT model for speech recognition on Pixel phones achieves 99% of FP32 quality at 4x less memory.
Common Pitfalls
Significantly more expensive than post-training quantization (full training needed). Not always necessary – PTQ often suffices for INT8. Hyperparameter-sensitive.
Origin & History
Jacob et al. (Google, 2018) formalized QAT for CNNs. With LLMs, QAT became relevant in 2024 through LLM-QAT and BitNet for extreme quantization (1-2 bit). Microsoft's BitNet b1.58 showed ternary weights with QAT in 2024.
Comparisons & Differences
Quantization-Aware Training (QAT) vs. Post-Training Quantization (PTQ)
PTQ quantizes after training (fast, simple); QAT simulates quantization during training (better at extreme quantization).
Quantization-Aware Training (QAT) vs. GPTQ
GPTQ is a PTQ method with calibration data; QAT trains the full model with quantization simulation.