Focal Loss
Modified cross-entropy loss that up-weights hard-to-classify examples and down-scales easy examples.
Focal Loss up-weights hard examples and down-weights easy ones – solves class imbalance without resampling, developed for RetinaNet object detection.
Explanation
Introduced for object detection, Focal Loss solves the problem of dominant background classes through the focusing parameter γ.
Marketing Relevance
Focal Loss is the standard solution for extreme class imbalance in detection and classification without resampling.
Common Pitfalls
γ parameter must be tuned. Not always better than weighted cross-entropy. Can make training unstable with wrong parameters.
Origin & History
Introduced in 2017 by Lin et al. (Facebook AI Research) in the RetinaNet paper. Focal Loss first enabled one-stage detectors to compete with two-stage (Faster R-CNN).
Comparisons & Differences
Focal Loss vs. Weighted Cross-Entropy
Weighted CE weights classes globally equally; Focal Loss weights individual examples by difficulty – more adaptive and fine-grained.
Focal Loss vs. SMOTE
SMOTE creates new data points; Focal Loss modifies the loss function without changing data – can be combined.