Depthwise Separable Convolution
An efficient convolution variant that decomposes a standard convolution into two steps – depthwise (per channel) and pointwise (1x1 convolution) – for 8-9x fewer computations.
Depthwise Separable Convolution decomposes standard convolutions into depthwise + pointwise steps – 8-9x more efficient, basis for MobileNet and all mobile vision models.
Explanation
Instead of a 3x3 convolution across all channels simultaneously: First 3x3 convolution per channel separately (depthwise), then 1x1 convolution to combine channels (pointwise). MobileNet used this for mobile-suitable CNNs.
Marketing Relevance
Depthwise separable convolutions are the foundation of all efficient mobile vision models (MobileNet, EfficientNet) – 8-9x less computation with minimal quality loss.
Example
MobileNetV3 uses depthwise separable convolutions for real-time image classification on smartphones – 75% ImageNet accuracy at only 5.4M parameters and 219M FLOPs.
Common Pitfalls
Depthwise phase is memory-bandwidth-bound, not compute-bound – benefits less from GPU parallelization. Channel interaction only possible through pointwise.
Origin & History
Sifre & Mallat (2013) introduced depthwise separable convolutions. Chollet used them for Xception in 2017. Howard et al. (Google, 2017) built MobileNet on them – launching efficient mobile AI.
Comparisons & Differences
Depthwise Separable Convolution vs. Standard Convolution
Standard convolution: O(K²·Cin·Cout) FLOPs; Depthwise Separable: O(K²·Cin + Cin·Cout) – factor K² savings (8-9x for 3x3).