Disagree. As soon as you throw sparsity in (and depthwise/tiny-group conv is a form of sparsity) FLOPs detach from reality. That's why sparse nets are hard (arxiv.org/abs/2006.10901), and EffNetV2 actually UNDOES a lot of depthwise. EffNetV1 == MobileNetV3 == designed for CPU.
Disagree. As soon as you throw sparsity in (and depthwise/tiny-group conv is a form of sparsity) FLOPs detach from reality. That's why sparse nets are hard (arxiv.org/abs/2006.10901), and EffNetV2 actually UNDOES a lot of depthwise. EffNetV1 == MobileNetV3 == designed for CPU.
@giffmana True. Maybe someday when we have GPUs that do sparse tensor computations efficiently, then sparsity can really reduce computation