Some interesting discussion on r/machinelearning about EfficientNet and CNN efficiency. reddit.com/r/MachineLearn… TBH, I think FLOPS as a measurement of models sometimes gets a bad rap. It has its downsides, but it's one of the harder metrics to "game".

Disagree. As soon as you throw sparsity in (and depthwise/tiny-group conv is a form of sparsity) FLOPs detach from reality. That's why sparse nets are hard (arxiv.org/abs/2006.10901), and EffNetV2 actually UNDOES a lot of depthwise. EffNetV1 == MobileNetV3 == designed for CPU. twitter.com/cHHillee/statu…

@giffmana True. Maybe someday when we have GPUs that do sparse tensor computations efficiently, then sparsity can really reduce computation