PS A few examples: in 30 min, we can train ResNet-18 to 67% ImageNet acc on *one A100*. In 20 mins, ResNet-50 to 75.6% on a p4d AWS machine (<$5!). CIFAR costs 2 cents/model. My students tell me this is fast ;) [More seriously, we haven't seen anything in PyTorch that compares.]

And I'm sure we will *still* see "we cannot afford an ImageNet experiment" in papers for years to come. The 3200$ publication fee and 2000$ conference travel are no problemo though. twitter.com/aleks_madry/st…

But I should add the disclaimer that I have not tried their library 😅