Benchmarking CI across cloud providers. Got some counterintuitive result on runcloud
1 × 16 vCPU box → 60s, $0.00227
20 × 1 vCPU sandboxes → 44s, $0.00209
25% more vCPU. 26% faster. 8% cheaper. Why?
My theory: at 1:06 of the 1:20 run, 477 of 478 test modules were done and load average was 3.93 on 16 cores, meaning you're paying for 12 idle cores to wait on one straggler, while sharding lets each core stop billing the moment it finishes.
Does that hold up, or am I missing something? Curious whether the other providers show this same patter and what other quirks are hiding in there.
“Competing on price is the dumbest strategy” is one of those guru lines that sounds smart until you look at reality.
China built the world’s manufacturing powerhouse by competing hard on price. Same playbook created massive winners across industries.
In cloud infrastructure, prices for agent sandboxes and microVMs can be marked up by as much as 13x. His company and Modal are clear examples of providers charging far more than the costs of the service.
Price competition is how real scale is built. Dismissing it just hands the market to someone hungrier.
Ignore the “never compete on price” gurus, thats how you lose the long game.
Competing on price is the dumbest strategy.
You're not buying a Porsche by dollar-per-horsepower.
Three Volkswagen Beetles might have more aggregate horsepower than a Porsche for the same money. But nobody wants three Beetles.
Hey @dbworku, really solid work on the hpc-sandbox-benchmarks!
Im working on a CI-shredding demo, taking the full CPython test suite and sharding it across hundreds of sandboxes to show how a long sequential CI run can be collapsed into something much faster. It mainly looks at total runtime, cost, and how evenly the tests distribute.
Im also planning to push the vCPU count as high as the providers allow to see how well they scale under heavy parallel load.
Any advice on config details or tips for the high parallelism tests would be appreciated!😁
This is fluid dynamics, the same math used to design wings and predict weather
267 billion calculations
16 minutes on one machine
$0.0729
this used to need a supercomputer and a waiting list
now it's seven cents in the cloud run.cloud
6K Followers 1K Followingbuilding box, the best full VM sandbox @asciidotdev w/ @luaroncrew
prev: HPC for black holes hunting @esa, gamedev, NN in rust, CAD AI
218K Followers 0 FollowingOG of: P and T in ChatGPT, 100x deeper learning, meta learning and RSI, neural distillation, GAN/World Model... Co-authored most-cited AI paper of 20th century
10.4M Followers 107 FollowingThe first word in business news | Watch Live: https://t.co/nHEpHOAfg3 | Newsletters: https://t.co/nWaCxHTiks | Podcasts: https://t.co/096e9xMJF7
2K Followers 365 FollowingAsst Professor at @JohnsHopkins (AMS, CS, DSAI). Previously: @SimonsInstitute, @oxfordstats, @Polytechnique. I like to scale up things!
1.9M Followers 1K FollowingCo-Founder of Coursera; Stanford CS adjunct faculty. Former head of Baidu AI Group/Google Brain. #ai #machinelearning, #deeplearning #MOOCs
508K Followers 1K FollowingML/AI research engineer. Ex stats professor.
Author of "Build a Large Language Model From Scratch" (https://t.co/O8LAAMRzzW) & reasoning (https://t.co/5TueQKx2Fk)
88K Followers 1K FollowingCo-founder and CTO at @huggingface 🤗. ML/AI for everyone, building products to propel communities fwd. @Stanford + @Polytechnique
48K Followers 915 FollowingCreator of bitsandbytes. Professor @CarnegieMellon and Research Scientist @allen_ai . I blog about deep learning and PhD life at https://t.co/Y78KDJJFE7.
2K Followers 416 FollowingStatistical consultant, research analyst and statistics lecturer | #rstats dev @easystats4u
Also on 🦋 @mattansb.bsky.social
{He/Him} 🇮🇱
586K Followers 3K FollowingNVIDIA Director of Robotics & Distinguished Scientist. Co-Lead of GEAR lab. Solving Physical AGI, one motor at a time. Stanford Ph.D. OpenAI's 1st intern.
44K Followers 266 FollowingProfessor of Machine Learning, University of Oxford
@OATML_Oxford Group Leader
Expert Advisor to AISI
"One of the top machine-learning people" - Tim Berners-Lee
7K Followers 975 Followingmember of technical staff @openai, cofounder @aidkit_tech, former head of sync @dropbox, cofounded bubbli (acquired by dropbox), previously made yelp monocle.