rohan @global__void
engineering && writing kernels smem Joined May 2025-
Tweets80
-
Followers16
-
Following148
-
Likes364
What a privilege to be tired from the work you prayed for. What a privilege to feel overwhelmed by growth you used to dream about. What a privilege to be challenged by a life you created on purpose. What a privilege to outgrow things you used to settle for.
triton, gluon, cutedsl, hopper, blackwell, tensorcores, layouts, composition, local_tile, partitionS, partitionD, wgmma, tcgen05, TMA, block scaling, coalesced access, ampere, ada lovelace, cutlass, cublas, cudnn, flash attention, gemm, sgemm, fp16, bf16, mxfp8, nvfp4, int4,…
half of my model runtime is spent on sampling and the other half of them time it ooms on collab 🥀
this makes so much sense wow
I really do think about videos like this a lot , there's prolly so many stories between teams like this that built great products.
I really do think about videos like this a lot , there's prolly so many stories between teams like this that built great products.
the era has just begun
CUDA 13.0 just dropped. I compressed their 26 page pdf into a thread:
Sisyphus had to simply enjoy the journey to break the eternal curse .
Sisyphus had to simply enjoy the journey to break the eternal curse .
he also has an amazing youtube channel, one of the first ever videos I watched on training was from him: youtu.be/hc0u4avAkuM
he also has an amazing youtube channel, one of the first ever videos I watched on training was from him: youtu.be/hc0u4avAkuM
This guy literally dropped the best life advice you'll ever hear
just when I think I'm beginning to understand CuTe layouts I'll see another one that shatters me
Cuda Warps make the sweet parallelism that we love possible , but what about Divergence ? In this blog - I talk about how it has been handled over the years and If you do give it a read I would love to hear what you have to say ! medium.com/@rxohan/warps-…
courses.cs.washington.edu/courses/cse599… very nice read !
been a while but we stay cooking ! ( there so much to learn ahhhhh )
huhhh
I used to think warp level execution is guaranteed to be lockstep and many sources, llms made it seem that way but this is because before volta this was the case , it no longer is guaranteed . Im enlightened but that means I have to unlearn some notions ⚰️
shoutout to the pope didn't know he was technical
Patricia @patricia23johns
312 Followers 3K Following
Vinay Mahadik @vinayxmahadik
105 Followers 296 Following co-founder/CEO: https://t.co/119zYY02Xt, https://t.co/s33aophGmx
ClosedAI @ClosedAITaken
1K Followers 4K Following
Léo @Leik0w0
652 Followers 991 Following Research intern @ snowflake | ♡ @OniAeda ♡ | Views are my own & do not reflect those of my employer
ankush @darxtrix
732 Followers 1K Following Ex-@Google tech lead • Building AI cloud @AIONxyz • Prev: @paypal, @koinexindia • Alum @IITBHU_Varanasi
Sichu Lu(Sichu.Lu218@... @lu_sichu
3K Followers 6K Following nlab fan account, arxiv surveyor, pubmed enjoyer, two culture bridger, vacuous high gossiper, dearth of any domain expertise, reluctant g theorist, gpu poor,
Sam Patterson @samxpatterson
342 Followers 1K Following Research Engineer at RBC Borealis. Prev G-Research, Getco.
Imama Shehzad @Caffeinix_alche
121 Followers 973 Following Gpu heavy thoughts at 1024 threads, melody lights mood, token-deep focus, coffee devotee.
decelerated gradient�... @ethicating
79 Followers 587 Following CS Undergrad | I like MLOps and Deep Learning | Monotheist 🤲🏼
Atharva Maskar @AtharvaMaskarML
23 Followers 294 Following Join me in my journey learning ML, DL & LLMs 🚀
Ranjith M S @ranjith_m_s
18 Followers 33 Following
Sasha Krassovsky @bztree
938 Followers 453 Following Performance @AnthropicAI. Programmer who's gotta go fast. Love playing with new hardware and compilers. Formerly databases. Opinions my own.
Vinay Mahadik @vinayxmahadik
105 Followers 296 Following co-founder/CEO: https://t.co/119zYY02Xt, https://t.co/s33aophGmx
anandmaj @Almondgodd
2K Followers 405 Following path of childhood's end | gap @penn | prev ai @tesla_optimus @dynarobotics
Ali Taha @AliesTaha
652 Followers 170 Following gpu perf @modular | ex @Tesla comp eng @uwaterloo [email protected]
AER Labs @aerlabs_
15 Followers 6 Following Decoding how AI works. Researching inference, optimizing pipelines & building open-source tools. Labs in 🇰🇷 & 🇮🇳
Henry Ko @henryHM_ko
851 Followers 361 Following performance and efficiency in ML | CS @ UC Berkeley, @BerkeleyML
maja 🔭🍒 @majamediaco
7K Followers 1K Following searching for the miraculous in the mundane. yearning, learning and earning.
Adhik Joshi @adhik_Joshi
1K Followers 756 Following $0 👉 $1m 🔜 $10m , scaling Modelslab bootstrapped way
Vikram Sekar @vikramskr
5K Followers 860 Following I write a newsletter with easy but detailed explainers on semiconductor technology for busy engineers and investors | Views mine | Sr. Staff Engineer at $QCOM
/MachineLearning @slashML
124K Followers 1 Following
lodestone-rock @LodestoneRock
711 Followers 134 Following Researching pixels @freepik. Independently created the Chroma & Radiance models as personal projects. https://t.co/d8h2mIi9zS https://t.co/RrPsPku3y6
Banghua Zhu @BanghuaZ
7K Followers 1K Following Assistant Professor @UW, Principal Research Scientist @Nvidia. Prior Cofounder @NexusflowX, @Berkeley_EECS @Google @Microsoft. I work on LLMs.
Omer Cheema @OmerCheeema
4K Followers 607 Following All about Chips. Semiconductor industry expert. ex AMD, ex Samsung. currently managing a product division at Renesas. Opinions are mine.
Léo @Leik0w0
652 Followers 991 Following Research intern @ snowflake | ♡ @OniAeda ♡ | Views are my own & do not reflect those of my employer
ankush @darxtrix
732 Followers 1K Following Ex-@Google tech lead • Building AI cloud @AIONxyz • Prev: @paypal, @koinexindia • Alum @IITBHU_Varanasi
Ayush Satyam @ayushsatyam146
374 Followers 248 Following Yet another Ayush | SWE-II @RedHat Committer to @vllm_project @theDiceDB
Ege @egeberkina
25K Followers 395 Following Art Director | Daily Image Drops | AI Explorer | Prompt & JSON & SREF share
Aakash Kumar Nain @A_K_Nain
12K Followers 909 Following ML Research | Multimodality, Self-improvement, LRMs | Keras3 collaborator, contributes to JAX ecosystem, TF-addons maintainer | @GoogleDevExpert in ML-JAX | OSS
Jacob Austin @jacobaustin132
7K Followers 920 Following Research at @GoogleDeepMind. Currently making LLMs go fast. I also play piano and climb. NYC. Opinions my own
Susan Zhang @suchenzang
34K Followers 657 Following @ Google Deepmind. Past: @MetaAI, @OpenAI, @unitygames, @losalamosnatlab, @Princeton etc. Always hungry for intelligence.
michalpitr @pitrmichal
46 Followers 94 Following MLE @ Google | From Scratch @ Substack | University of Edinburgh
Christian Gilli @nirw4nna
218 Followers 235 Following I like building software | Working on a tensor library from scratch: https://t.co/CPhWN3O8rq | Blog: https://t.co/aoUjpbPpw9
Muyu He @HeMuyu0327
1K Followers 226 Following Post-training @CollinearAI | Trying to be an expert of mixtures
Sartaj @sartaj_rajpal
1K Followers 578 Following
青龍聖者 @bdsqlsz
10K Followers 729 Following CPP:@pika_labs @SkyReels BusinessCooperation [email protected] [email protected] Architectural Model↓ [email protected] https://t.co/aGSCaF4wt4
Sichu Lu(Sichu.Lu218@... @lu_sichu
3K Followers 6K Following nlab fan account, arxiv surveyor, pubmed enjoyer, two culture bridger, vacuous high gossiper, dearth of any domain expertise, reluctant g theorist, gpu poor,
the tiny corp @__tinygrad__
59K Followers 134 Following We make tinygrad and sell tinybox, the best perf/$ AI computer. $25k for 4x 5090 in a quiet box. Our mission is to commoditize the petaflop.
Jonathan Chang @ChangJonathanC
1K Followers 887 Following ML/AI Engineer, building https://t.co/uEbfxzF7jm
Zhuohan Li @zhuohan123
9K Followers 866 Following mts @ openai | cs phd @ 🌁 uc berkeley | building @vllm_project | machine learning system | the real agi is the friends we made along the way
Emre Albayrak @emre570_
61 Followers 346 Following AI engineer, wannabe CUDA whisperer. I break things till they make sense.
Arthur Douillard @Ar_Douillard
8K Followers 2K Following Distributed Learning @ deepmind | DiLoCo, DiPaCo. Continual Learning PhD @ Sorbonne
Fern @hi_tysam
3K Followers 214 Following Neural network speedrunner and community-funded open source researcher. Set the CIFAR-10 record several times. Send me consulting/contracting work!
Tanay Mehta @serious_mehta
10K Followers 785 Following engineering agents and LLMs @aleph__alpha | kaggle grandmaster | alum @uniofbath '24 | thoughts my own
Prashanth (Manohar) V... @PMV_InferX
160 Followers 85 Following AI Infrastructure Researcher | Co-founder at @InferX, a Multi-Tenant Serverless Platform for Scalable Inference
Zach Mueller @TheZachMueller
13K Followers 604 Following Let's make billions of parameters go brr https://t.co/rUxXIfNpwh
Sam Patterson @samxpatterson
342 Followers 1K Following Research Engineer at RBC Borealis. Prev G-Research, Getco.
Santosh Mohan @theycallmeMohan
460 Followers 2K Following GPUs @Meta. Opinions stated here are my own, not those of my current employer.
HXL @9550pro
12K Followers 104 Following
Alpin @AlpinDale
5K Followers 880 Following Every age, it seems, is tainted by the greed of men. Rubbish to one such as I, devoid of all worldly wants. — I work on HPC and making AI run faster.
Peter Schmidt-Nielsen @ptrschmdtnlsn
8K Followers 748 Following Please tweet at me with CUDA performance tips.Trends for United States
You might like
