zkash @asyncakash

learning to make gpus go brrr | 🦀 | prev: @availproject, @puffer_finance, @class_lambda, topology | alum @iitroorkee Joined May 2022

Tweets

1K
Followers

413
Following

337
Likes

4K

Charles 🎉 Frye @charles_irl

4 days ago

We reverse-engineered Flash Attention 4.

20 69 693 97K 430

Download Image

Really enjoyed @samsja19’s talk on the challenges of decentralized training (e.g. DiLoCo) under low-bandwidth conditions. Was surprised to learn how much weather can destabilize training 🤯 @PrimeIntellect is doing some wild stuff with decentralized RL! 🚀 Thanks for the…

2 3 36 5K 10

Ben Burtenshaw @ben_burtenshaw

6 days ago

too much new learning material! we're releasing a few chapters of hard study on post training AI models. it covers all major aspects plus more to come. - Evaluating Large Language models on benchmarks and custom use cases - Preference Alignment with DPO - Fine tuning Vision…

13 37 347 35K 365

Download Image

Alex Zhang @a1zhang

2 weeks ago

hi! if you’re interested in using or writing mega kernels for AI (one big GPU kernel for an entire model) you should tune in to today’s @GPU_MODE livestream today in ~3 hours we have the authors of MPK talking about their awesome new compiler for mega kernels! see you there :)

5 31 244 17K 119

Download Image

JingyuanLiu @JingyuanLiu123

2 weeks ago

I was lucky to work in both China and the US LLM labs, and I've been thinking this for a while. The current values of pretraining are indeed different: US labs be like: - lots of GPUs and much larger flops run - Treating stabilities more seriously, and could not tolerate spikes…

Charuru Charuru @CharuruCha14310

2 weeks ago

2 0 38 516K 23

59 346 3K 518K 2K

zkash @asyncakash

3 weeks ago

Qwen is basically the Samsung (smartphone) of llms. They ship nice new models everything month.

Ahmad @TheAhmadOsman

3 weeks ago

Qwen is basically the Samsung (smartphone) of llms. They ship nice new models everything month.

69 143 2K 104K 402

0 0 1 123 0

zkash @asyncakash

3 weeks ago

imagine trying to “learn to code” in cursor when the tab key is basically god mode 💀

Cursor @cursor_ai

3 weeks ago

imagine trying to “learn to code” in cursor when the tab key is basically god mode 💀

128 177 3K 977K 701

Download Video

0 0 0 102 0

zkash @asyncakash

3 weeks ago

ai bros really out here teaching each other how to draw assholes 😭

0 0 0 114 0

Download Image

zkash @asyncakash

3 weeks ago

chinese ai labs slaying it 🔥

Qwen @Alibaba_Qwen

3 weeks ago

chinese ai labs slaying it 🔥

172 723 4K 905K 2K

Download Image

0 0 0 165 0

zkash @asyncakash

3 weeks ago

Just had the most amazing Transformers (with flash attention) lecture from @danielhanchen — he broke down the guts of Transformers and walked us through the full backprop step-by-step, all by hand. Huge thanks to @TheZachMueller for organizing!

3 1 13 2K 9

Elliot Arledge @elliotarledge

3 weeks ago

DO NOT buy a gpu to write kernels. use @modal notebooks. take 2 mins out of your day to learn this simple trick and kick off your work without paying a shit ton for electricity or cloud gpu run 24/7

10 24 426 42K 440

Download Video

Aditya Sharma @Adityaaa_Sharma

3 weeks ago

🚨 career update i’ve joined @bulletxyz_ to build the growth engine driving the next million on-chain traders. excited to build a @solana native trading layer that brings CEX performance fully on-chain. more ↓

134 26 471 43K 65

Download Video

Mitchell Hashimoto @mitchellh

3 weeks ago

@dhh I get asked the same about terminals all the time. “How will you turn this into a business? What’s the monetization strategy?” The monetization strategy is that my bank account has 3 commas mate.