Vijay @tensorcore

MLIR, CUTLASS,Tensor Core arch @NVIDIA. Mechanic @hpcgarage. Exercise of any 1st amendment rights are for none other than myself. thakkarv.dev Joined July 2015

Tweets

1K
Followers

2K
Following

525
Likes

8K

Anne Ouyang @anneouyang

2 weeks ago

Excited to share what friends and I have been working on at @Standard_Kernel We've raised from General Catalyst (@generalcatalyst), Felicis (@felicis), and a group of exceptional angels. We have some great H100 BF16 kernels in pure CUDA+PTX, featuring: - Matmul 102%-105% perf…

50 90 971 182K 565

Download Image

Thinking Machines @thinkymachines

3 weeks ago

Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference” We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to…

240 1K 8K 3.2M 5K

Download Image

Vijay @tensorcore

a month ago

“TogetherAI’s chief scientist @tri_dao announced Flash Attention v4 … uses CUTLASS CuTe Python DSL” As always, thanks for being the tip of the spear and pushing us along too 💚

SemiAnalysis @SemiAnalysis_

a month ago

“TogetherAI’s chief scientist @tri_dao announced Flash Attention v4 … uses CUTLASS CuTe Python DSL” As always, thanks for being the tip of the spear and pushing us along too 💚

23 76 624 102K 225

Download Image

2 4 130 9K 35

SemiAnalysis @SemiAnalysis_

a month ago

Using CUTLASS CuTe-DSL, TogetherAI's Chief Scientist @tri_dao announced that he has written kernels that is 50% faster than NVIDIA's latest cuBLAS 13.0 library for small K reduction dim shapes on Blackwell during today's hotchip conference. His kernels beats cuBLAS by using 2…

7 23 289 59K 130

Download Image

xjdr @_xjdr

a month ago

Cute-DSL is basically perfect (for me). thank you nvidia and cutlass team. i no longer need to wait for long compile times because i underspecified a template param. i hope everyone involved gets an extra chicken nugget in their happy meal

8 6 145 11K 28

Mark Saroufim @marksaroufim

2 months ago

On Sep 6 in NYC, this won't be your typical hackathon where you do your own thing in a corner and then present at the of the day. You'll deploy real models to the market, trades will happen, chaos should be expected. The fastest model is great but time to market matters more.

5 16 99 16K 23

Download Image

SemiAnalysis @SemiAnalysis_

2 months ago

ariXv gpu kernel researcher be like: • liquid nitrogen cooling their benchmark GPU • overclock their H200 to 1000W "Custom Thermal Solution CTS" • nvidia-smi boost-slider --vboost 1 • nvidia-smi -i 0 --lock-gpu-clocks=1830,1830 • use specially binned GPUs where the number…

5 7 121 13K 32

Download Image

Vijay @tensorcore

2 months ago

Part 2: developer.nvidia.com/blog/cutlass-3… Covers the design of CUTLASS 3.x itself and how it builds a 2 layer GPU microkernel abstraction using CuTe as the foundation.

0 4 18 1K 4

Vijay @tensorcore

2 months ago

CUTLASS 4.1 is now available, which adds support for ARM systems (GB200) and block scaled MMAs

Vijay @tensorcore

5 months ago

CUTLASS 4.1 is now available, which adds support for ARM systems (GB200) and block scaled MMAs

16 86 424 76K 158

Download Image

4 12 122 7K 19

Tri Dao @tri_dao

2 months ago

Hierarchical layout is super elegant. Feels like the right abstraction for high performance GPU kernels. FlashAttention 2 actually started bc we wanted to rewrite FA1 in CuTe

Vijay @tensorcore

2 months ago

Hierarchical layout is super elegant. Feels like the right abstraction for high performance GPU kernels. FlashAttention 2 actually started bc we wanted to rewrite FA1 in CuTe

5 53 305 43K 234

1 13 157 18K 64

Michael Goin @mgoin_

2 months ago

CuTe is such an elegant library that we stopped working on our own system and wholeheartedly adopted CUTLASS for vLLM in the beginning of 2024. I can happily report that was a very wise investment! Vijay and co should be so proud of the many strong OSS projects built on top 🥳

Vijay @tensorcore

2 months ago

5 53 305 43K 234

0 4 87 5K 24

Vijay @tensorcore

2 months ago

This is what the internet was made for 🥹

typedfemale @typedfemale

2 months ago

This is what the internet was made for 🥹

85 195 2K 178K 640

Download Video

0 0 12 997 2

Ali Hassani @AliHassaniJr

3 months ago

Cosmos-Predict2 meets NATTEN. We just released variants of Cosmos-Predict2 where we replace most self attentions with neighborhood attention, bringing up to 2.6X end-to-end speedup, with minimal effect on quality! github.com/nvidia-cosmos/… (1/5)

1 8 39 17K 23

Download Video

Tri Dao @tri_dao

3 months ago

Getting mem-bound kernels to speed-of-light isn't a dark art, it's just about getting the a couple of details right. We wrote a tutorial on how to do this, with code you can directly use. Thanks to the new CuTe-DSL, we can hit speed-of-light without a single line of CUDA C++.

Wentao Guo @WentaoGuo7

3 months ago

13 74 339 79K 207

Download Image

7 57 522 48K 362

Wentao Guo @WentaoGuo7

3 months ago

🦆🚀QuACK🦆🚀: new SOL mem-bound kernel library without a single line of CUDA C++ all straight in Python thanks to CuTe-DSL. On H100 with 3TB/s, it performs 33%-50% faster than highly optimized libraries like PyTorch's torch.compile and Liger. 🤯 With @tedzadouri and @tri_dao

13 74 339 79K 207

Download Image

Vijay @tensorcore

4 months ago

Another 🔥 blog about CUTLASS from @colfaxintl, this time focusing on the gory details of block-scaled MXFP and NVFP data types and Blackwell kernels for them. research.colfax-intl.com/cutlass-tutori…

0 37 159 12K 81

Moon @MoonL88537

4 months ago

did i mention that this is totally nuts?

187 439 6K 673K 2K

Download Image

Tri Dao @tri_dao

4 months ago

We've been thinking about what the "ideal" architecture should look like in the era where inference is driving AI progress. GTA & GLA are steps in this direction: attention variants tailored for inference: high arithmetic intensity (make GPUs go brr even during decoding), easy to…