Vijay @tensorcore

MLIR, CUTLASS,Tensor Core arch @NVIDIA. Mechanic @hpcgarage. Exercise of any 1st amendment rights are for none other than myself. thakkarv.dev Joined July 2015

Tweets

1K
Followers

2K
Following

515
Likes

8K

Vijay @tensorcore

2 weeks ago

“TogetherAI’s chief scientist @tri_dao announced Flash Attention v4 … uses CUTLASS CuTe Python DSL” As always, thanks for being the tip of the spear and pushing us along too 💚

SemiAnalysis @SemiAnalysis_

2 weeks ago

“TogetherAI’s chief scientist @tri_dao announced Flash Attention v4 … uses CUTLASS CuTe Python DSL” As always, thanks for being the tip of the spear and pushing us along too 💚

24 76 626 99K 224

Download Image

2 4 130 8K 35

Using CUTLASS CuTe-DSL, TogetherAI's Chief Scientist @tri_dao announced that he has written kernels that is 50% faster than NVIDIA's latest cuBLAS 13.0 library for small K reduction dim shapes on Blackwell during today's hotchip conference. His kernels beats cuBLAS by using 2…

7 22 287 55K 126

Download Image

xjdr @_xjdr

3 weeks ago

Cute-DSL is basically perfect (for me). thank you nvidia and cutlass team. i no longer need to wait for long compile times because i underspecified a template param. i hope everyone involved gets an extra chicken nugget in their happy meal

8 6 142 10K 28

Mark Saroufim @marksaroufim

a month ago

On Sep 6 in NYC, this won't be your typical hackathon where you do your own thing in a corner and then present at the of the day. You'll deploy real models to the market, trades will happen, chaos should be expected. The fastest model is great but time to market matters more.

5 16 100 15K 23

Download Image

SemiAnalysis @SemiAnalysis_

2 months ago

ariXv gpu kernel researcher be like: • liquid nitrogen cooling their benchmark GPU • overclock their H200 to 1000W "Custom Thermal Solution CTS" • nvidia-smi boost-slider --vboost 1 • nvidia-smi -i 0 --lock-gpu-clocks=1830,1830 • use specially binned GPUs where the number…

5 7 122 13K 32

Download Image

Vijay @tensorcore

2 months ago

Part 2: developer.nvidia.com/blog/cutlass-3… Covers the design of CUTLASS 3.x itself and how it builds a 2 layer GPU microkernel abstraction using CuTe as the foundation.

0 4 18 1K 4

Vijay @tensorcore

2 months ago

CUTLASS 4.1 is now available, which adds support for ARM systems (GB200) and block scaled MMAs

Vijay @tensorcore

4 months ago

CUTLASS 4.1 is now available, which adds support for ARM systems (GB200) and block scaled MMAs

16 86 426 75K 157

Download Image

4 12 121 7K 19

Tri Dao @tri_dao

2 months ago

Hierarchical layout is super elegant. Feels like the right abstraction for high performance GPU kernels. FlashAttention 2 actually started bc we wanted to rewrite FA1 in CuTe

Vijay @tensorcore

2 months ago

Hierarchical layout is super elegant. Feels like the right abstraction for high performance GPU kernels. FlashAttention 2 actually started bc we wanted to rewrite FA1 in CuTe

5 52 303 43K 233

1 13 157 17K 64

Michael Goin @mgoin_

2 months ago

CuTe is such an elegant library that we stopped working on our own system and wholeheartedly adopted CUTLASS for vLLM in the beginning of 2024. I can happily report that was a very wise investment! Vijay and co should be so proud of the many strong OSS projects built on top 🥳

Vijay @tensorcore

2 months ago

5 52 303 43K 233

0 4 87 5K 24

Vijay @tensorcore

2 months ago

This is what the internet was made for 🥹

typedfemale @typedfemale

2 months ago

This is what the internet was made for 🥹

84 187 2K 169K 621

Download Video

0 0 12 977 2

Ali Hassani @AliHassaniJr

2 months ago

Cosmos-Predict2 meets NATTEN. We just released variants of Cosmos-Predict2 where we replace most self attentions with neighborhood attention, bringing up to 2.6X end-to-end speedup, with minimal effect on quality! github.com/nvidia-cosmos/… (1/5)

1 8 39 17K 23

Download Video

Tri Dao @tri_dao

2 months ago

Getting mem-bound kernels to speed-of-light isn't a dark art, it's just about getting the a couple of details right. We wrote a tutorial on how to do this, with code you can directly use. Thanks to the new CuTe-DSL, we can hit speed-of-light without a single line of CUDA C++.

Wentao Guo @WentaoGuo7

2 months ago

13 73 332 77K 204

Download Image

7 57 522 48K 362

Wentao Guo @WentaoGuo7

2 months ago

🦆🚀QuACK🦆🚀: new SOL mem-bound kernel library without a single line of CUDA C++ all straight in Python thanks to CuTe-DSL. On H100 with 3TB/s, it performs 33%-50% faster than highly optimized libraries like PyTorch's torch.compile and Liger. 🤯 With @tedzadouri and @tri_dao

13 73 332 77K 204

Download Image

Vijay @tensorcore

3 months ago

Another 🔥 blog about CUTLASS from @colfaxintl, this time focusing on the gory details of block-scaled MXFP and NVFP data types and Blackwell kernels for them. research.colfax-intl.com/cutlass-tutori…

0 37 158 12K 81

Moon @MoonL88537

3 months ago

did i mention that this is totally nuts?

187 443 6K 673K 2K

Download Image

Tri Dao @tri_dao

3 months ago

We've been thinking about what the "ideal" architecture should look like in the era where inference is driving AI progress. GTA & GLA are steps in this direction: attention variants tailored for inference: high arithmetic intensity (make GPUs go brr even during decoding), easy to…