Introducing FlashConv, a new technique for training state space models. Runs up to 35X faster than FlashAttention and runs the new H3 language model 2.4X faster than Transformers! Research by @tri_dao and our own @realDanFu. together.xyz/blog/h3
2
30
149
20K
39
Thanks to everyone who contributed to this work: @KhaledSaab11, @ai_with_brains, Atri Rudra, and @HazyResearch! Thanks to @StanfordAILab, @StanfordHAI, @StanfordCRFM for helping us train some of these models.