Yu Zhang 🐈🐙 @yzhang_cs
@Kimi_Moonshot; PhD Student @ Soochow University; working on efficient methods for LLMs; disciple of parallel programming; INTP yzhang.site Joined February 2023-
Tweets676
-
Followers615
-
Following661
-
Likes4K
Big win for @Lei_Wang_1999 on Tilelang endorsement from the whale !
Big win for @Lei_Wang_1999 on Tilelang endorsement from the whale !
Deepseek is using TileLang instead of Triton. TileLang is a rlly elegant language! Also reminds me of this surface-level blog I wrote when first learning about it. It only takes less than 100 lines of code to achieve 630 TFLOPS for softmax attn fwd in TileLang (1.3x of FA2)
Deepseek is using TileLang instead of Triton. TileLang is a rlly elegant language! Also reminds me of this surface-level blog I wrote when first learning about it. It only takes less than 100 lines of code to achieve 630 TFLOPS for softmax attn fwd in TileLang (1.3x of FA2) https://t.co/hCLu73npSW
man how is Kimi so different a model from a parallel universe
Some h800 performance results of the Attention Sink in tilelang. Take a look if you want to make some variants :) github.com/tile-ai/tilela…
Sharing our second Connectionism research post on Modular Manifolds, a mathematical approach to refining training at each layer of the neural network
Sharing our second Connectionism research post on Modular Manifolds, a mathematical approach to refining training at each layer of the neural network
(1/6) triton kernels are a great way to understand ML models. but tutorials are scattered the learning method for me was jst to read real, high performance code so i wrote a blog which walkthroughs the design and intuitions behind FLA's softmax attention kernel 🧵also a thread
Check out @nathancgy4's awesome Deltaformer PR and stay tuned for a post on the architecture soon!
Check out @nathancgy4's awesome Deltaformer PR and stay tuned for a post on the architecture soon!
@techeconomyana I really don't want this guy winning..
SEED's paper on associative memory and DeltaFormer is still one of my favorites 🎉so I'm happy share that DeltaFormer is now supported on FLA (flash linear attention)! Learned incredibly much from @yzhang_cs and Mingyu
SEED's paper on associative memory and DeltaFormer is still one of my favorites 🎉so I'm happy share that DeltaFormer is now supported on FLA (flash linear attention)! Learned incredibly much from @yzhang_cs and Mingyu https://t.co/1UU5U9uIBx
Still one of the most insane lines I've read in any ML paper.
swiglu-style gates working so well for attention (and not just in the ffn layers) is a beautiful thing. as it turns out, the "divine benevolence" might just be caused by better inductive biases for controlling where information goes.
Big day for AI agents! Tongyi Lab (@Ali_TongyiLab) just dropped half a dozen new papers, most focused on Deep Research agents. I’ll walk you through the highlights in this thread. (1/N)
Our engineer wrote about the thinking and technical story behind Checkpoint Engine. 👉 moonshotai.github.io/checkpoint-eng…
Our engineer wrote about the thinking and technical story behind Checkpoint Engine. 👉 moonshotai.github.io/checkpoint-eng…
#COLM2025 We introduce Adaptive Computation Pruning (ACP) for the Forgetting Transformer (FoX) — a provably safe pruning method that significantly speeds up our Forgetting Attention kernel, especially for long-context pretraining. Our simple Triton kernel with ACP is 1.7x to 2.4x…
I was lucky to work in both China and the US LLM labs, and I've been thinking this for a while. The current values of pretraining are indeed different: US labs be like: - lots of GPUs and much larger flops run - Treating stabilities more seriously, and could not tolerate spikes…
I was lucky to work in both China and the US LLM labs, and I've been thinking this for a while. The current values of pretraining are indeed different: US labs be like: - lots of GPUs and much larger flops run - Treating stabilities more seriously, and could not tolerate spikes…
Congrats to @SonglinYang4 that DeltaNet series finally scaled up! Also Glad to see Qwen Team uses 3:1 GatedDeltaNet:Attention hybrid ratio as our Hybrid Linear Attention analysis arxiv.org/abs/2507.06457 recommended😊
Congrats to @SonglinYang4 that DeltaNet series finally scaled up! Also Glad to see Qwen Team uses 3:1 GatedDeltaNet:Attention hybrid ratio as our Hybrid Linear Attention analysis arxiv.org/abs/2507.06457 recommended😊
Video and blog post of @SonglinYang4 explaining DeltaNet used in the newest Qwen-Next model * sustcsonglin.github.io/blog/2024/delt… * youtu.be/d0HJvGSWw8A
Apologies that I haven't written anything since joining Thinking Machines but I hope this blog post on a topic very near and dear to my heart (reproducible floating point numerics in LLM inference) will make up for it!
Apologies that I haven't written anything since joining Thinking Machines but I hope this blog post on a topic very near and dear to my heart (reproducible floating point numerics in LLM inference) will make up for it!
hybrid is the future:)
hybrid is the future:)
Harold Benoit @harold_matmul
478 Followers 282 Following Another day of being a researcher in theory but an engineer in practice | tech staff @LiquidAI_
S @SuiCide47840548
13 Followers 554 Following
roanoke_gal @roanoke_gal
397 Followers 1K Following
Yoram Bachrach @yorambac
3K Followers 7K Following Research Scientist at Meta (prev Google DeepMind and Microsoft Research). Working on LLM Agents and Multi-Agent Systems.
Ofir Press @OfirPress
15K Followers 7K Following I build tough benchmarks for LMs and then I get the LMs to solve them. SWE-bench & SWE-agent. Postdoc @Princeton. PhD @nlpnoah @UW.
John Huang @lyapunov_2000
67 Followers 861 Following
Thomas Joshi @thomastjoshi
1K Followers 6K Following Coauthor of DSPy @stanford (most popular Stanford AI library) - AI and EE degree @columbia
Vartan Polodyan @VarPol
194 Followers 6K Following
Younes Bouhadjar @YounesBouhadjar
225 Followers 773 Following Postdoctoral researcher in Computational Neuroscience and Neuromorphic Engineering
Nostalgic southerner @ListedonSale
54 Followers 2K Following
Thomas Sohmers @trsohmers
2K Followers 1K Following Building new AI hardware at @Positron_AI. 2013 Thiel Fellow, hardware hacker, entrepreneur. Previously founded @REXComputing | https://t.co/vqJ6oJMqWG
𝙶𝚒𝚘𝚝𝚝�... @wGiotto
33 Followers 345 Following 🎬 Branding Video Designer @Kimi_Moonshot｜Ex @Gate_io @ADVANCEAI_
Deku @KenshinAribabui
27 Followers 434 Following
Alexander Pondaven @alexpondaven
82 Followers 511 Following Working on controllable video generation. PhD student @UniofOxford @aims_oxford @OxfordTVG @Snap. MEng @Imperialcollege
Neil Band @neilbband
1K Followers 702 Following PhD student @StanfordAILab @StanfordNLP @Stanford advised by Tatsunori Hashimoto and Tengyu Ma. Prev: @OATML_Oxford @CompSciOxford
Afroz Mohiuddin @afrozenator
1K Followers 5K Following @OpenAI, ex @Google, @AIAtMeta. Interested in Science, Psychology, Investing and generally everything. Good Thoughts, Good Words, Good Deeds.
nwyin @_nwyin
508 Followers 580 Following
Peter Lesslie @pclesslie
41 Followers 716 Following
Quanquan Gu @QuanquanGu
16K Followers 2K Following Professor @UCLA, Pretraining and Scaling at ByteDance Seed | Recent work: Build AGI | Opinions are my own
wilam yang @wilambatch
88 Followers 2K Following
Ayush Chakravarthy @achakravarthy01
148 Followers 1K Following convincing plots to go to the up and right @StanfordAILab
bezdarnost @bezdarnost_brr
4 Followers 163 Following Opinions are from the voices in my head(they achieve consensus faster than I do).
Jiminator @Jiminator31
49 Followers 1K Following MSCS Student at UIUC | Former MLE Intern @Waymo | Aspiring Researcher / MLE | Esports Fan
Kacper @_kacper____
57 Followers 568 Following
unatirs @GhostOfIrodov
69 Followers 456 Following Building GhostOfИродов. 🧘 Maths, Algorithms, Theoretical Physics, Black coffee ❤️🔥
Jeroen Van Goey @BioGeek
914 Followers 6K Following Staff Research Engineer in BioAI at @InstaDeepAI (part of @BioNTech_Group) ML for de novo peptide sequencing. https://t.co/KOjeWuazsk
Yacine Mahdid @yacinelearning
13K Followers 848 Following (neuro/ai) I make technical deep learning tutorials 👺
Gil Dias @gil_cdias
123 Followers 3K Following
Edouardo Honig @e__honig
165 Followers 2K Following
aastha @being__aastha
45 Followers 1K Following (learning) machine learning @ amazon. do #NLProc & #AI!
michael @MichaelDmrks
87 Followers 215 Following AI, maths, biology || creative director of biotech || repost ≠ support || ex HS, @ cambridge
Augustin Marks de Cha... @augmdc
8 Followers 155 Following Ph.D Student | AI & Automation in Underground Mining | Based out of Sudbury
Itz Vrtx @ItzVrtx
4 Followers 69 Following
Yacine Mahdid @yacinelearning
13K Followers 848 Following (neuro/ai) I make technical deep learning tutorials 👺
Conference on Languag... @COLM_conf
5K Followers 6 Following https://t.co/GhGCMEoHU8 Conference: October 7, 2025
Alex Zhang @a1zhang
13K Followers 598 Following phd student @MIT_CSAIL, ugrad @Princeton, 🫵🏻 go participate in the @GPU_MODE kernel competitions!
Jiashuo Liu @liujiashuo77
2K Followers 639 Following Research Scientist at ByteDance Seed | Advanced & Interesting LLM/Agent Evaluation. Opinions are my own.
Kevin Lu @_kevinlu
10K Followers 227 Following @thinkymachines. formerly: - @openai: RL, synthetic data, efficient models - @berkeley_ai: decision transformer, universal computation
Ying Sheng @ying11231
12K Followers 731 Following @lmsysorg @sgl_project | Prev. @xAI @Stanford | Assist Prof @UCLA. (delayed) | Do it anyway | Live to fight another day
Timor Averbuch @timorchik
31 Followers 28 Following
Liyuan Liu (Lucas) @LiyuanLucas
1K Followers 373 Following Researcher @MSFTResearch | 🎓 @UofIllinois Working on deep learning heuristics (aka tricks) He/him
Chris Lu @_chris_lu_
4K Followers 625 Following Research @OpenAI Prev: DPhil Student @UniofOxford, RS Intern @SakanaAILabs @DeepMind and RS @CovariantAI
Xuechen Li @lxuechen
16K Followers 944 Following Previously @xai. Interested in the engineering and science for scaling. Opinions are my own. @Stanford PhD.
Junli Wang @JunliWang2021
72 Followers 215 Following Undergraduate @Tsinghua_Uni | Prev visiting student @XLangNLP, advised by @taoyds | Intern @Alibaba_Qwen
World of Engineering @engineers_feed
3.5M Followers 69 Following The most fun way to learn something new everyday. Brother page of @stats_feed YouTube https://t.co/rAUIl0V1xC
Artificial Analysis @ArtificialAnlys
60K Followers 567 Following Independent analysis of AI models and hosting providers - choose the best model and API provider for your use-case
Matt Deitke @mattdeitke
13K Followers 301 Following AI Researcher @ Meta Superintelligence Lab Ph.D. dropout at @uwcse
Dylan X. Hou @XinmingHou
547 Followers 2K Following undergrad studying AI at Renmin Univ. of China, NLP researcher, intelligence explorer&trainer, interned@Tencent AI Lab. Carpe Diem🍀
張小珺 Xiaojùn @zhang_benita
18K Followers 80 Following 财经作者，写作中国商业深度报道，包括AI/科技巨头/风险投资/人物，也是播客《张小珺商业访谈录》主持人、制作人。Financial writer covering China business world, also the producer and host of "Zhang Xiaojun Podcast."
Z.ai @Zai_org
17K Followers 154 Following The AI lab behind GLM models, dedicated to inspiring the development of AGI to benefit humanity. https://t.co/b6zGxJvzzS
Zhihu Frontier @ZhihuFrontier
889 Followers 79 Following 🚀Bringing China's AI & tech trends, voices and perspectives to the global stage. ⚡️Powered by 知乎/https://t.co/OkIemRZdcj, China's leading knowledge community.
OpenRouter @OpenRouterAI
55K Followers 308 Following Discover and use the latest LLMs. 500+ models (incl. 50+ free), explorable data, private chat, & a unified API. https://t.co/qJG5mKrigL
Crystal @crystalsssup
13K Followers 664 Following Staff @Kimi_Moonshot prev. co-maker of ModelizeAI & gemsouls "Personality goes a long way" @UCSanDiego
Xingcheng Yao @StuartYao22139
288 Followers 270 Following Member of technical staff at @Kimi_Moonshot, Prev @uclanlp, @Tsinghua_IIIS, @princeton_nlp.
Lechao Xiao @Locchiu
1K Followers 596 Following Research Scientist @GoogleDeepMind / Google Brain. Tackle scaling, along the path to AGI.
Zonghan Yang @yang_zonghan
2K Followers 2K Following PhD student at Tsinghua NLP & AIR, studying agents that automate tasks ranging from daily activities to creative endeavors. Two drifters with the world to see.
M-A-P @MM_Art_Project
155 Followers 11 Following An open-source AI research community, known as SuperGPQA, YuE, MERT, OpenCodeInterpreter https://t.co/wiC7aNBZhU
Nathan Lambert @natolambert
57K Followers 857 Following Figuring out AI @allen_ai, open models, RLHF, fine-tuning, etc Contact via email. Writes @interconnectsai Wrote The RLHF Book Mountain runner
Wentao Guo @WentaoGuo7
328 Followers 163 Following CS PhD student @PrincetonCS, Previously CS MEng + BS @CornellCIS
Nathan Chen @nathancgy4
2K Followers 643 Following understanding models @tilderesearch, (hardware-aligned) ml & open-source, 16
Bitdeer AI @Bitdeer_AI
790 Followers 7 Following One-Stop Neocloud with AI solutions: Empower AI Growth Innovation NASDAQ: $BTDR
World of Statistics @stats_feed
4.5M Followers 427 Following There are three kinds of lies: lies, damned lies, and statistics. Sister page of @engineers_feed
Xinyu Zhou @zxytim
2K Followers 1K Following
Dinghuai Zhang 张鼎... @zdhnarsil
4K Followers 2K Following Researcher at @MSFTResearch. Prev: PhD at @Mila_Quebec, intern at @Apple MLR and FAIR Labs @MetaAI, math undergraduate at @PKU1898.
Johannes Oswald @oswaldjoh
1K Followers 640 Following Research Scientist, Paradigms of Intelligence Team, Google Zurich
chen zhuoming @chenzhuoming911
417 Followers 83 Following Ph.D. @SCSATCMU; undergraduate @Tsinghua_Uni
Lucas Beyer (bl16) @giffmana
110K Followers 523 Following Researcher (now: Meta. ex: OpenAI, DeepMind, Brain, RWTH Aachen), Gamer, Hacker, Belgian. Anon feedback: https://t.co/xe2XUqkKit ✗DMs → emailTrends for United States
You might like
