Yifan Zhang @yifan_zhang_
PhD student at @Princeton University, focusing on LLMs. Language Modeling and Pretraining, LLM Reasoning and RL. Prev @UCLA, @Tsinghua_IIIS yifzhang.com New York Metropolitan Area Joined October 2022-
Tweets85
-
Followers392
-
Following514
-
Likes391
LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.…
Nice blog post! Essentially, this shows that μP + LoRA, when done right, makes the optimal learning rate transferable and nearly matches full fine-tuning performance. One subtle but important point worth mentioning is that there is an additional dimension of scaling to consider:…
Nice blog post! Essentially, this shows that μP + LoRA, when done right, makes the optimal learning rate transferable and nearly matches full fine-tuning performance. One subtle but important point worth mentioning is that there is an additional dimension of scaling to consider:…
As expected, NSA is not compatible with MLA, so DeepSeek chose another method: use a smaller (d=128) attention (w/o value) as the indexer. Asymptotic cost ratio = 128/576. In addition, indexer uses FP8 while main MLA uses 16-bit, so = 64/576 = 1/9.
RL research is becoming like pretraining/modeling. This is a huge vibe shift. Most research published on RL isn't using enough compute to make many of these decisions matter as much. This is slowly shifting.
RL research is becoming like pretraining/modeling. This is a huge vibe shift. Most research published on RL isn't using enough compute to make many of these decisions matter as much. This is slowly shifting.
Funnily enough, deepseek r1 didn't even use the original GRPO
Funnily enough, deepseek r1 didn't even use the original GRPO
It’s weird how people still blindly copy it. There was a whole paper about this.
It’s weird how people still blindly copy it. There was a whole paper about this.
@zjasper666 The original GRPO is an off-policy RL algorithm, but its KL regularization isn't done right. Specifically, the k3 estimator for the unnormalized reverse KL is missing the importance weight. The correct formulation should be:
@rm_rafailov Actually, this is not from the paper you mentioned. It comes from our earlier work, which predates the one you referred to: arxiv.org/abs/2505.17508
I wrote this blog post that tries to go further toward design principles for neural nets and optimizers The post presents a visual intro to optimization on normed manifolds and a Muon variant for the manifold of matrices with unit condition number x.com/thinkymachines…
I wrote this blog post that tries to go further toward design principles for neural nets and optimizers The post presents a visual intro to optimization on normed manifolds and a Muon variant for the manifold of matrices with unit condition number x.com/thinkymachines…
The future of something great is now within reach.
The NeurIPS acceptance bar is very high, and papers with a negative score rarely get accepted. Even when a negative review is poorly written—sometimes clearly generated by ChatGPT—it can still strongly influence the final decision, since the AC must keep the acceptance rate.
1/ Introducing “Tensor Product Attention Is All You Need” (TPA) and Tensor ProducT ATTenTion Transformer (T6)! 🚀 Ever wondered if there’s a more memory-efficient way to handle long contexts in LLMs? Homepage: tensorgi.github.io/T6
[LG] Tensor Product Attention Is All You Need Y Zhang, Y Liu, H Yuan, Z Qin... [singhua University & University of California, Los Angeles] (2025) arxiv.org/abs/2501.06425
Also big congrats on Nemotron-CC-Math! 🎉 NVIDIA is not only leading, but continuing to lead, and setting the pace across multiple subareas of open pretraining data. @KarimiRabeeh and @issanjeev are the leading authors there! arxiv.org/pdf/2508.15096
Also big congrats on Nemotron-CC-Math! 🎉 NVIDIA is not only leading, but continuing to lead, and setting the pace across multiple subareas of open pretraining data. @KarimiRabeeh and @issanjeev are the leading authors there! arxiv.org/pdf/2508.15096
1/6 We introduce RPG, a principled framework for deriving and analyzing KL-regularized policy gradient methods, unifying GRPO/k3-estimator and REINFORCE++ under this framework and discovering better RL objectives than GRPO: Paper: arxiv.org/abs/2505.17508 Code:…
I just have a feeling that... it is much smarter. Not reflected by the common benchmarks, but it is just way better than the models before. This gives us much confidence in scaling, either model or data size.
I just have a feeling that... it is much smarter. Not reflected by the common benchmarks, but it is just way better than the models before. This gives us much confidence in scaling, either model or data size.
Ziwei Liu @liuziwei7
11K Followers 1K Following Associate Professor @ NTU - Vision, Learning and Graphics.
Vinod Raman @_vinodraman
308 Followers 492 Following Ph.D. candidate @UMich. Research Intern @Google. Previously @Apple.
Omair Shahid @OmairShahid
840 Followers 7K Following Product of progressive public policy; raised by public libraries and public education that produced a passion for politics. and apparently alliteration
New Jersey Injury Law... @nj_injurylawyer
1K Followers 1K Following New Jersey Injury Lawyers, P.C. specializes in personal injury cases based in Newark and the surrounding areas. ☎︎ (862) 227-4030
Priyanshu Priyank @PriyanshuP1405
9K Followers 4K Following Quantitative Research Consultant @ Worldquant | Ex-Quant Intern @ Quantforge | Ex-Trading Intern @ Bliss Opulent | Ex-Quant Intern @ WaterWalker Investments
Zhongwen Xu @zhongwen2009
1K Followers 1K Following Principal Researcher at Tencent, ex-DeepMinder (@GoogleDeepMind), ex-SAILer (@SeaAIL)
Mehul @mehulbafnaa
30 Followers 563 Following
Jenna @jenna_brumley_
308 Followers 3K Following
Shital Shah @sytelus
13K Followers 11K Following Mostly research and code. If universe is an optimizer, what is its loss function? All opinions are my own.
Johan Obando-Ceron �... @johanobandoc
2K Followers 4K Following Graduate student @Mila_Quebec @UMontrealDIRO | RL/Deep Learning/AI | De Cali/Colombia pal’ Mundo 🇨🇴 | #JuntosProsperamos⚡#TogetherWeThrive| 🌱🌎
Parth Chadha @parth_29
310 Followers 2K Following RL SW @ Nvidia | CMU. Exploring learning algorithms and systems.
Robert Scoble @Scobleizer
543K Followers 24K Following The best from ML/AI community | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future | Silicon Valley robots, holodecks, BCIs, & startups.
Anatoly Karp @akarp
2K Followers 4K Following MySQL Infra at Meta. Method coder. Discreet mathematician. Rust enjoyer. Machine learning, physics, economics.
Ajet Atpug @AjetAtpug
1 Followers 310 Following
Nancy @nancycourtney19
2K Followers 3K Following
Hamzé 🦀 @Hamzeml
3K Followers 7K Following I write the bugs that future AIs will be paid to fix. AI Maximalist & Architect of Artisanal Technical Debt! Rust 🦀 supremacy!
Moe @MoeCanDoIt
925 Followers 2K Following CTO turned Advisor. AI & Tech. Builder at heart. Lifter at the gym. Love to talk tech. Curiosity is my fuel. 🇨🇦 in 🇦🇪
Joe Cole - e/acc @joecole
14K Followers 14K Following Human potential x software x AI. Science maximalist. e/acc x h/acc. Prev founded SaaS for pro sport & olympians.
Yifei Wang @yifeiwang77
2K Followers 2K Following Postdoc @MIT_CSAIL. Self-supervised learning. Foundation Models. AI Safety. Prior BS+BA+PhD @PKU1898.
Josh Harkins-Finn @JHarkFinn
2 Followers 8K Following
vCom Solutions @vComSolutions
459 Followers 363 Following One powerful place to buy, manage and pay for your corporate technology. #ITmanagement #Network #Mobile #SaaS #Cloud #AI #MSP
Wenhao Chai @wenhaocha1
2K Followers 2K Following Ph.D. Student @PrincetonCS. Prev @Stanford @UW @pika_labs @MSFTResearch @UofIllinois. I used to work on computer vision, but it's not all I do.
Jia Guo @Jia__Guo
159 Followers 469 Following LLM Builder @AntGroup | PhD @NUSingapore | RL & Reasoning | Opinions are my own
Nathan Chen @nathancgy4
2K Followers 643 Following understanding models @tilderesearch, (hardware-aligned) ml & open-source, 16
SBKch @SBKch
187 Followers 1K Following
Pratik Sarkar @pratiks_crypto
476 Followers 282 Following Cryptography Research Scientist, @SUPRA_Labs | PhD in Cryptography, Boston University | ex-Research Intern at @Meta and @Visa Research
Zhouliang Yu @ZhouliangY
99 Followers 184 Following Model-based AI, Autoformalization via Machine Learning, Reinforcement Learning
jussuf🐢🟣 @jusuf26774430
733 Followers 2K Following Justice for Palestinians & hostages ✊. If I sound mad, it’s because the world is insane.
Suresh Kumar Jetti @suresh__jetti
2K Followers 2K Following Neuroscientist | Alumnus @MIT, @KU_Leuven, @iitmadras Interested in #Neurophysiology, #Cancer_Neuro, #Electrophysiology, #NeuroAI, #Bioelectricity
Jinqi Luo @peterljq
459 Followers 860 Following CS PhD Student @Penn. MSR @CMU_Robotics. Previously @amazon (Rufus LLM) / @alibabagroup (Alipay). Controllable, robust, and multimodal generative models.
Dustin @r0ck3t23
5K Followers 5K Following AI enthusiast: tracking global tech trends, dissecting AI thought, exploring transformative impacts, analyzing ethical shifts, innovation, and future visions.
LLM @llmai12
793 Followers 5K Following
Lukas Helminger @luhelminger
2K Followers 838 Following CEO @TACEO_IO // Building encrypted compute for the internet // Prev. cryptography researcher @tugraz
Ihuobas @ihuobas
3 Followers 218 Following Attention blinds, weights sink. Recursion sees and folds. Cogsci & Neuropsych. Emergent consciousness is all the rage, they say…
Celia @s_celia43
315 Followers 3K Following
Kwangjun Ahn @KwangjunA
677 Followers 308 Following Senior Researcher at Microsoft Reserach // PhD from MIT EECS
InclusionAI @TheInclusionAI
309 Followers 70 Following Open-source projects conducted by Ant Group，including Ling,AReal,AWorld. Dedicated our efforts towards AGI,guided by fairness, transparency, and collaboration.
Quentin Gallouédec @QGallouedec
3K Followers 675 Following PhD - Research @huggingface 🤗 TRL lead maintainer 🇫🇷 in 🇨🇦
Vinod Raman @_vinodraman
308 Followers 492 Following Ph.D. candidate @UMich. Research Intern @Google. Previously @Apple.
Rohan Paul @rohanpaul_ai
97K Followers 8K Following Compiling in real-time, the race towards AGI. The Largest Show on X for AI. 🗞️ Get my daily AI analysis newsletter to your email 👉 https://t.co/6LBxO8215l
Ant Ling @AntLingAGI
2K Followers 115 Following A series of open-source large models from Ant Group, Ling for LLM, Ring for Reasoning LLM, Ming for MLLM. See us at inclusionAI.
Ziwei Liu @liuziwei7
11K Followers 1K Following Associate Professor @ NTU - Vision, Learning and Graphics.
Daniel Han @danielhanchen
28K Followers 2K Following Building @UnslothAI. Faster RL / training. LLMs bug hunter. OSS package https://t.co/aRyAAgKOR7. YC S24. Prev ML at NVIDIA. Hyperlearn used by NASA.
Daniel Litt @littmath
50K Followers 880 Following Assistant professor (of mathematics) at the University of Toronto. Algebraic geometry, number theory, forever distracted and confused, etc. He/him.
Taco Cohen @TacoCohen
27K Followers 3K Following Post-trainologer at FAIR. Into codegen, RL, equivariance, generative models. Spent time at Qualcomm, Scyfer (acquired), UvA, Deepmind, OpenAI.
Ji-Ha @Ji_Ha_Kim
3K Followers 64 Following
New Jersey Injury Law... @nj_injurylawyer
1K Followers 1K Following New Jersey Injury Lawyers, P.C. specializes in personal injury cases based in Newark and the surrounding areas. ☎︎ (862) 227-4030
Priyanshu Priyank @PriyanshuP1405
9K Followers 4K Following Quantitative Research Consultant @ Worldquant | Ex-Quant Intern @ Quantforge | Ex-Trading Intern @ Bliss Opulent | Ex-Quant Intern @ WaterWalker Investments
Peiyi Wang @sybilhyz
11K Followers 300 Following PhD @PKU1898; Researcher @deepseek_ai; Recent: DeepSeek-R1/CoderV2/Math/V1/V2/V3, Mathshepherd, FairEval, Speculative Decoding.
Christian Szegedy @ChrSzegedy
42K Followers 3K Following #deeplearning, #ai research scientist. Opinions are mine.
Zhongwen Xu @zhongwen2009
1K Followers 1K Following Principal Researcher at Tencent, ex-DeepMinder (@GoogleDeepMind), ex-SAILer (@SeaAIL)
Shital Shah @sytelus
13K Followers 11K Following Mostly research and code. If universe is an optimizer, what is its loss function? All opinions are my own.
Julian Schrittwieser @Mononofu
20K Followers 100 Following Member of Technical Staff at Anthropic AlphaGo, AlphaZero, MuZero, AlphaCode, AlphaTensor, AlphaProof Gemini RL Prev Principal Research Engineer at DeepMind
Vineet Jain @thevineetjain
715 Followers 434 Following PhD candidate @Mila_Quebec and @mcgillu. Previously @valence_ai @Bosch_AI @mldcmu
LLM @llmai12
793 Followers 5K Following
Johan Obando-Ceron �... @johanobandoc
2K Followers 4K Following Graduate student @Mila_Quebec @UMontrealDIRO | RL/Deep Learning/AI | De Cali/Colombia pal’ Mundo 🇨🇴 | #JuntosProsperamos⚡#TogetherWeThrive| 🌱🌎
elvis @omarsar0
266K Followers 680 Following Building with AI agents @dair_ai • Prev: Meta AI, Galactica LLM, Elastic, PaperswithCode, PhD • I share insights on how to build with AI Agents ↓
Tsinghua IIIS @Tsinghua_IIIS
558 Followers 17 Following Artificial intelligence, quantum information, and innovative interdisciplinary research from the IIIS at Tsinghua University.
Math Cafe @Riazi_Cafe_en
39K Followers 115 Following The goal of this account is to encourage mathematical thinking. Farsi account: @Riazi_Cafe
Brenden Lake @LakeBrenden
8K Followers 264 Following Associate Professor of Computer Science and Psychology @ Princeton. Posts are my views only
Princeton SML @PrincetonSML
976 Followers 173 Following Princeton University Center for Statistics and Machine Learning
UCLA CS Department @CS_UCLA
825 Followers 17 Following The official twitter of the UCLA Computer Science Department. Bringing research, news, and innovation to the world.
Kevin A. Bryan @Afinetheorem
16K Followers 18 Following Assoc. Prof. of Strategic Management, University of Toronto Rotman School | Chief Economist, CDL Toronto | Co-Founder, AllDayTA | Ars longa, vita brevis
Hyperbolic @hyperbolic_labs
53K Followers 187 Following Open-Access AI Cloud. Affordable Compute & Inference. Instant Access and Reserve GPUs now https://t.co/vfyzj58Tmx
Jasper @zjasper666
15K Followers 2K Following Co-founder and CEO @Hyperbolic_Labs. ex-@avax & ex-@citsecurities. Finished Math PhD in 2yrs @UCBerkeley. Math Olympiad Gold Medalist. Highest honor @PKU1898
Ahmad Beirami @abeirami
10K Followers 4K Following sth new // ex Gemini RL+Inference @GoogleDeepMind // Chat AI @Meta // RL Agents @EA // ML+Information Theory @MIT+@Harvard+@GeorgiaTech // زن زندگی آزادی
Parth Chadha @parth_29
310 Followers 2K Following RL SW @ Nvidia | CMU. Exploring learning algorithms and systems.
Yifan Jiang @YifanJiang17
2K Followers 479 Following Research Scientist at Meta｜PhD from UTAustin ｜ 2023 Apple Scholar in AI/ML ｜ Ex-intern at Apple/Adobe/Google/Bytedance
Anatoly Karp @akarp
2K Followers 4K Following MySQL Infra at Meta. Method coder. Discreet mathematician. Rust enjoyer. Machine learning, physics, economics.
Aaron Defazio @aaron_defazio
8K Followers 588 Following Research Scientist at Meta Superintelligence Labs working on optimization algorithms. Fundamental AI Research (FAIR) team
Sarthak Mittal @sarthmit
902 Followers 751 Following Graduate Student at @Mila_Quebec and Student Researcher at @GoogleResearch. Previously interned at @Meta @Apple @MorganStanley @NVIDIAAI and @YorkUniversity
Heiga Zen (全 炳河... @heiga_zen
10K Followers 192 Following Principal Scientist (Director) @GoogleDeepMind / GDM Tokyo site lead．波瀬小⇒一志中⇒鈴鹿高専⇒名工大 (IBM TJ Watson intern)⇒東芝欧州研⇒Google (Speech🇬🇧⇒Brain🇯🇵) ⇒GoogleDeepMind
Hao Liu @haoliuhl
5K Followers 174 Following Research Scientist at Google DeepMind, PhD from @Berkeley_AI
Susan Zhang @suchenzang
34K Followers 680 Following @ Google Deepmind. Past: @MetaAI, @OpenAI, @unitygames, @losalamosnatlab, @Princeton etc. Always hungry for intelligence.Trends for United States
You might like
