Fusheng Liu @mathlfs
PhD student @ National University of Singapore [email protected] mathematicallfs.github.io Singapore Joined September 2022-
Tweets51
-
Followers18
-
Following99
-
Likes534
Fusheng Liu retweeted
🚀 Introducing our new tech report: Muon is Scalable for LLM Training We found that Muon optimizer can be scaled up using the follow techniques: • Adding weight decay • Carefully adjusting the per-parameter update scale ✨ Highlights: • ~2x computational efficiency vs AdamW…
84
306
2K
750K
1K
Download Image
Fusheng Liu retweeted
🚀 Introducing DeepSeek-V3! Biggest leap forward yet: ⚡ 60 tokens/second (3x faster than V2!) 💪 Enhanced capabilities 🛠 API compatibility intact 🌍 Fully open-source models & papers 🐋 1/n
Highly recommend this user-friendly project if you start with LM pretraining and want to build your own model/optimizer. The repo is easy to understand, easy to edit and easy to implement new ideas with minimum workloads. Well done Keller! Looking forward to your records on VIT:)
Highly recommend this user-friendly project if you start with LM pretraining and want to build your own model/optimizer. The repo is easy to understand, easy to edit and easy to implement new ideas with minimum workloads. Well done Keller! Looking forward to your records on VIT:)
Fusheng Liu retweeted
Fixed a bug which caused all training losses to diverge for large gradient accumulation sizes. 1. First reported by @bnjmn_marie, GA is supposed to be mathematically equivalent to full batch training, but losses did not match. 2. We reproed the issue, and further investigation…
Fusheng Liu retweeted
Harm's Law of Smol Models (HLSM) tells us how much we need to scale up the data size (k_D) as we scale down the model size (k_N), if we wish to preserve the loss of a Chinchilla-optimal model. harmdevries.com/post/model-siz…
Fusheng Liu retweeted
Afraid of sum-of-squares (SOS) relaxations? Read this new blog post for a smooth ride in the Fourier domain. francisbach.com/sums-of-square…
Quentin Berthet @qberthet
3K Followers 2K Following Research Scientist at Google DeepMind Machine Learning - Paris
Julien Siems @julien_siems
318 Followers 668 Following PhD student advised by Frank Hutter working on linear RNNs and state-tracking.
Jing Zhang @JingZhang133376
1 Followers 6 Following
𝔻𝕃 @dlb07
192 Followers 7K Following
leloy! @leloykun
7K Followers 4K Following Math @ AdMU • NanoGPT speedrunner • Muon fan 🤍 • prev ML @ XPD • 2x IOI & 2x ICPC • https://t.co/nfO038itfn
Du Mingzhe @spirit__song
24 Followers 140 Following PhD Student in CS @NTUsg / @NUSingapore / @UniMelb
Yingnan Liu @yninthefuture
11 Followers 50 Following Machine learning PhD student from National University of Singapore. Work in Out-of-Distribution generalization and AI for healthcare.
Zhiyuan @ZhiyuanCS
393 Followers 202 Following PhD student in @NUSingapore Visiting Researcher in @MIT
Taco Cohen @TacoCohen
27K Followers 3K Following Post-trainologer at FAIR. Into codegen, RL, equivariance, generative models. Spent time at Qualcomm, Scyfer (acquired), UvA, Deepmind, OpenAI.
Alan Jeffares @ ICML ... @Jeffaresalan
2K Followers 733 Following Multiplying matrices @Cambridge_Uni & @MSFTResearch | PhD student in Machine Learning | Previously MSc @ucl & BSc @ucddublin
Claudio Gallicchio @claudiogallicc1
781 Followers 901 Following Associate Professor of ML at the University of Pisa (Italy). Deep Randomized Neural Networks, Reservoir Computing, Stable Architectures, Deep Learning 4 Graphs
Tan Minh Nguyen @TanNguyen689
1K Followers 745 Following Assistant Professor of Mathematics (Presidential Young Professor) at the National University of Singapore (@NUSingapore). #DeepLearning, #RobustAI, #ScalableAI
Bingchen Wang @BingchenWang1
7 Followers 55 Following Oxford MPhil Econ 21' | Columbia Math-Stats 19' | CityUHK Computing Math 19'
Ayaka @ayaka14732
2K Followers 311 Following Ex-JAX maintainer @Google, LLM, low-resource languages | Japanese account @ayaka4f
Rough Researcher @SapkotaTsuman
125 Followers 246 Following Rough AI blogger. Writing poems with code. Neural Networks. Currently on Interpretability, Sparsity, Continual Learning, Programmable NNs.
Roger Grosse @RogerGrosse
11K Followers 796 Following
Jonathan Lee @jon_lee0
749 Followers 106 Following research @GoogleDeepMind. co-developed gemini deep think. co-led model training for IMO 🥇 | prev: RL PhD at @StanfordAILab
Thang Luong @lmthang
27K Followers 95 Following Lead Superhuman Reasoning team @GoogleDeepMind. AI IMO Gold. Co-led #DeepThink, #AlphaGeometry, #Bard (now Gemini) Multimodality, #MeenaBot. LuongAttention.
Fuzhao Xue (Frio) @XueFz
5K Followers 683 Following Research Scientist @GoogleDeepmind | Scaling & Distillation #Gemini | Zero-shot Cooking Learner🧑🍳
Shai Shalev-Shwartz @shai_s_shwartz
3K Followers 173 Following Computer Science @HebrewU; CTO @Mobileye #selfdrivingcars #ai #machinelearning Opinions are my own.
Shengjia Zhao @shengjia_zhao
52K Followers 231 Following Chief Scientist @ Meta MSL. Formerly MTS @ OpenAI, PhD @ Stanford. I train models. All opinions my own.
Nino Scherrer @ninoscherrer
995 Followers 2K Following Research Scientist at @Google | Rigorous evaluations, cognitive science & causality | Ex: {@PatronusAI, @VectorInst, @Mila_Quebec, @MPI_IS, @ETH_en}
Johannes Oswald @oswaldjoh
1K Followers 641 Following Research Scientist, Paradigms of Intelligence Team, Google Zurich
Tairan He @TairanHe99
6K Followers 794 Following Robotics&AI PhD Student @CMU_Robotics Research Intern at @NVIDIA Prev: @MSFTResearch @sjtu1896 Emboddied AI; Humanoid; Robot Learning
Quentin Berthet @qberthet
3K Followers 2K Following Research Scientist at Google DeepMind Machine Learning - Paris
Riccardo Grazzi @riccardograzzi
108 Followers 131 Following Researcher at MSR Cambridge, previously at IIT in Genoa, Italy. Working on principled, efficient optimization for machine learning and on LLMs' expressivity.
Julien Siems @julien_siems
318 Followers 668 Following PhD student advised by Frank Hutter working on linear RNNs and state-tracking.
Nazneen Rajani @nazneenrajani
5K Followers 2K Following building @collinearAI 🧪 | MIT 35u35 | UN AI Advisory Body | Featured in NYT, Quanta, Science, MIT TR| Previously: @huggingface 🤗, @SFResearch, PhD @utcompsci
jianlin.su @Jianlin_S
3K Followers 14 Following Grad&Clip is all you need @Kimi_Moonshot Blog: https://t.co/YVxsWykMw2 , Cool Papers: https://t.co/scS1n1o0lg
Nancy Pelosi Stock Tr... @PelosiTracker_
1.2M Followers 524 Following Highlighting Politicians' trades so we can invest alongside Goal: get them banned from trading. $800,000,000 invested on @joinautopilot_ so far
Noam Razin @noamrazin
668 Followers 335 Following Postdoctoral Fellow at @PrincetonPLI | Past: Computer Science PhD @TelAvivUni & Apple Scholar in AI/ML | Interested in the foundations of deep learning
Jing Zhang @JingZhang133376
1 Followers 6 Following
DeepSeek @deepseek_ai
971K Followers 0 Following Unravel the mystery of AGI with curiosity. Answer the essential question with long-termism.
Jimmy Smith @jimmysmith1919
483 Followers 291 Following Founding Scientist @LiquidAI_. Previously PhD @Stanford and member of @scott_linderman's lab.
Scott Linderman @scott_linderman
5K Followers 822 Following Assistant Professor @Stanford Statistics and @StanfordBrain. Computational Neuroscience, Machine Learning, Bayesian Statistics. Tweets are my own.
leloy! @leloykun
7K Followers 4K Following Math @ AdMU • NanoGPT speedrunner • Muon fan 🤍 • prev ML @ XPD • 2x IOI & 2x ICPC • https://t.co/nfO038itfn
Du Mingzhe @spirit__song
24 Followers 140 Following PhD Student in CS @NTUsg / @NUSingapore / @UniMelb
Yingnan Liu @yninthefuture
11 Followers 50 Following Machine learning PhD student from National University of Singapore. Work in Out-of-Distribution generalization and AI for healthcare.
Institute of Data Sci... @NUS_IDS
16 Followers 33 Following 🌐 https://t.co/sEoYviavrz ✍ https://t.co/ttD0XrMTj9
Zhiyuan @ZhiyuanCS
393 Followers 202 Following PhD student in @NUSingapore Visiting Researcher in @MIT
Taco Cohen @TacoCohen
27K Followers 3K Following Post-trainologer at FAIR. Into codegen, RL, equivariance, generative models. Spent time at Qualcomm, Scyfer (acquired), UvA, Deepmind, OpenAI.
Simons Institute for ... @SimonsInstitute
9K Followers 311 Following The world's leading venue for collaborative research in theoretical computer science. Follow us at https://t.co/KvcuGI7WM0.
Alexander Wei @alexwei_
24K Followers 194 Following Reasoning @OpenAI. Co-built CICERO @MetaAI | @Berkeley_AI PhD '23 | @Harvard '20
Daniela Massiceti @dannimassi
2K Followers 406 Following Machine learning researcher at Microsoft Research | based in Sydney | PhD @UniofOxford | South African 🇿🇦 | @DeepIndaba 🐘 | maker of things
Alan Jeffares @ ICML ... @Jeffaresalan
2K Followers 733 Following Multiplying matrices @Cambridge_Uni & @MSFTResearch | PhD student in Machine Learning | Previously MSc @ucl & BSc @ucddublin
Andrej Karpathy @karpathy
1.4M Followers 1K Following Building @EurekaLabsAI. Previously Director of AI @ Tesla, founding team @ OpenAI, CS231n/PhD @ Stanford. I like to train large deep neural nets.
William Merrill @lambdaviking
5K Followers 667 Following Incoming Assistant Prof, Toyota Technical Institute at Chicago @TTIC_Connect Recruiting PhD students (start 2026) 👀 Will irl - TC0 enthusiast
Zhiyuan Li @zhiyuanli_
2K Followers 317 Following Assistant Professor @TTIC_Connect. Previously Postdoc @Stanford and PhD @PrincetonCS. Deep Learning Theory.
Trends for United States
35,6 B posts
13,9 B posts
648 B posts
19,7 B posts
5.024 posts
22,3 B posts
1.921 posts
45,9 B posts
1.692 posts
3.439 posts
10,2 B posts
1.664 posts
1.745 posts
2.748 posts
35 B posts
1.644 posts
160 B posts
1.112 posts
You might like
Elon Musk
@elonmusk
226.6M Followers
Barack Obama
@BarackObama
130.2M Followers
Cristiano Ronaldo
@Cristiano
115.4M Followers
Donald J. Trump
@realDonaldTrump
109.4M Followers
Justin Bieber
@justinbieber
108.1M Followers
Rihanna
@rihanna
107.1M Followers
Taylor Swift
@taylorswift13
93.1M Followers
NASA
@NASA
88.0M Followers
Lady Gaga
@ladygaga
81.2M Followers
YouTube
@YouTube
78.6M Followers
Kim Kardashian
@KimKardashian
74.1M Followers
The Ellen Show
@EllenDeGeneres
73.2M Followers
X
@X
68.5M Followers
Virat Kohli
@imVkohli
67.8M Followers
Bill Gates
@BillGates
66.3M Followers
Selena Gomez
@selenagomez
64.6M Followers
CNN Breaking News
@cnnbrk
63.7M Followers
Neymar Jr
@neymarjr
63.7M Followers
CNN
@CNN
63.6M Followers
PMO India
@PMOIndia
59.1M Followers