Our flagship paper on how far careful quantization can really go in practice got accepted as an oral at ACL 2025 (top 8%)! 🥳
Turns out, old-school methods like GPTQ, SmoothQuant, and RTN are quite good when tuned properly.
All of the tricks are already in LLM-Compressor!
This is something I've been working on with some amazing collaborators for a while. Model-software-hardware co-design. Making things run fast on real devices. A lot of learning.
And happy to share this with the open-source community and beyond.
developers.googleblog.com/en/introducing…
📣 The Journey Matters: Our #ICLR2025 paper shows how to pretrain sparse LLMs with half the size of dense LLMs while maintaining quality. We found that the average parameter count during sparse pre-training predicts quality, not final size. An MIT/Rice/Google/ISTA collab 🧵 1/N
56 Followers 156 FollowingInvestigador independiente de Modelos pequeños de IA, vtuberMex, VtuberESP, Trainer LLM, actualmente estoy desarrollando mi propio modelo mini
290 Followers 235 Following🤖 Exploring open source TTS/LLMs & sometimes blogging about it.
☁️ Building GenAI applications in the cloud for work. DMs open.
6K Followers 218 FollowingIncoming assistant professor at UCSD CSE in MLSys. Currently recruiting students! Also running the kernels team @togethercompute.
640 Followers 21 FollowingBuilt by researchers and engineers from MIT, we are pursuing Artificial Efficient Intelligence (AEI). Try GPT-OSS support: https://t.co/BQfsnXIGFo.
2K Followers 76 FollowingGoogle Fellow, VP | Gemini Data Area Lead | Algorithms, GraphML, ML efficiency, Economics @ Google Research. Former MSR, Amazon, MIT PhD, Sharif Univ. BSc
11K Followers 685 FollowingML theory nerd & AI non-enthusiast. thinking a lot about online learning these days!
BTW you should go find me on another website where i post more actively
253 Followers 37 FollowingACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
March 2nd – March 6th, 2024, Edinburgh, UK
Official hashtag this year: #ppopp24
11K Followers 3K FollowingSenior director of Cisco Foundation AI, Former Chief Scientist at Robust Intelligence. ex Professor at Yale University, ex staff research scientist at Google.
24K Followers 1 Followingcovering the latest AI & LLM research /// see "highlights" for all previous weekly threads /// building the best AI paper search engine @findmypapersai
29K Followers 1K FollowingAI, national security, China. Part of the founding team at @CSETGeorgetown (opinions my own). Author of Rising Tide on substack: https://t.co/LKAoyL00iB
357 Followers 37 FollowingEfficient Systems for Foundation Models Workshop, ICML2025.
Join us if you are interested in the challenges associated with large models training & inference!
437K Followers 762 FollowingComplex systems, wicked problems. Society, technology, science and more. @Princeton professor. @NYTimes columnist. My newsletter @insight https://t.co/6Ky01N9JwA