Simon Mo @simon_mo_
@vllm_project Joined July 2018-
Tweets125
-
Followers1K
-
Following342
-
Likes146
🔥 This is a great read!! Exercise for the reader: how would Blackwell Ultra with 2x exponential cores impact the design of FA4? developer.nvidia.com/blog/inside-nv…
🔥 This is a great read!! Exercise for the reader: how would Blackwell Ultra with 2x exponential cores impact the design of FA4? developer.nvidia.com/blog/inside-nv…
Day 1 with @modal notebook and it's so much fun! Switching from CPU to GPU easily between cells while maintaining environments and volumes is 🤌 * Run CPU nodes to download checkpoints and simple dev work with vLLM for testing * Scale out to B200 when ready!
Just enabled full cudagraphs by default on @vllm_project! This change should offer a huge improvement for low latency workloads on small models and efficient MoEs For Qwen3-30B-A3B-FP8 on H100 at bs=10 1024/128, I was able to see a speedup of 47% 🔥
It has been 1+ month of intense work! Now time to get some sleep 😴
It has been 1+ month of intense work! Now time to get some sleep 😴
I didn't expect the first section "KV-cache hit rate is the single most important metric for a production-stage AI agent" but 🤯
I didn't expect the first section "KV-cache hit rate is the single most important metric for a production-stage AI agent" but 🤯
Long time in the making and I'm beyond excited about the future of vLLM!
Long time in the making and I'm beyond excited about the future of vLLM!
Announcing the first Codex open source fund grant recipients: ⬩vLLM - inference serving engine @vllm_project ⬩OWASP Nettacker - automated network pentesting @iotscan ⬩Pulumi - infrastructure as code in any language @PulumiCorp ⬩Dagster - cloud-native data pipelines @dagster…
😲 super cool !!! Reminded me of Kevin's thesis "Structured Contexts For Large Language Models" and this is such a natural continuation of the idea.
😲 super cool !!! Reminded me of Kevin's thesis "Structured Contexts For Large Language Models" and this is such a natural continuation of the idea.
🙏 @deepseek_ai's highly performant inference engine is built on top of vLLM. Now they are open-sourcing the engine the right way: instead of a separate repo, they are bringing changes to the open source community so everyone can immediately benefit! github.com/deepseek-ai/op…
Having been at every single vLLM meetup, I won't miss this one :D Looking forward to meet all the vLLM users in Boston!
Having been at every single vLLM meetup, I won't miss this one :D Looking forward to meet all the vLLM users in Boston!
Landed my first PR in @vllm_project 1 year ago today (github.com/vllm-project/v…) 38K LOC and 100+ PRs later and we are just getting started
Robert and I started contributing to vLLM around the same time and today is my turn. Back then vLLM had only about 30 contributors. One year later, today the project has received contributions from 800+ community members! and we're just getting started github.com/vllm-project/v…
Robert and I started contributing to vLLM around the same time and today is my turn. Back then vLLM had only about 30 contributors. One year later, today the project has received contributions from 800+ community members! and we're just getting started github.com/vllm-project/v…
We landed the 1st batch of enhancements to the @deepseek_ai models, starting MLA and cutlass fp8 kernels. Compared to v0.7.0, we offer ~3x the generation throughput, ~10x the memory capacity for tokens, and horizontal context scalability with pipeline parallelism.
sarthak @srthkdev
889 Followers 2K Following techno-optimist, ind/acc, dharma, fitness. agents, applied ai.
SigridValentine @01SxBNgX2wZmxy
2 Followers 268 Following
Hemil Desai @HemilDesai10
376 Followers 6K Following Senior Software Engineer @NVIDIA, opinions are my own
Kit @KITTpatel
26 Followers 88 Following
Jason Cui @JasonSCui
3K Followers 1K Following Partner @a16z investing in infra & AI | prev product @databricks & @uber, founder @jemihq (YC S20, acq). Technology optimist ☀️
Mohammad Angkad @mmangkad
0 Followers 125 Following
Daniel Han @danielhanchen
28K Followers 2K Following Building @UnslothAI. Finetune train LLMs faster. LLMs bug hunter. OSS package https://t.co/aRyAAgKOR7. YC S24. Prev ML at NVIDIA. Hyperlearn used by NASA.
Men1scus @Men1scus
168 Followers 4K Following Junior@Nankai University | Major in CS | Research in GenAI & Infra | Full Stack Developer | Beginner in Crypto | Runner, Cyclist, Gym-goer | Rap enthusiast
Robert Scoble @Scobleizer
543K Followers 23K Following The best from ML/AI community | Ex-Microsoft, Rackspace, Fast Company | Wrote eight books about the future | Silicon Valley robots, holodecks, BCIs, & startups.
I07XNbUI4 @DeepFeed2
88 Followers 6K Following
Aryan Bansal @aryan_banana
980 Followers 468 Following 19 | researching @berkeley_ai | prev. co-founder @useKled | @ucberkeleymet
star @DankeJian666
0 Followers 140 Following
Jianwei @chinalujw
34 Followers 238 Following
Long Le @le_11a9
131 Followers 847 Following Software engineer https://t.co/UkB5Zfg7uy - a memory layer for AI coding agents
losejobaftertakephoto... @nojobafterphoto
100 Followers 552 Following A researcher who has 100+ ICML/NeurIPS/ICLR papers, lose job after taken photos with fans at ICML 2025 Advising high school students to write papers.
Lenny Pruss @lennypruss
4K Followers 1K Following VC @AmplifyPartners. Tepid takes on distributed systems, dev tools and #SJSharks
marathon bee @sidrbee
0 Followers 69 Following
Eric Zelikman @ericzelikman
21K Followers 2K Following building for humans // was lgtm-ing @xAI, phd-ing @stanford
Piyush Byahut @piyush_byahut
3 Followers 530 Following
Bram @BramVanroy
1K Followers 832 Following @ku_leuven @ccl_kuleuven: Creative #NLG 🖋️ @ivdnt: Dutch #NLProc and #LLMs Creator of Dutch LLMs 🤖 Fellow at @huggingface 🤗 Prev. @lt3ugent, @SignON
Ashok Pon Kumar @ashokponkumar
274 Followers 2K Following Creator of Konveyor Move2Kube. I build Cloud Native and AI systems to solve real world problems. I manage the Hybrid Cloud Platforms team in IBM Research India
Mayank Bhaskar @cataluna84
3K Followers 4K Following Machine Learning Consultant 🧑🏽💻 | @twimlai & @Cohere_Labs Community Lead 👥 | @AILucknow ⌨ | #engineer 🛠 🧮 | #datavisualization 📊 | #sports ⚽ 🏓 🏋🏽 🎮
tonyinnk @tonyinnk
1 Followers 58 Following
Shekhar @indianspeedster
242 Followers 235 Following AI at @AMD | New York University 24" Former SDE @Boschsoftware, MLE @bytez :) -_- :(
Shreya Shekhar @_shreya_s
3K Followers 1K Following investing in AI, Cyber, Infra @GreylockVC | @bedrocksec @AbnormalSec @glean @RocksetCloud | @ucberkeleymet
HL hl @kuchiioct25
21 Followers 2K Following
Shashwat Gupta @ShashwatGupta23
77 Followers 1K Following AI Guy; loving my time at Microsoft Research.
Yuming DU @yuming_du
1K Followers 4K Following Research Scientist at @Meta GenAI. | PhD in Computer Vision @ImagineEnpc | Opinions are my own
Paige Beane @BeanePaige
0 Followers 16 Following
Yahya @AliSherGhumman_
0 Followers 45 Following Parallel Programming | Distributed Systems | Deep Learning
Augustas Macijauskas @augustasmac
30 Followers 570 Following ML engineer at Spike Technologies. Previously, a machine learning master's student at @Cambridge_Uni. Excited about opportunities in deep-learning and LLMs.
Bryce Schmidtchen @_bschmidtchen
756 Followers 1K Following building something new · ex- Vision Pro · SF 🌁
!.! @xypyth
57 Followers 5K Following
Charles 🎉 Frye @charles_irl
15K Followers 3K Following gpu enjoyer at @modal. he/him. ex @full_stack_dl, @weights_biases (acq. @CoreWeave), phd Berkeley @Redwood_Neuro. try https://t.co/SYWVMCazZ3
James Alcorn @JamesAlcorn94
1K Followers 2K Following AI/infra at Lightspeed. Prev Zetta VC, Spectrum Equity, UCBerkeley 🇦🇺 🇺🇸
Character.AI @character_ai
149K Followers 13 Following Unleash your imagination—chat, create, explore.
Ion Stoica @istoica05
5K Followers 20 Following Professor at UC Berkeley, co-founder of Databricks, Anyscale, LMArena, Conviva.
SemiAnalysis @SemiAnalysis_
37K Followers 18 Following
NovaSky @NovaSkyAI
3K Followers 15 Following Next-generation Open Vision and AI @BerkeleySky Contact: [email protected]
OpenAI @OpenAI
4.4M Followers 3 Following OpenAI’s mission is to ensure that artificial general intelligence benefits all of humanity. We’re hiring: https://t.co/dJGr6Lg202
Yichao 'Peak' Ji @peakji
25K Followers 170 Following Co-founder & Chief Scientist of Manus @ManusAI_HQ
Letta @Letta_AI
4K Followers 0 Following Stateful agents that remember and learn https://t.co/4upATozfXj
Hao Zhang @haozhangml
6K Followers 477 Following Asst. Prof. @HDSIUCSD and @ucsd_cse running @haoailab. Cofounder and runs @lmsysorg. 20% with @Snowflake
Liangsheng Yin （尹... @lsyincs
266 Followers 260 Following B.E. @sjtu1896 Maintainer @sgl_project Incoming Ph.D. Student @UCBerkeley
Michael Luo @michaelzluo
672 Followers 218 Following Project Lead @Agentica_ | Prev. Researcher @GoogleDeepMind | PhD at UC Berkeley @berkeley_ai
Amog Kamsetty @AmogKamsetty
330 Followers 219 Following AI Systems @BerkeleySky← Building @raydistributed + founding ML eng @anyscalecompute ← ML research @ucbrise
Zihao Ye @ye_combinator
2K Followers 551 Following Proud to be an engineer. I'm building flashinfer (https://t.co/PabCM3ksjN) Opinions are my own.
Kaichao You @KaichaoYou
4K Followers 134 Following phd student in tsinghua university, working on @vllm_project
Justin Wong @justinywong_
228 Followers 225 Following CS PhD Student working on diverse AI generation advised by @profjoeyg and Sanjit Seshia. formerly @aiatmeta, @nuro, @columbia
Tianle Cai @tianle_cai
8K Followers 5K Following ML PhD @Princeton. Life-long learner, hacker, and thinker. Tech consultant & angel investor. Prev @togethercompute @GoogleDeepMind @MSFTResearch @citsecurities.
Myle Ott @myleott
2K Followers 542 Following research @thinkymachines, previously @GoogleDeepMind, @character_ai and @MetaAI
Bowen Liang @LiangBowen
276 Followers 223 Following Research Engineer @GoogleDeepMind, formerly @character_ai
Tian Xie @tianxie233
312 Followers 452 Following Research Scientist @GoogleDeepMind | previously @character_ai @SFResearch @USC_ISI
Stephen Roller @stephenroller
5K Followers 1K Following MoTS @thinkymachines. previously pre-training @googledeepmind ,@character_ai, and @aiatmeta.
Ying Xiao @YingXiao
818 Followers 300 Following ML Researcher at @character_ai; previously @googIeresearch, @Twitter Cortex, and @GoldmanSachs. Making haste slowly.
Sam Shleifer @sam_shleifer
3K Followers 857 Following Founding Team Thinking Machines. Previously @c.ai. I like zsh aliases, audiobooks, and running. This does not reflect the opinions of my future self.
Xiangyi Li 李向一 @xdotli
2K Followers 1K Following making dangos. backed by @fdotinc. made @benchflow_ai @lemmahq
Kenneth Ding @kd78277
141 Followers 126 Following AI Research Engineer @LeptonAI | Creator of GPTFast | Math & CS @ NYU Courant
Jason Wei @_jasonwei
98K Followers 639 Following ai researcher @meta superintelligence labs, past: openai, google 🧠
Yi Tay @YiTayML
47K Followers 82 Following research scientist @googledeepmind ✨♊, model co-lead/captain of gemini deepthink imo gold medal 🥇, opinions are my own.
Hyung Won Chung @hwchung27
38K Followers 303 Following AI Research Scientist @Meta Superintelligence Labs. Past: @OpenAI / @Google Brain / PhD @MIT
Kai Fricke @kaifri
9 Followers 53 Following
lmarena.ai @arena
94K Followers 207 Following LMArena: Open Platform for Community-driven AI Benchmarking. Graduated from UC Berkeley / @lmsysorg. We’re hiring: https://t.co/1OkfLq2n0I
Romil Bhardwaj @bromil101
401 Followers 185 Following Building SkyPilot @skypilot_org | PhD AI+Systems @Berkeley_EECS @ucbrise
Bill Gurley @bgurley
720K Followers 2K Following VC @benchmark, Investor: @grubhub, @zillow, @uber, @stitchfix, @nextdoor, @glassdoor, @hacker0x01, @solvhealth, @instawork Trustee @sfiscience, Podcast: @bg2pod
Xingyao Wang @xingyaow_
6K Followers 1K Following Co-founder @allhands_ai, building OpenHands | PhD candidate @IllinoisCDS | BS @UMichCSE ('22) | Ex Intern @GoogleAI @Microsoft | Opinions are my own
Ben Firshman @bfirsh
8K Followers 1K Following @replicate / https://t.co/P3bIJyMtkU / https://t.co/vt6LozqLGX / previously: @docker, @guardian, @govuk
Ji Lin @jilin_14
6K Followers 947 Following Research @Meta Superintelligence Lab | Prev: Research @OpenAI; PhD @MIT
soham @SohamGovande
3K Followers 825 Following mlsys @openai | stanford cs | prev @nvidia @hazyresearch
Lisa Dunlap @lisabdunlap
1K Followers 280 Following messin around with model evals @berkeley_ai | prev @lmarena_ai
Zhihao Jia @JiaZhihao
3K Followers 690 Following Assistant professor of Computer Science at Carnegie Mellon University. Research on systems and machine learning.
Dacheng Li @DachengLi177
1K Followers 781 Following Student Lead @NovaSkyAI, PhD @BerkeleySky, @lmsysorg, @lmarena_ai | Prev: @Nvidia @Google @SCSatCMU | @istoica05 @profjoeyg @songhan_mit @haozhangml @ericxing
Alex Krentsel @AlexKrentsel
253 Followers 181 Following Senior Sys Research Eng @ Google, Systems PhD Student @Berkeley_EECS. Prev. @YouTube, @Meta, Lecturer @HowardU read my writing: https://t.co/mxRccOCFDp
Shiyi Cao @shiyi_c98
1K Followers 605 Following Student Lead @NovaSkyAI, PhD student @UCBerkeley @BerkeleySky, MSc @ETH, B.S @sjtu1896, llm and system
Dylan Patel @dylan522p
96K Followers 945 Following SemiAnalysis Boutique AI & Semiconductor Research and Consulting DMs are open for consulting, quotes, or to talk shopTrends for United States
You might like
