vLLM @vllm_project
A high-throughput and memory-efficient inference and serving engine for LLMs. Join https://t.co/lxJ0SfX5pJ to discuss together with the community! github.com/vllm-project/v… Joined March 2024-
Tweets527
-
Followers18K
-
Following20
-
Likes260
Missed our latest vLLM office hours? We covered hybrid models as first-class citizens in @vllm_project. ✅ Hybrid model support in v1 ✅ Mamba, Mamba2, linear attention ✅ Performance from v0 → v1 ▶️ Recording: youtube.com/live/uWQ489ONv… 📑 Slides: docs.google.com/presentation/d…
[1/N]🚀New decoding paradigm drop!🚀 Introducing Lookahead Reasoning(LR): step-level speculation that stacks with Speculative Decoding(SD). It has been accepted to #NeurIPS2025 🎉 📖 Blog: hao-ai-lab.github.io/blogs/lookahea… 💻 Code: github.com/hao-ai-lab/Loo… 📄 Paper: arxiv.org/abs/2506.19830
Pro Tip💡Fast and simple way to deploy DeepSeek-V3.1-Terminus with vLLM ⚡️ Run it with: vllm serve deepseek-ai/DeepSeek-V3.1-Terminus -tp 8 -dcp 8 (as simple as appending -dcp 8 after -tp 8) Thanks to the @Kimi_Moonshot team, vLLM 0.10.2 adds Decode Context Parallel (DCP)…
Pro Tip💡Fast and simple way to deploy DeepSeek-V3.1-Terminus with vLLM ⚡️ Run it with: vllm serve deepseek-ai/DeepSeek-V3.1-Terminus -tp 8 -dcp 8 (as simple as appending -dcp 8 after -tp 8) Thanks to the @Kimi_Moonshot team, vLLM 0.10.2 adds Decode Context Parallel (DCP)… https://t.co/NjuJTpC1xR
Day-0 support on one of the most anticipated model releases🚀🚀🚀 Detailed deployment guide coming soon at vllm-recipes docs.vllm.ai/projects/recip…
Day-0 support on one of the most anticipated model releases🚀🚀🚀 Detailed deployment guide coming soon at vllm-recipes docs.vllm.ai/projects/recip… https://t.co/dwXiEVZHEM
Just enabled full cudagraphs by default on @vllm_project! This change should offer a huge improvement for low latency workloads on small models and efficient MoEs For Qwen3-30B-A3B-FP8 on H100 at bs=10 1024/128, I was able to see a speedup of 47% 🔥
The thinker model is already available on vLLM via a pre-built wheel but we’re working together with Qwen team to bring this to the upstream main branch - Stay tuned!
The thinker model is already available on vLLM via a pre-built wheel but we’re working together with Qwen team to bring this to the upstream main branch - Stay tuned!
Congrats to @deepseek_ai ! DeepSeek-R1 was published in Nature yesterday as the cover article, and vLLM is proud to have supported its RL training and inference🥰
v0.10.2 marks the first release with official aarch64 support for vLLM! You can now install vLLM directly onto @nvidia 's GB200. Along with the PyPI release, our docker image is also multi-platform so pulling the right image just works. More perf enhancements on the way!
Prefix cache-aware routing is now available in Ray 2.49 🚀 Scaling input token-heavy workloads (like multi-turn convos & agent loops) requires maintaining prefix cache hit rate across 100s of vLLM engine replicas, and PrefixCacheAffinityRouter makes it easy. Here’s how it…
The new open-source Qwen3-Next Instruct and Thinking models put state-of-the-art long-context reasoning into the hands of everyone. We collaborated with #opensource frameworks from SGLang (@lmsysorg) and @vllm_project to enable communities to deploy Qwen3-Next across the…
Disaggregated Inference at Scale with #PyTorch & #vLLM: Meta’s vLLM disagg implementation improves inference efficiency in latency & throughput vs its internal stack, with optimizations now being upstreamed to the vLLM community. 🔗 hubs.la/Q03J87tS0
At Thinking Machines, our work includes collaborating with the broader research community. Today we are excited to share that we are building a vLLM team at @thinkymachines to advance open-source vLLM and serve frontier models. If you are interested, please DM me or @barret_zoph!…
Welcome Qwen3-Next! You can run it efficiently on vLLM with accelerated kernels and native memory management for hybrid models. blog.vllm.ai/2025/09/11/qwe…
Welcome Qwen3-Next! You can run it efficiently on vLLM with accelerated kernels and native memory management for hybrid models. blog.vllm.ai/2025/09/11/qwe… https://t.co/M19eWoXAj2
🚀 Introducing Qwen3-Next-80B-A3B — the FUTURE of efficient LLMs is here! 🔹 80B params, but only 3B activated per token → 10x cheaper training, 10x faster inference than Qwen3-32B.(esp. @ 32K+ context!) 🔹Hybrid Architecture: Gated DeltaNet + Gated Attention → best of speed &…
Deep dive into optimizing weight transfer step by step and improving it 60x!
Deep dive into optimizing weight transfer step by step and improving it 60x!
Thank you @cHHillee for the great explanation and demo of how to implement deterministic inference on vLLM!!! github.com/vllm-project/v…
Thank you @cHHillee for the great explanation and demo of how to implement deterministic inference on vLLM!!! github.com/vllm-project/v…
⚡️ Efficient weight updates for RL at trillion-parameter scale 💡 Best practice from Kimi @Kimi_Moonshot vLLM is proud to collaborate with checkpoint-engine: • Broadcast weight sync for 1T params in ~20s across 1000s of GPUs • Dynamic P2P updates for elastic clusters •…
⚡️ Efficient weight updates for RL at trillion-parameter scale 💡 Best practice from Kimi @Kimi_Moonshot vLLM is proud to collaborate with checkpoint-engine: • Broadcast weight sync for 1T params in ~20s across 1000s of GPUs • Dynamic P2P updates for elastic clusters •…
Wow, thanks to @charles_irl , you can understand internals of vLLM with a live notebook from @modal 🥰
Wow, thanks to @charles_irl , you can understand internals of vLLM with a live notebook from @modal 🥰
The amazing blogpost from @gordic_aleksa is alive at vLLM's blogpost blog.vllm.ai/2025/09/05/ana… (after more proofreading and clarifications)! Looking forward to future series of tech deep dive blogposts😍
The amazing blogpost from @gordic_aleksa is alive at vLLM's blogpost blog.vllm.ai/2025/09/05/ana… (after more proofreading and clarifications)! Looking forward to future series of tech deep dive blogposts😍
Guoteng Wang @GuotengWang
1 Followers 99 Following
Miller H @millerhmillerh
0 Followers 1K Following
سلطان النعي... @alnuaimi_Sultan
0 Followers 3 Following
Taher A. Bahashwan @taher9990
704 Followers 4K Following Cloud | Networks | Security | Expert & Architect @Solutions_BySTC
Bhargav Velpula @BhargavVelpula
9 Followers 74 Following
chisel scala @ChiselScala
9 Followers 57 Following
Mukhtar O. Raji @eaverine1
48 Followers 348 Following Artificial Intelligence Engineer...and everything python
Tom Dunn @Dunn_and_Dunner
122 Followers 4K Following
幣圈總司令 @snskkznxbwksndm
85 Followers 987 Following
Lucas Wilkinson @wilkinsonlucas3
1 Followers 7 Following
johnson @johnson429279
0 Followers 283 Following
Rakesh Patel @rakesh657
171 Followers 3K Following
Jennifer Mulready @jennlelm
0 Followers 30 Following
나주 @sysnaaa111777
8 Followers 38 Following
shimazu tomoki @shimazu_tomoki
71 Followers 784 Following
Aviv rovshitz @rovshitz
10 Followers 515 Following The perfect drink: Sports, Mathematics, Programming, Music, Philosophy with two ice cubes ⚽📊
jalused @jalusedx
1 Followers 8 Following
Barry @barryxv6
2 Followers 9 Following
MyBizAI @MyBizAI
2 Followers 176 Following
yuntao peng @pyt80x86
0 Followers 10 Following
Jason Cui @JasonSCui
3K Followers 1K Following Partner @a16z investing in infra & AI | prev product @databricks & @uber, founder @jemihq (YC S20, acq). Technology optimist ☀️
renyu @renyu651482
1 Followers 52 Following
Chris Klaus @cklaus1
3K Followers 5K Following CEO of Fusen. Connecting students with mentors, investors, and funding opportunities through our Fusen accelerators. @cklaus.bsky.social on Bluesky.
Bo Qin @qboster
7 Followers 310 Following
Rafael Fonseca @rafaelf0nseca
376 Followers 614 Following computer engineering @uwaterloo | building sidekick @shopify
ΛMΛПΛM @AmanamIsrael
891 Followers 2K Following Software Engineer 🍚 | LLMS are awesome 🧠 | I love music
AI StackControl @StackControl
0 Followers 24 Following
Jesús Barrio @techjb
300 Followers 121 Following Haciendo de los idealistas irrelevantes. https://t.co/VGO34uKj3m https://t.co/g1UougdIDd
Marius @codingstube
41 Followers 518 Following
Shengyuan @ShengyuanS
516 Followers 886 Following Tech lover. Staff of @Kimi_Moonshot. Opinions are my own.
Carlos Garcia @madebybeto
21 Followers 276 Following
Yazo @Yazano
175 Followers 624 Following
feiwang1984 @feiwang19876188
15 Followers 824 Following
Albert Villanova @avillanovamoral
2K Followers 5K Following ML Engineer @huggingface. Data Scientist, PhD Theoretical Particle Physics, BSc Computer Science. Always learning. he/him
Davide Laverga @DavideGrowth8
1 Followers 38 Following
DeepPDF @deeppdf_ai
9 Followers 16 Following AI-Powered Deep Learning for PDFs with Chat and Translate.
StepFun @StepFun_ai
3K Followers 121 Following Scale-up possibilities for everyone. HuggingFace: https://t.co/Soz5ZuwCrd
ManusAI @ManusAI_HQ
204K Followers 26 Following Manus is the general AI agent that bridges minds and actions: it doesn't just think, it delivers results. Download our app: https://t.co/XSfjRhjdgo
Kimi.ai @Kimi_Moonshot
52K Followers 100 Following Built by Moonshot AI to empower everyone to be superhuman.
PyTorch @PyTorch
453K Followers 77 Following Tensors and neural networks in Python with strong hardware acceleration. PyTorch is an open source project at the Linux Foundation. #PyTorchFoundation
Ion Stoica @istoica05
5K Followers 20 Following Professor at UC Berkeley, co-founder of Databricks, Anyscale, LMArena, Conviva.
SangBin Cho @Saaaang94
3K Followers 483 Following reasoning @xAI | prev-founding engineer @anyscalecompute | senior committer of @raydistributed | committer @vllm_project Sglang | Github: rkooo567
Michael Goin @mgoin_
1K Followers 386 Following inference optimization @RedHat_AI building @vllm_project | you can call me misha
Roger Wang @rogerw0108
480 Followers 183 Following Flowers and friendship | ML Platform & Infra @Roblox | Committer @vllm_project | @uwaterloo @uwcse
Nick Hill @nickhill33
63 Followers 3 Following
Philipp Moritz @pcmoritz
534 Followers 5 Following Co-founder and CTO at @anyscalecompute. Co-creator of @raydistributed. Interested in ML, AI, computing.
Kaichao You @KaichaoYou
4K Followers 134 Following phd student in tsinghua university, working on @vllm_project
Lily Liu @eqhylxx
1K Followers 435 Following CS PhD Student, Sky Lab @UCBerkeley, @vllm_project, @OpenAI
Woosuk Kwon @woosuk_k
6K Followers 626 Following @thinkymachines | @vllm_project | PhD-ing @Berkeley_EECS
Zhuohan Li @zhuohan123
9K Followers 866 Following mts @ openai | cs phd @ 🌁 uc berkeley | building @vllm_project | machine learning system | the real agi is the friends we made along the wayTrends for United States
You might like
