SGLang @sgl_project
SGLang project https://t.co/0zdxDWA5qn This is an alias account for SGLang, please follow @lmsysorg Palo Alto Joined May 2025-
Tweets13
-
Followers280
-
Following15
-
Likes15
What an incredible journey
⚡️ Big update from Kimi K2! 256k context, Stronger coding & tool-calling, Smoother agent integration. Already tested with SGLang runtime — stable 60-100+ TPS with turbo API! 👉 Check it out: huggingface.co/moonshotai/Kim…
⚡️ Big update from Kimi K2! 256k context, Stronger coding & tool-calling, Smoother agent integration. Already tested with SGLang runtime — stable 60-100+ TPS with turbo API! 👉 Check it out: huggingface.co/moonshotai/Kim…
🚀Summer Fest Day 3: Cost-Effective MoE Inference on CPU from Intel PyTorch team Deploying 671B DeepSeek R1 with zero GPUs? SGLang now supports high-performance CPU-only inference on Intel Xeon 6—enabling billion-scale MoE models like DeepSeek to run on commodity CPU servers.…
🚨SGLang Summer Fest Bonus Drop🚨 Proud to share a joint effort from Mooncake by @Kimi_Moonshot, @Oracle , and SGLang: Kimi K2 trillion-scale deployment—running on 128 H200 GPUs sponsored by @NVIDIAAIDev DGX Cloud. OME + SGLang = MoE inference at production scale.👇
Proud to support this lightning-fast launch⚡️ ️ Accelerated through #NVIDIADGX Cloud and in partnership with Moonshot AI, @SGlang, and @Oracle Open Model Engine, we helped bring Kimi K2 to customers just days after its debut. Now, organizations can “Think Smart” and scale MoE…
Proud to support this lightning-fast launch⚡️ ️ Accelerated through #NVIDIADGX Cloud and in partnership with Moonshot AI, @SGlang, and @Oracle Open Model Engine, we helped bring Kimi K2 to customers just days after its debut. Now, organizations can “Think Smart” and scale MoE…
🚀 Introducing SpecForge – our open-source framework for speculative decoding training, built for SGLang and Eagle3. Train draft models that just work—scalable, efficient, and inference-ready. Supports LLaMA 4, DeepSeek, MoE, FSDP, TP & more. Up to 2.18× speedup. Huge thanks to…
.@lmsysorg (SGLang) now achieves 7,583 tokens per second per GPU running @deepseek_ai R1 on the GB200 NVL72, a 2.7x leap over H100. We're excited to see the open source ecosystem advance inference optimizations on GB200 NVL72, driving down cost per token for the industry at…
The SGLang team just ran DeepSeek 671B on NVIDIA’s GB200 NVL72, unlocking 7,583 toks/sec/GPU for decoding w/ PD disaggregation + large-scale expert parallelism — 2.7× faster than H100. Don’t miss this work! 🔥 Thanks to Pen Li from NVIDIA who kicked off this collaboration and…
We're excited to release OME, which is a Kubernetes operator for enterprise-grade management and serving of Large Language Models (LLMs). It optimizes the deployment and operation of LLMs by automating model management, intelligent runtime selection, efficient resource…
We @a16z just launched the third batch of Open Source AI Grants (cc @mbornstein) 🎉 This round includes projects focused on LLM evaluation, novel reasoning tests, infrastructure, and experimental research at the edge of capability and cognition: • SGLang: High-performance LLM…
🚀 Breaking: SGLang provides the first open-source implementation to serve @deepseek_ai V3/R1 models with large-scale expert parallelism and prefill-decode disaggregation on 96 GPUs. It nearly matches the throughput reported by the official DeepSeek blog, achieving 52.3K input…
The @lmsysorg team in charge of SGLang, a pioneering LLM inference framework, teamed up with Nebius AI Cloud to supercharge @deepseek_ai R1’s performance for real-world use. Read the full story: nebius.com/customer-stori… 🔹 Goal: Maximize inference throughput for DeepSeek models…
HL hl @kuchiioct25
23 Followers 2K Following
Jack Chen @twNumbaOne
3 Followers 42 Following
Banghua Zhu @BanghuaZ
7K Followers 1K Following Assistant Professor @UW, Principal Research Scientist @Nvidia. Prior Cofounder @NexusflowX, @Berkeley_EECS @Google @Microsoft. I work on LLMs.
ns @nsautopool
0 Followers 76 Following
NadiaJiang @jiangbonadia
774 Followers 355 Following 🥑 Growth Hacker at @Ant_OSS | Board Chair of KAIYUANSHE | Builder of @ZenMuxAI @AnswerDev | Organizer of ApacheCon, OpenInfra Days, Dev.Together and KCD
swh @swhsiang
51 Followers 148 Following building humanoid | prev ML @CashApp Infra @salesforce Purdue ECE
zetxqx @txqxze
17 Followers 326 Following
Wirxel @Wirxel768
19 Followers 527 Following Success isn’t about being the best. It’s about always getting better.
Fatih⏩⤴️ @taskinfatih
675 Followers 6K Following Lover of all novel and hard concepts: especially machine learning and systems theory
James Fang @polymath_james
186 Followers 486 Following 21 | ML & Fullstack Engineer for Chip Design | UIUC CS '23, Master's '25 | Prev built https://t.co/eZBcXM8F7r , https://t.co/LsxfKgYYxt
Crynux @crynuxio
3K Followers 232 Following k8s on EdgeAI truly permissionless and trustless #DecentralizedAI #DeAI Learn more: https://t.co/EoxbxCvXTR
Vikram @msharmavikram
2K Followers 593 Following @NVIDIA Sr. Research Scientist | UIUC PhD All opinions and tweets are personal. Tweets about AI Inference, CUDA and GPU systems.
杨云皓 @yngynho4581191
0 Followers 70 Following
ggaowp @ggaowp
138 Followers 756 Following builder and founder, ex-Sui/Libra/Instagram/HRT, UT Austin
emerdenny @emerdenny
179 Followers 3K Following you better start swimmin’ or you’ll sink like a stone
Ethan (Yusheng) Su @thu_yushengsu
700 Followers 1K Following Researcher @AMD | Postdoc @ https://t.co/eNDjysZ4dD | Ph.D @TsinghuaNLP | Intern @Microsoft.
P @IiHq7kK57ZxosbT
90 Followers 2K Following
Zeyu @ZeyuChen_
5 Followers 99 Following
aastha @being__aastha
45 Followers 1K Following (learning) machine learning @ amazon. do #NLProc & #AI!
Rishi Khare @rishiskhare
152 Followers 307 Following Efficient reasoning research @GeorgiaTech @Berkeley_AI
Michael Stewart @mps_vc
446 Followers 826 Following Scientist-engineer turned VC at @M12vc: Generative AI, gaming, semiconductors, materials science, and all things deep tech. h-index: 31
Quant Hustle @QuantHustle
0 Followers 4K Following
Ying Sheng @ying11231
12K Followers 731 Following @lmsysorg @sgl_project | Prev. @xAI @Stanford | Assist Prof @UCLA. (delayed) | Do it anyway | Live to fight another day
caoye @caoye42
124 Followers 1K Following hacker and builder prev product dev @ThirdAILab recent grad @RiceUniversity
Inference Startup in ... @BrrrLLM_Stealth
6 Followers 12 Following Next-Gen LLM Inference Fabric. Up-to 25X better 🦀
Codex @Codex10030030
0 Followers 29 Following
John Holmes @jsholmes
341 Followers 2K Following optimist; e/acc; vibe coding; dark matter skeptic; possible pantheist; movies, books, games, hikes, friends, food, D&D, and fatherhood. Architect @weHRTyou
Alex @alexqdh
11 Followers 111 Following
Losiki @LosikiS
0 Followers 31 Following
ZimaBlue @ZimaBlueeeeee
3 Followers 77 Following
Tilak Marupilla @tilaknarasimha
116 Followers 2K Following Explorer | Embedded Systems | Signal processing | Deep Learning (just really trying to learn)
1iin @liin1211
531 Followers 460 Following
Jin D @JinD332
1 Followers 66 Following
Dalton_lovegood @dalton_lovegood
54 Followers 465 Following Venture Capitalist @Monolith; Research Associate @Cambridge_Uni; Alumni @CUHKofficial; Focus on #AI #MachineLearning #Cloud M-A-P @mm_art_project
Obi @_omahdi
5 Followers 974 Following
Abhishek @abhishek538d
18 Followers 699 Following
le fromage @lefromage123
108 Followers 3K Following
Shikamaru Nara @aravindgohan24
4 Followers 215 Following
Adarsh @adarshxs
1K Followers 2K Following 20 | Founder @tensoic | multimodal team @ SGLang(@lmsysorg) | Prev: @iiscbangalore
Ligeng Zhu @LigengZhu
2K Followers 2K Following Research Scientist at @Nvidia exploring efficient training , previously @MIT, @SFU and @ZJU_China.
Banghua Zhu @BanghuaZ
7K Followers 1K Following Assistant Professor @UW, Principal Research Scientist @Nvidia. Prior Cofounder @NexusflowX, @Berkeley_EECS @Google @Microsoft. I work on LLMs.
Beidi Chen @BeidiChen
15K Followers 400 Following Asst. Prof @CarnegieMellon, @amazon Scholar, Prev: Visiting Researcher @Meta, Postdoc @Stanford, Ph.D. @RiceUniversity, Large-Scale ML, a fan of Dota2.
Ji Lin @jilin_14
6K Followers 947 Following Research @Meta Superintelligence Lab | Prev: Research @OpenAI; PhD @MIT
Zhijian Liu @zhijianliu_
2K Followers 824 Following Research Scientist @NVIDIA. Assistant Professor @UCSanDiego. PhD @MIT. Efficient AI. Views are my own.
Ryan Hanrui Wang @hanrui_w
2K Followers 748 Following Building @Eigen_AI_Labs | AEI = Artificial Efficient Intelligence | PhD @MIT | Ex-@NVIDIA @XilinxInc | Efficient AI, Computer Architecture.
Song Han @songhan_mit
9K Followers 171 Following
NVIDIA AI Developer @NVIDIAAIDev
83K Followers 324 Following All things AI for developers from @NVIDIA. Additional developer channels: @NVIDIADeveloper, @NVIDIAHPCDev, and @NVIDIAGameDev.
DeepSeek @deepseek_ai
972K Followers 0 Following Unravel the mystery of AGI with curiosity. Answer the essential question with long-termism.
Lianmin Zheng @lm_zheng
14K Followers 618 Following Member of technical staff @xAI | Prev: Ph.D. @UCBerkeley, Co-founder @lmsysorg
Ying Sheng @ying11231
12K Followers 731 Following @lmsysorg @sgl_project | Prev. @xAI @Stanford | Assist Prof @UCLA. (delayed) | Do it anyway | Live to fight another day
zhyncs @zhyncs42
3K Followers 538 Following 🌁 OPINIONS ARE MY OWN, Homepage https://t.co/saCowtppUm, Just for fun @lmsysorg SGLang, Prev @basetenco @meituan @Baidu_Inc
LMSYS Org @lmsysorg
8K Followers 180 Following Large Model Systems Organization: Join our Slack: https://t.co/mSPNyKTLTS We developed SGLang https://t.co/jEqIJcGwGA, Chatbot Arena (now @lmarena_ai), and Vicuna!Trends for United States
You might like
