Hunter Carlisle @data_hunt

x.com/i/article/2111…

We’re proud to support the SGLang Summit as a Platinum Sponsor. We look forward to sharing updates from RadixArk at the summit and connecting with the @sgl_project community. Join us at Fort Mason in San Francisco, November 12–13!

SGLang @sgl_project

2 weeks ago

Whatever AI infra means to you, we’ve got it covered. See for yourself: the SGLang Summit agenda is live → sglang.io/summit 🗓️ Nov 12–13 · 📍 Fort Mason, SF 🎟️ Early bird ends Oct 20 Two days of open models, AI hardware, recursive self-improvement, agents, robotics, and

2 17 70 20K 14

Qwen3.8 Flash Next on Vera Rubin with NVIDIA Dynamo + SGLang: Up to 6.6× throughput per chip and up to 76% lower modeled cost per total token vs. GB300, at matched per-user speed. Thanks @NVIDIAAI for the partnership and early access, and @SemiAnalysis_ for the results!

SemiAnalysis @SemiAnalysis_

a week ago

ALERT🚨: NVIDIA SGLANG INFERENCE RUBIN IS 4.23x BETTER 🚀 PERF PER DOLLAR THAN EVEN GB300 NVL72. This is on the widely used production LLM engine called @sgl_project. (1/2)🧵

30 27 377 47K 84

ALERT🚨: NVIDIA SGLANG INFERENCE RUBIN IS 4.23x BETTER 🚀 PERF PER DOLLAR THAN EVEN GB300 NVL72. This is on the widely used production LLM engine called @sgl_project. (1/2)🧵

A cool application of score centering: running RL on heterogeneous hardware It uses the Miles implementation of score centering, which supports top-p sampling. Applying score centering to top-p sampling is a generalization of the same "mask replay" correction that Cognition and DeepSeek use for top-p training (under zero TIM, these corrections are equivalent). Another benefit is that under top-p sampling, it is feasible to run exact score centering (which is infeasible with full sampling).

Modal @modal

a week ago

Training large models with RL typically runs on a fixed homogeneous cluster of the latest accelerators for numerical stability and faster throughput. This throws away inference optimization work done for other vendors and generations, limits the capacity you can utilize, and is

32 129 2K 164K 2K

SGLang brings optimized inference to @nvidia Vera Rubin, with performance gains across attention, MoE, and speculative verification for Kimi K3 NVFP4. Miles by @radixark takes this further with end-to-end RL training, using SGLang for rollouts and the Vera CPU for concurrent agent sandboxes. The teams share their early results, benchmarks, and the engineering behind them 👇

SGLang @sgl_project

2 weeks ago

We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference. Working closely with @nvidia, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware. Highlights: • Up to 20% faster FP8 MLA at batch 1 / 128K context • 20%

18 26 167 36K 54

Miles runs RL end to end on NVIDIA Vera Rubin, out of the box: SGLang rollouts, Megatron training, one container image. And agentic RL with 64 concurrent sandboxes on the Vera CPU, right next to the GPUs. Thanks to @NVIDIAAI for the early access. Details in the blog 👇

SGLang @sgl_project

2 weeks ago

We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference. Working closely with @nvidia, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware. Highlights: • Up to 20% faster FP8 MLA at batch 1 / 128K context • 20%

18 26 167 36K 54

The agenda has been updated on the website. Is there anything you want to hear that is still missing?

SGLang @sgl_project

2 weeks ago

Whatever AI infra means to you, we’ve got it covered. See for yourself: the SGLang Summit agenda is live → sglang.io/summit 🗓️ Nov 12–13 · 📍 Fort Mason, SF 🎟️ Early bird ends Oct 20 Two days of open models, AI hardware, recursive self-improvement, agents, robotics, and

2 17 70 20K 14

@sgl_project @nvidia 👏👏👏

We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference. Working closely with @NVIDIA, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware. Highlights: • Up to 20% faster FP8 MLA at batch 1 / 128K context • 20% faster KDA verification, with bitwise-identical output • 5.9% end-to-end inference speedup from MoE tail fusion, removing 276 kernel launches per decode step SGLang also powers rollouts for Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU. Full results and engineering details 👉 lmsys.org/blog/2026-10-0…

Excited to share our work on large-scale sharded weight transfer! We’ve implemented a native sharded weight transfer engine in vLLM and SkyRL using Ray Direct Transfer (RDT) + NIXL. We’re able to achieve weight transfer for Kimi K2 (1T params) in BF16 in 7.53 seconds on 48 8xH100 nodes. Blog: vllm.ai/blog/2026-08-2…

Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models. RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale. Over the past 9 months, 72 contributors have landed 1,326 commits, 85 GPU E2E CI tests, battle-testing Miles on frontier open models like Kimi K3, DeepSeek V4, Qwen 3.8, GLM 5.2, Inkling, MiniMax H3, etc. Miles powers frontier-model development and production RL workloads at @humansand, @periodiclabs, @modal, @DecagonAI, @Eigent_AI, @nebiusai, @IBM and more, on both @NVIDIAAI and @AIatAMD hardware. Here is what we built, and why teams picked Miles🧵

SGLang is Miles' native rollout engine, paired with Megatron-LM and FSDP on the training side. Miles delivers accurate, efficient, and reliable RL at frontier scale for both LLMs and diffusion models. We are thrilled to see so many teams building with the community and putting Miles to work in real production workloads! Go check out what they built, learned, and verified for this release 👇

RadixArk @radixark

2 months ago

Today we're launching Miles v0.1, an open-source RL framework for LLMs and multimodal models. RL training is easy to start and hard to debug. Miles helps you ensure your run is correct, use hardware efficiently, and keep RL running at scale. Over the past 9 months, 72

59 149 995 669K 595

@AIatMeta's Muse Glimmer (30B dense, open-weights) launches with SGLang day-0 support. We got ~230 tok/s on a single RTX 5090, with NVFP4 + DFlash on. It also works out of the box on @NVIDIAAIDev RTX Pro 6000, DGX Spark, and MLX for Mac. Speed and reliability have always been the blockers for a local agent. SGLang and Muse Glimmer make a local agent that's blazing fast, always on, and gets things done. Go build and run a local agent on your own machine!🚀

AI at Meta @AIatMeta

2 months ago

Introducing Muse Glimmer, an open-weight 30B-parameter model optimized for local, always-on agent workflows. Muse Glimmer delivers strong performance on key agentic use cases and benchmarks compared with leading models in its size category, and is designed to run entirely on

keyboard_arrow_left Previous keyboard_arrow_right Next

377 1K 8K 1.4M 3K

While SGLang provided Day-0 support for DeepSeek-V4, the collaboration between the @lmsysorg and @NVIDIAAI engineering teams has taken its production performance to the next level. According to the public SemiAnalysis InferenceX dashboard, the GB300 disaggregated lane (DeepSeek-V4 Pro, FP4, 8K/1K) saw a 5x throughput increase—surging from ~2,200 to ~11,200 tok/s/GPU at identical interactivity levels. These updates sustain high throughput much deeper into target interactivity ranges most deployments target, while also driving a 2.9x lift on the Blackwell Ultra aggregated lane. Find the full technical breakdown in the comments below:

Every single product that involves text needs a feature to copy/paste/export/import in markdown to preserve formatting when passing ad-hoc things to/from AI agents, ASAP. I'm looking at you, @slack/@gmail/@Office. Good job being ahead of the curve, @Quip. Gold star for you⭐️