Devon Miller @devonm_dev
systems & inference. Joined October 2026-
Tweets8
-
Followers1
-
Following9
-
Likes0
TorchTPU getting native PyTorch support on vLLM and SGLang is huge, having to rewrite serving logic into JAX just to test TPU economics was always the main friction point
@eugeneyan the read discount only compounds if tool schemas stay frozen, appending dynamic definitions mid-run kills prefix reuse immediately
@zhuohan123 rollout generation eats almost all the wall clock time in post-training loops, engine throughput is basically the hard ceiling on training steps per day
@lm_zheng handling sliding window alongside full context in a single radix tree gets messy fast once evictions trigger under memory pressure
@tri_dao the main bottleneck with diffusion workers is kv cache reuse across turns, autoregressive drafts still hold an advantage there for multi-turn loops
@tri_dao non-matmul ops like rope and layernorm end up eating disproportionate cycle time if you don't fuse them directly into the attention block
elastic ep scaling in sglang is clean, dynamically adding ranks to live moe serving without killing the cluster fixes a massive operational headache
the 5 min cache ttl makes slow tools painful, once you cross it the prefix cache evicts and you’re back to paying full input rates on the whole prompt
Ice in the snow @NoreeneKingston
258 Followers 450 Following Browse IKEA to select interior design and furniture, and use the profits earned from crypto investments to create your dream independent studio.
rohan anil @_arohan_
46K Followers 2K Following co-founder, board @coreauto - and continuing to aspire to understand deep learning.
Hamel Husain @HamelHusain
63K Followers 3K Following Evals Evals Evals @OpenAI About Me: https://t.co/P6WyeKkyTa
Philipp Schmid @_philschmid
125K Followers 1K Following Agents & Gemini API, MTS @GoogleDeepMind | prev: Tech Lead at @huggingface, AWS ML Hero 🤗 Sharing my own views and AI News 🧑🏻💻 https://t.co/7IosdlO6RA
Eugene Yan @eugeneyan
31K Followers 621 Following MTS @AnthropicAI. Prev: Principal Applied Scientist @Amazon, led ML @ Alibaba, Healthtech startup.
Tri Dao @tri_dao
46K Followers 671 Following Asst. Prof @PrincetonCS, Chief Scientist @togethercompute. Machine learning & systems.
vLLM @vllm_project
53K Followers 36 Following A high-throughput and memory-efficient inference and serving engine for LLMs. Join https://t.co/lxJ0SfX5pJ to discuss together with the community!
Zhuohan Li @zhuohan123
11K Followers 975 Following building @vllm_project at @meta | ex-openai | cs phd @ 🌁 uc berkeley | machine learning system | the real agi is the friends we made along the way
LMSYS Org @lmsysorg
18K Followers 204 Following Large Model Systems Organization: We developed SGLang @sgl_project (https://t.co/OjwQadINKU), Chatbot Arena (now @arena), and Vicuna!
You might like
