Ryan @RyanRamirezSI

If anyone is looking to have models trained/evals made for them and dont want to pay an ML engineer/ looking to convert to local AI i'm looking for contract work! huggingface.co/DJLougen huggingface.co/GestaltLabs

this is deepseek v4.1 flash, the fastest vllm build on 2x dgx sparks, and the weights are public now 🐋 i quantized them myself from deepseek's official checkpoint, every one of the 15,360 routed experts sits at 2 to 5 bits by how sensitive it measured, 2.70 bits on average, and the whole folder is 196 gb with the two engram shards pulled straight from deepseek's own repo and a sha256 manifest that checks every file on my two sparks it serves 141.3 tok/s on structured output, 103.0 tok/s on code and 58.3 tok/s on prose, and with the drafter off it still holds 44.5 tok/s on every prompt the speed would mean nothing if the model got dumber, so i scored it against native deepseek, kl divergence 0.0243, 18% lower than the 2.9 bit pack the next best builds run while mine is smaller, top-1 agreement 95.57%, closer to native on 7 of 8 domains, and humaneval+, mbpp+ and ifeval all land within noise of the original the full 262k window is served, a 255,709 token prompt found all 3 needles and decode still ran at 76 tok/s after it, and a second request on the same prefix starts in 1.34 s you need two sparks on a connectx-7 cable and about 430 gb free on each, the serve image builds from my recipe in one command and a read-only check script tells you pass it's mit like the original, pull it and check my numbers huggingface.co/sudoingX/DeepS…

@AnthropicAI I miss being able to see Claude’s thinking as he arrives at answer. 🥲

Crazy upgrade. 100+ tokens/sec on 2 DGX Sparks. 🤯 If you have two sparks, I implore you check this out.

Have this downloading as we speak. If these numbers are consistent (and will likely improve further), it makes it hard to justify picking up the M5U 512g in the next few weeks. DGX Sparks are aging like fine wine.

Sudo su @sudoingX

a day ago

holy shit, i actually did it. deepseek v4.1 flash is flying on 2x dgx spark, serving with vllm 🐋 > structured output: 141.3 tok/s > code: 103.0 tok/s > prose: 58.3 tok/s this is the fastest vllm build of deepseek v4.1 flash on 2x dgx sparks, 2.5x the best vllm kit before mine,

33 28 326 18K 190

@sudoingX @nvidiaAI @nvidia this would be a drop in the bucket (for you) to continue SudoingX’s work. Please make this happen and support him with some hardware! 💚

I mapped out 48 months of intense Silicon Valley warfare to show you exactly how fast the 'smartest AI on earth' gets dethroned.

We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case). I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves. We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them: "/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started" I imagine we'll have some backend services that could be replaced entirely by this new architecture.

keyboard_arrow_left Previous keyboard_arrow_right Next

@finkd zucc.si

@RyanRamirezSI these two boxes have more in them than people think, four days ago the build i started on did 39 tok/s on this model

@sudoingX Wow. The community keeps pushing the DGX Sparks to unbelievable heights. Thank you so much for this Sudo. I hope you have a great day!

the recipe and the weights are public now, run it yourself on two sparks, the serve image builds from the recipe in one command > weights: huggingface.co/sudoingx/DeepS… > recipe: github.com/sudoingX/dgx-s… the weights are 196 gb, the two engram shards come straight from deepseek's own repo, and a sha256 manifest checks the whole folder you need 2x dgx sparks on a connectx-7 cable and about 430 gb free on each, a read-only check script gives pass, warn or fail per check on both boxes and on the live serve the raw speed, kld and eval files ship with it, the calibration set i quantized with too, check my numbers on your own boxes

holy shit, i actually did it. deepseek v4.1 flash is flying on 2x dgx spark, serving with vllm 🐋 > structured output: 141.3 tok/s > code: 103.0 tok/s > prose: 58.3 tok/s this is the fastest vllm build of deepseek v4.1 flash on 2x dgx sparks, 2.5x the best vllm kit before mine, and on code and structured output it's ahead of tensorfold v0.6.0 on my sparks full context, served at 262k tokens per request on the two sparks > needles found 3 of 3 at 8k, 32k, 64k, 128k and 256k, the last one a 255,709-token prompt > decode after the prompt 79, 82, 75, 80 and 76 tok/s, it barely moves > prefill 1,395 to 1,475 tok/s, my weak side for now > 9 gib free on each node after load i also checked what the speed costs in quality, my own weights quantized from the official checkpoint and scored against native deepseek > smaller than the pack the next best build runs, 2.70 vs 2.9 bits on the experts, and still closer to native > kl divergence 0.0243 from native, 18% lower than that pack > closer to native on 7 of 8 domains, code, agentic, math, long, knowledge, writing and chat > top-1 agreement with native 95.57% > humaneval+, mbpp+ and ifeval all within noise of native i got it here in under 4 days, every one of the 15,360 experts in my weights sits at 2 to 5 bits by measured sensitivity, 2.70 on average, the engine is upstream vllm main with 22 patches and no fork. my own cuda kernels handle the exl3 decode linear and the routed experts, and the speculative draft depth gets picked every round from a measured cost table.

Be skeptical of standalone tok/s numbers! Headline: 690.9 tok/s on two DGX Sparks! Corrected: 690.9 tok/s on two DGX Sparks (448 streams / 256 token output / 1.5 tok/s per stream)! Also, 448 streams decoding 256 tokens at 1.5 tok/s is what I would call "benchmark theatrics"

🚨 BREAKING 🚨 ATF Releases NFA guidance in response to the Texas NFA lawsuit: "Effective October 9, 2026, ATF will no longer require individuals to seek ATF approval under the NFA before making or transferring short-barreled rifles, short-barreled shotguns, silencers, and “any other weapon[s]” as defined in 26 U.S.C. § 5845(e). ATF will also no longer require such items to be registered in the National Firearms Registration and Transfer Record. These policies apply uniformly to anyone otherwise subject to these requirements, not just to the plaintiffs in the litigation."

1/ Day 2 of porting OBS Studio to Rust. Yesterday: a bit reader whose position is a uint8_t, so it wraps at byte 255 and re-reads byte 0. It has been in libobs since 2019: 🐛 github.com/obsproject/obs… Today: a release, and a pile of decade-old bugs fixed. 🧵

I have finally started to use up my weekly limit with Opus 5.5. I went to set up Codex to carry on the work for the rest of the week… time to run ultra and 1.5x speed!

absolutely do not take justin's advice btw you are way more likely to lose your money by superficially moving it around than it is for fundamental cryptography to break overnight (if that happened, the whole world would break, not just coins) an unnecessarily performative post

Updated to default to using @spiritbuun amazing inference engine and give them a follow if you havent 80-125tok/s on a 3090 ! github.com/spiritbuun/buu… huggingface.co/DJLougen/Qwen3…

$300 for a bug that gives free access to OpenAI’s paid models with no API key or account. Zero reason to report anything else I find to them.