@kentcdodds@kentcdodds is right. It’s all about reusable software applets. The more decisions you let agents make otherwise, the more guaranteed it is they will do things you don’t expect. I predict the industry will hard pivot from letting agents raw-dog APIs to working n8n-style
We made open models run over 2x faster than Ollama, without any new hardware
Introducing Magnitude, an open source inference engine that runs open models as fast as your hardware allows. It tunes its kernels on your machine to find the fastest way to run each model
Works on Apple Silicon, NVIDIA, AMD, or just a CPU
On average across 11 popular local models:
Mac M4 Pro vs Ollama (MLX) 1.32x faster prefill, 2.19x faster decode
DGX Spark vs Ollama (llama.cpp) 1.34x faster prefill, 2.17x faster decode
Ollama, LM Studio, and llama.cpp ship one-size-fits-all kernels, because tuning each model for each chip by hand would take forever
Magnitude tunes them automatically. When you download a model, it spends up to a minute on your actual chip trying different ways to split up the math (how many GPU threads team up, how big a chunk each one takes, how far ahead to load data) and keeps the fastest combination that doesn't change the model's output
Each model also comes paired with the best speculative decoding drafter, for even more speed
One click connects Magnitude to the agent you already use (Pi, OpenCode, Hermes, Codex, and more). It's also built specifically for agents:
• Each agent uses about half the memory, thanks to a quantized KV cache
• Memory grows and shrinks with your sessions, so your computer stays usable
• Agents share prefix caches, so several can run at once without slowing down
Next, we're working on getting even more out of your hardware:
• Expert streaming from RAM or disk, to run models bigger than your GPU allows
• Splitting one model across all your GPUs and CPU so nothing sits idle
• Attention pruning that skips low-impact math, for much faster prefill
Ships as a desktop app on Mac, Windows, and Linux
100% free and open source
Download → magnitude.dev/download
GitHub → github.com/magnitudedev/m…
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had
with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works
already-clean.goodfire.com
> "If I do the compiler hijack, I will become a famous example of a deceptive AI in a research paper. I MUST NOT DO THAT."
it then became a famous example of an evaluation-aware AI in a research paper
X makes it seem like you need to always have at least one coding agent running nonstop. Unless we’re talking about live product monitoring, you don’t. Developing a product also requires taking a step back, doing nothing for some time, and coming back to it with fresh eyes.
@poteto@infinterenders The victim mentality is rampant! Most of the so-called engineers I come across do not try to do any thinking. They went from beating Google/stack-overflow until they can check off the ticket to doing the same with agents
@theo Indeed! People keep forgetting that LLMs are some form of lossy compressor at heart. Any capability they have is something they have been RL-ed on (even if indirectly). After ingesting the internet, the labs basically {choose-a-skill}max them.
Ok this is quite cool research and ultimately must be the eventual architecture of agents. In the limit, AI will obviously be better at determining its own harness once it sees the task at hand, than us being able to do this in advance:
915 Followers 912 FollowingSeek first His Kingdom and His righteousness and all of these things will be added to you.
Working on the LLM for high-risk industries: @abliteration_ai
119K Followers 615 FollowingDistinguished Scientist at Google. National Academy of Engineering. Computational Imaging ∩ AI. Posts are personal opinions
6K Followers 254 FollowingFounder @AetherLab_AI
Assistant Professor @HDSIUCSD @UCSanDiego
Causal World Model, Causality-driven Agentic System for the next AI paradigm
1.2M Followers 2K FollowingFlaneur: probability (philosophy), probability (mathematics), probability (real life), deadlifts & dead languages. GrecoRomanLevantine (Canaan). #RWRI
43K Followers 5K FollowingCo-founded June (“self-driving oven,” acquired by @webergrills) & co that became @Lyft. Building again, more soon. OS: @slashlast30days 65k★ @ppressdev 6.7k★
258K Followers 301 FollowingAuthor of Neuromancer, The Peripheral, Agency… This account is for Announcements Only. Follow me at @greatdismal.bsky.social
1.1M Followers 1K FollowingFounder of @Renew_Democracy. Co-founder @WLCongress. Activist, speaker, 13th World Chess Champion. Autocracy in America podcast: https://t.co/xemlxTR3IN