vinn joseph @_Vinnj

Running @UnslothAI's Qwen3.6-35B-A3B UD-Q4_K_M (22 GB) on a 16 GB Mac mini M4, through stock llama.cpp with our Kraits plugin. 4-agent coding crew: 16–24 tok/s writing, ~47 tok/s reading at 12K tokens, executed-code scores 6/8 to 9/9. github.com/1picassoai/kra…

An 80B coder on a 16 GB Mac, fully offline. Qwen3-Coder-Next at ~13 tok/s. Free, open source, one-line install, 29 GB download. Your code never leaves your machine. github.com/1picassoai/coa…

@Niraj_Dilshan The model only wants 4 of its 128 experts per word, so we read those 4 off the SSD and keep the recent ones on the GPU. 4.2 tok/s with every expert read, about 10 in fast mode. If you've got an Apple silicon Mac, try it and send me your number.

OpenAI's gpt-oss-120b, running fully offline on a 16 GB Mac at 10 tok/s. Free, open source, one-line install. Your code never leaves your machine.

@RobGuerra90 It is. OpenAI's own gpt-oss-120b, running on a 16 GB Mac mini, nothing sent anywhere. If you've got an Apple silicon Mac, I'd like to know how it runs on yours.

Thanks for the write-up. Two corrections on how it works. It isn't sub-2-bit. Only the experts are squeezed (gate and up to Q2_K, down to Q3_K); attention and everything else stay exactly as the publisher shipped them. And the whole model never sits in 16 GB: it's a mixture of experts, so each word reads only the experts it uses from the SSD, and a bank on the GPU keeps the recent ones. On reasoning, we test it rather than assume it. Six agent tasks through Aider, in Python, TypeScript and Rust, with hidden tests: fast mode passed 5, and exact mode passed the one it missed. It's open source, so please give it a try on your own Mac and tell us how it does, good or bad. Issues are open: github.com/1picassoai/coa…

122B model. 16 GB Mac. Coachwhip runs a 122B model fully offline on Apple silicon — 4.2 tok/s. Free, open source, one-line install. No cloud bills, no signups, no API keys.

Coachwhip v0.3.0 is live! We heard the flood of requests for the new Qwen model, so support is officially here. Update your install, run it locally on Apple silicon, and let us know what you want to see next—we'd love your feedback and feature requests:

80B model. 16 GB Mac. Built Coachwhip so you can run Qwen3-Coder-Next fully offline on Apple silicon. Free, open source, one-line install. No cloud bills, no signups, no API keys.

The always-used part lives on the GPU; the experts stay on the SSD and are streamed as the router picks them, with recent ones kept in a bank on the GPU. Measured on a 16 GB Mac mini M4: about 8 s to load, then the first word about 5 s after a short prompt. Details: github.com/1picassoai/coa…

Good question. Most of the 80B's layers use linear attention, so the cost per word grows only slowly as the chat gets longer. Chats go up to 16K tokens, and the page shows the live speed under every answer. Try it on your Mac and tell me what you see; per-model measurements are coming to the repo.

@itsvlady Same 😅 So I built a way to run an 80B coder on a 16 GB Mac. $0 a month, and nothing leaves the machine.

@CryptoUniBull @codingraj Not at all, that link was the issues page, and there are none yet 😄 Here's the code: github.com/1picassoai/coa…

@codingraj Hi no Linux or Windows support yet but it’s in pipeline. Let me know how it goes 🙏

@codingraj Thanks Raj! Love what you're building, same belief here: nothing leaves the house. If you run it, tell me how it does on your Mac, and if anything breaks, please open an issue in the repo: github.com/1picassoai/coa…

@flipbit03 Thanks! 16K tokens per chat today. It's a fixed setting in v0.2.0, sized to fit a 16 GB Mac. The models themselves go much further, so it's the first knob to open up for bigger Macs.

@RobGuerra90 Thanks Rob! If you give it a spin, I'd love to hear how it runs on your Mac.

Thanks! That one's qwen35moe (Qwen3.6), which Coachwhip doesn't run yet. At Q4 it's 8.5 GB: on a 16 GB Mac it mostly fits, on 8 GB it overflows, and that's where streaming experts pays. Could you open an issue with the model and your machine (chip + memory)? That's how we pick the next one: github.com/1picassoai/coa…

Depends on the model. Dense 27B/14B (like Gemma 27B): no, every word uses every weight. Small MoEs: yes in principle, but the win grows with how far the model overflows your memory, so the big ones gain most. Today Coachwhip runs the Qwen3 MoE family; Qwen2-MoE and gpt-oss are next. Which model did you have in mind?

@Jtxccjdbb 在16GB的M4 Mac mini上（Q4_K_M量化），80B大约每秒4–5个token。比能完全装进内存的小模型慢，但它能离线跑起来。M2还没测过，如果你试了，很想看看你的数据。

@WahyuCodes A linter matches patterns. Sarge asks a local model to read the file — and it gets better at your repo over time: break the same rule twice and it gets written into your book. A linter warns. Sarge refuses — the app doesn't run until it passes.

You put the rules in your prompt. Your agent reads them, agrees, and still hardcodes the connection string. Then it runs the broken app anyway, leaving you to clean up the mess. Take a look:

@ABG__UB If someone is already inside one of your agent sessions, shared state is the least of your problems. The session is the attack surface. Everything after that is theirs anyway.

Your agent reads your CLAUDE file, agrees with it, then ships something sloppy and useless. Started a board to collect what agents get wrong. Open source, community effort.