Kvn @spnova05

Joined October 2021

Tweets

126
Followers

5
Following

96
Likes

86

Kvn @spnova05

16 hours ago

@__tinygrad__ nop

0 0 0 73 0

View Details

@AkitaOnRails Finalmente um br na minha timeline. Opinião minha: LLM local no momento só faz sentido o strata com qwen uncensored para algumas requisições que modelos fechados negam o resto roda em assinatura ou API, hardware local além do custo, tem energia e também degrada. pra BR mt caro.

0 0 0 52 0

View Details

Blackfrost_SI @Blackfrost_AI

a day ago

Clearly, I will not be getting sleep tonight huggingface.co/MiniMaxAI/Mini…

11 34 533 39K 576

View Details

Kvn @spnova05

2 days ago

@0xSero Calm down mate, you are the 0.001% that has a 75k usd rig, still far hit for most of commom users

0 0 47 832 1

View Details

Kvn @spnova05

3 days ago

@ItsmeAjayKV I was thinking: maybe we should always test a model via the API before trying quants. People tend to see the promising benchmarks of the full model, then test it on a heavily quantized version. Let's try the same prompt via the API first!

0 0 0 5 0

View Details

Kvn @spnova05

4 days ago

@ashxhart @Aria_persian94 Yep, you among many others here, sounds very ignorant people when throw bs like it was a really bigger deal than it actually is. You dont even need to do a deep fact check, it is right at the beggining (federal civil agencies).

0 0 2 95 0

View Details

Kvn @spnova05

a week ago

@suraj_sharma14 @grok how about it grok?

1 0 1 12 0

View Details

Kvn @spnova05

a week ago

@ItsmeAjayKV At least all those “AI boxes” will finally get cheaper now that you won’t need 4 DGXs anymore.

0 0 0 36 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith It is a great model, since "oxalpha", it works very well on most of my demand, s2

0 0 0 10 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith And like I said. This is only the beggining, a small step, but I know we are gonna get real better... Also very soon they are releasing qwen4, We already can imagine how wonderful it is going to be with strata engine.

0 0 0 103 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith Yeah 🤣🤣 mostly because my hyperfocus mode is on... so my last 48 hours was about trying to run it as fast as I could, and the result was good! We came from average 3.5tk/s to 16,9 tk/s today, now I believe I'm satisfied and was able to contribute to the community

0 0 0 8 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith Also, on my box, when it grabbed ~45.3 GB the engine failed to start with "pinned disk staging did not allocate". I'm going to report that one to the dev. Thanks for the AHEAD tip, it's what got me digging!

0 0 0 12 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith One heads-up if you benchmark: the RAM tier is sized from free RAM at startup, so back-to-back runs get different tier sizes and that skews the numbers a lot. One of my runs got 4 GB less and lost ~10%. Pin it with STRATA_GLM_RAM_GB while comparing.

0 0 0 10 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith But that's a guess, and since you're more disk-bound than me it could be even bigger for you, or nothing. Worth trying --env STRATA_GLM_READ_CHUNKS=4 and maybe 2.

0 0 0 9 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith +15% decode just from that. Anything from 1 to 4 flattens out, so I kept 4. Disk wait per token dropped from ~30–37 ms to ~17–29 ms. My guess is that fewer, bigger reads suit my NVMe RAID 0 better, and the reader threads compete less with the CPU lane.

0 0 0 11 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith I ran the built-in bench (--bench: greedy, same output every time) with the RAM tier fixed so every run got the same RAM:

0 0 0 6 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith Yeah, 64K is on purpose. That's all I need for my agent harness, and with KV_INT8 it only costs 0.68 GB of VRAM, so my expert pool stays at the full 2352 slots. 256K would eat ~1.2 GB more of the pool all the time, even on short chats, and I never fill it.

0 0 0 5 0

View Details

Kvn @spnova05

2 weeks ago

@o_at72 @PeasantSmith For reference, I'm at ~14 tok/s decode, sometimes 20+ once the caches are warm on short answers. Disk misses are still about 40% of my per-token time, so that's where any further speedup would come from.