
BlackwellBoy
8.7K posts

BlackwellBoy
@Blackwellboy
Sovereign local AI architect • 1208 GB heterogeneous beast (RTX 5090 + 5×3090 + 8× DGX Spark + M4 Mac Mini) Hermes ftw.













soon 1TB of VRAM. the scary part isn't the number, it's that nobody stops you. my feed sees me at 824GB and whispers "1 more Spark." so u accumulate. my power company sends handwritten Xmas cards now. still waiting on my first Sparks Anonymous meeting. attendance: me. @NVIDIAAI


Are most of us these days just frontier chasing? Whichever lab has the best model, we go for it. The moment someone else catches up, we switch. Billions of dollars for just a few weeks of moat. What an era to live in.


744B parameters. On a laptop. With 25GB RAM. Colibri runs GLM-5.2 (744B MoE) in pure C with zero dependencies. The trick: only ~40B params activate per token, so it keeps the dense part resident and streams experts from disk on demand. A single 2,400-line C file. No GPU, no BLAS, no Python at runtime. This shouldn't work. But it does. ⭐ 2.1K #AI #OpenSource github.com/JustVugg/colib… Follow for daily dev finds 🔔

Capped each of my 4x3090 from 220W to 300W and re-ran Gemma 4 31B (AWQ, vLLM, TP4, single stream). Decode: 73 -> 77 -> 78 tok/s. That is +37% board power for +7% more speed. The curve goes flat after 220W. Stock power is already most of what these cards give you.

For those who are not able to obtain/run the Full GLM 5.2 quantrio INT4/INT8 Mixed - Abliterated. Here is a clear instruction and information on how this is done and get you setup and running. GIthub Repo updated for one-shot script (see link in Article).












