opensource.wtf

119 posts

opensource.wtf banner
opensource.wtf

opensource.wtf

@OpensourceWtf

Opensource software is changing and I am build cool opensource projects for where its going.

Katılım Ocak 2026
30 Takip Edilen59 Takipçiler
opensource.wtf
opensource.wtf@OpensourceWtf·
@jun_song @JoshuaJBouw Bah it sux to be front run but you should still release your version. Also I'm kinda disappointed when I said 40tps decode was impossible because I thought you were targeting sub $5000 systems with it and found a way to cheat physics.
English
0
0
0
6
Jun Song
Jun Song@jun_song·
@JoshuaJBouw It was loading the weights on SSD before and that was extremely slow. Speed matters here
English
1
0
1
444
Jun Song
Jun Song@jun_song·
Oh this was what I’ve been working on. They already released it 🥲 Great to see amazing future of inference.
dunik@dunik_7

a Tsinghua University lab just put a project on GitHub that replaces a $400,000 H100 rack with a single 24GB graphics card. it's called ktransformers, and the trick is almost stupidly simple: the experts you actually use stay on the gpu, the ones you don't sit on the cpu until they're called / deepseek-v3 and r1 with 139K context in 24gb of vram / up to 28x speedup over the standard setup / fine-tune deepseek-v3 across four rtx 4090s instead of a datacenter / built by tsinghua university's madsys lab, not a startup with a landing page apache 2.0, and already past 17,000 stars. -> github.com/kvcache-ai/ktr… bookmark it.

English
11
16
434
53.4K
noname
noname@malikwas1f·
In anticipation of running Kimi on RTX 3090, we've been making some optimizations. 🧐 Your RTX 3090 has INT8 tensor cores sitting idle every time it reads a prompt. We just switched them on for Qwen3.6-27B: ~50% faster prefill. one env flag, zero quality loss (8-pack tied). Free 1.5× on your agent & RAG prompts. 👇 github.com/noonghunna/clu…
English
11
5
107
6.7K
Theo - t3.gg
Theo - t3.gg@theo·
Kimi k3 is an incredible model. It is not an incredible value. In most tasks, it comes out to roughly the same cost as GPT-5.6 Sol. K3 is half the price of 5.6 Sol per token. GPT-5.6 uses half as many tokens. Price evens out. GPT-5.6 is 2x faster TPS, so it gets work done ~4x faster than K3 at roughly the same price. I still love K3 and will be using it for a TON of stuff. I'm just tired of people pretending it's way cheaper when it's not.
Theo - t3.gg tweet media
English
206
184
4.2K
238.2K
AJ
AJ@ItsmeAjayKV·
Poll time: If you had the hardware to fit it, how slow would you be willing to go to run the biggest/best version of a model locally?
English
10
1
5
825
opensource.wtf
opensource.wtf@OpensourceWtf·
@Youssofal_ The interesting thing is that its more expensive per job than GPT 5.6. Implies Sam actually give it to us at close to cost right now if China pricing is higher.
English
0
0
0
13
Youssof Al Toukhi
Youssof Al Toukhi@Youssofal_·
- 2.8 TRILLION parameters. - Beats Opus and nearly matches Fable - 3.5x cheaper than Fable. - Native vision - open weights soon. I know Dario is SEETHING!
Kimi.ai@Kimi_Moonshot

Introducing Kimi K3: Open Frontier Intelligence 🔹 2.8 Trillion Parameters, 1 Million Context, Native Multimodal 🔹 Kimi Delta Attention enables up to 6.3x faster decoding in million-token contexts 🔹 Attention Residuals deliver ~25% higher training efficiency at <2% additional cost 🔹 Built for long-horizon agentic coding and self-evolving workflows Kimi K3 is now live on on Kimi.com, Kimi Work, Kimi Code, and the Kimi API. Open Weights by July 27, 2026. 🔗 API: platform.kimi.ai 🔗 Tech blog: kimi.com/blog/kimi-k3

English
1
0
8
500
Taelin
Taelin@VictorTaelin·
"Usage credits are required for this model." Fable not working on plan anymore. Is this just here?
English
239
23
1.1K
79.6K
opensource.wtf retweetledi
Joe Muller
Joe Muller@BosonJoe·
GLM 5.2 @ 19.8 tok/sec on 2 DGX Sparks ⚡️ I switched to a dspark draft model with K=2 (model from @RedHat_AI) Acceptance is ~68% because the drafter was trained with the full FP8 model... Next step is to fine tune the drafter so acceptance on this super quant goes up
Joe Muller@BosonJoe

GLM 5.2 now pushing past 18 tok/sec on 2 DGX Sparks 🔥🔥🔥 This time I quantized the 3 biggest attention projections to NVFP4 (compared to the original FP8) There are still wins to be had on the speed and quality fronts so stay tuned

English
38
17
190
48K
opensource.wtf
opensource.wtf@OpensourceWtf·
@Giannisanii I have a question why did you move from c to rust? Did you see real performance gains? I'm going to be targeting rust from a memory standpoint on my MLX MOE streamer.
English
0
0
0
20
Gianni
Gianni@Giannisanii·
MiniMax M3 is a 428B parameter model. It just ran on my desktop: two consumer 16GB GPUs (4060ti + 5060ti) and 32GB of RAM. Ran on pulsar, my open source Rust + CUDA engine. The routed experts stream off NVMe per token, so only the 23B active slice ever touches VRAM. 3.4 tok/s sustained, fully local, no cloud anywhere. M3 was also one of the cleaner ports of the seven architectures pulsar runs: swiglu_oai and partial rotary were the only surprises, everything else just fit. MIT licensed: github.com/giannisanni/pu… @MiniMax_AI @RenLeanna check it out if u have the time :)
English
6
4
52
3.9K
opensource.wtf
opensource.wtf@OpensourceWtf·
@jjhartmann Seriously ther'es so much more juice to squeeze in just optimizing and fine tuning existing models.
English
0
0
0
12
AJ
AJ@ItsmeAjayKV·
Woke up to check, hit usage limit since i'm not on any plan😖
AJ tweet media
English
1
0
0
38
AJ
AJ@ItsmeAjayKV·
Time to see how good Kimi K3 is 🤩
AJ tweet media
English
1
0
10
479
opensource.wtf
opensource.wtf@OpensourceWtf·
I got Hy3 usable at Sub 100GB! Trying to get 96gb working for all the Mac Studio Bros. Writeup and release soon.
opensource.wtf tweet media
English
1
0
1
38

Keşfet

@jun_song @JoshuaJBouw @malikwas1f @Giannisanii @theo @ItsmeAjayKV @Youssofal_ @VictorTaelin