postmedia

3.5K posts

postmedia banner
postmedia

postmedia

@postmedia

Somewhere between paper and pixels

Katılım Eylül 2007
1.3K Takip Edilen181 Takipçiler
postmedia
postmedia@postmedia·
Guri Singh@heygurisingh

Holy shit... Microsoft open sourced an inference framework that runs a 100B parameter LLM on a single CPU. It's called BitNet. And it does what was supposed to be impossible. No GPU. No cloud. No $10K hardware setup. Just your laptop running a 100-billion parameter model at human reading speed. Here's how it works: Every other LLM stores weights in 32-bit or 16-bit floats. BitNet uses 1.58 bits. Weights are ternary just -1, 0, or +1. That's it. No floats. No expensive matrix math. Pure integer operations your CPU was already built for. The result: - 100B model runs on a single CPU at 5-7 tokens/second - 2.37x to 6.17x faster than llama.cpp on x86 - 82% lower energy consumption on x86 CPUs - 1.37x to 5.07x speedup on ARM (your MacBook) - Memory drops by 16-32x vs full-precision models The wildest part: Accuracy barely moves. BitNet b1.58 2B4T their flagship model was trained on 4 trillion tokens and benchmarks competitively against full-precision models of the same size. The quantization isn't destroying quality. It's just removing the bloat. What this actually means: - Run AI completely offline. Your data never leaves your machine - Deploy LLMs on phones, IoT devices, edge hardware - No more cloud API bills for inference - AI in regions with no reliable internet The model supports ARM and x86. Works on your MacBook, your Linux box, your Windows machine. 27.4K GitHub stars. 2.2K forks. Built by Microsoft Research. 100% Open Source. MIT License.

QHT
0
0
0
13
postmedia retweetledi
Massimo
Massimo@Rainmaker1973·
How couples met (1930-2024) [🎞️ EEAGLI]
Eesti
85
612
1.9K
357.4K
Gary Lamphier
Gary Lamphier@lamphieryeg·
Dean here swears a lot to distract from his thin understanding of how newsrooms actually operate. Similar to a schoolyard bully who feels compelled to compensate for his abject stupidity by beating up on little kids.
Dean Blundell🇨🇦@ItsDeanBlundell

There’s no such thing as liberal Media in Canada you dumb gooey fuck. Every major daily is owned by Conservative backers. 90% of Canadas newspapers are owned by Chatham asset management in the U.S.

English
4
21
176
4.3K