that1nerdy.eth

United we stand, divided we fall. For a united and strong Europe. 🇪🇺

This post got ZERO attention but is BY FAR the biggest AI news this week Google just published TurboQuant: a compression algorithm that makes AI inference 8x faster while using 6x less memory. No retraining. No accuracy loss. The biggest cost is inference which happens billions of times a day, scaling with every user and query. It’s the bill that never stops growing. Inference also eats memory alive. The reason why GPU memory is the scarcest, most expensive resource in AI. Previous compression methods had a little secret: shrinking the data required storing extra instructions about how it was shrunk. That overhead ate nearly half the savings. Google found a way to restructure the data so those instructions aren’t needed at all. The overhead just vanishes. 32 bits compressed to 3. The entire cost structure shifts. Context windows expand on existing hardware. API costs compress. Models that needed clusters start fitting on smaller machines. This seems like a pretty big deal for team google and the industry at large

He who saves his Country does not violate any Law.

if you're serious about local AI and actually want to understand it, the language of AI is CUDA. even AMD built ROCm to compete with NVIDIA, not with Apple. that should tell you everything. Mac is a beautiful machine. looks great on your desk. perfect for people who post desk setups and do nothing else. no flags, no tuning, no research, no optimization, no novel architecture work. just photos. when Karpathy drops autoresearch, does he say he runs on a cluster of Macs? when DeepSeek publishes training breakthroughs, is it on Apple Silicon? ask yourself what you actually want to learn. do you want to get your hands dirty, understand AI infrastructure from the metal up, share real findings, test frontier models and actually contribute? or do you want to be the guy farming engagements with a Mac photo on X. no substance. shallow. hollow. 0 depth. single RTX 3090. good PSU. one server board. EPYC. bifurcation cards and cables when you're ready. this is the starting line. my bro once you go GPU you go brrr.

What is the craziest prediction that you think could *actually* happen?

Everyone agrees ZK-STARKs are top-tier for building blockchains. This wasn't always the case and it's good that it became consensus. What’s not consensus yet and I predict will become so: if your chain relies on ZK-tech, your programming language and VM should be ZK-optimized. A program written in a ZK-friendly language (e.g. Cairo) can be smoothly processed by a ZK-based VM. If it's written in a PL that's not optimized for ZK (say, Solidity), then you'll need to go through several workarounds, before a zkVM can process it. These workarounds mean that (1), you're loosing accuracy and efficiency, and (2) you might even risk bugs. You need the right tool for working with ZK.