almaraz.eth

The Cost of Intelligence is Heading to Zero | Hyperspace P2P Distributed Cache We present to you our breakthrough cross-domain work across AI, distributed systems, cryptography, game theory to solve the primary structural inefficiency at the heart of AI infrastructure: most inference is redundant. Google has reported that only 15% of daily searches are truly novel. The rest are repeats or close variants. LLM inference inherits this same power-law distribution. Enterprise chatbots see 70-80% of queries fall into a handful of intent categories. System prompts are identical across 100% of requests within an application. The KV attention state for "You are a helpful assistant" has been computed billions of times, on millions of GPUs, identically. And yet every AI lab, every startup, every self-hosted deployment - computes and caches these results independently. There is no shared layer. No global memory. Every provider pays the full compute cost for every query, even when the answer already exists somewhere in the network. This is the problem Hyperspace solves where distributed cache operates at three levels, each catching a different class of redundancy: 1. Response cache Same prompt, same model, same parameters - instant cached response from any node in the network. SHA-256 hash lookup via DHT, with cryptographic cache proofs linking every response to its original inference execution. No trust required. Fetchers re-announce as providers, so popular responses replicate naturally across more nodes. 2. KV prefix cache Same system prompt tokens - skip the most expensive part of inference entirely. Prefill (computing Key-Value attention states) is deterministic: same model plus same tokens always produces identical KV state. The network caches these states using erasure coding and distributes them via the routing network. New questions that share a common prefix resume generation from cached state instead of recomputing from scratch. 3. Routing to cached nodes Instead of transferring KV state across the network for every request, Hyperspace routes the request to the node that already has the state loaded in VRAM. The request goes to the cache, not the cache to the request. Together, these three layers mean that 70-90% of inference requests at network scale never require full GPU computation. This work doesn't exist in isolation. It builds on research from across the industry: SGLang's RadixAttention demonstrated that automatic prefix sharing can yield up to 5x speedup on structured LLM workloads. Moonshot AI's Mooncake built an entire KV-cache-centric disaggregated architecture for production serving at Kimi. Anthropic, OpenAI, and Google all launched prompt caching products in 2024 - priced at 50-90% discounts - because system prompt reuse is so pervasive that it changes the economics of inference. What all of these systems share is a common limitation: they operate within a single organization's infrastructure. SGLang caches prefixes within one server. Mooncake disaggregates KV cache within one datacenter. Anthropic's prompt caching works within one API provider's fleet. None of them can share cached state across organizational boundaries. Hyperspace removes this boundary. The cache is global. A response computed by a node in Tokyo is immediately available to a node in Berlin. A KV prefix state generated for Qwen-32B on one machine is verifiable and reusable by any other machine running the same model. The routing network provides the delivery guarantees, the erasure coding provides the redundancy, and the cache proofs provide the trust. What this means for the cost of intelligence Big AI labs scale linearly: twice the users means twice the GPU spend. Every query is a cost center. Their internal caching helps, but it's siloed - Lab A's cache can't serve Lab B's users, and neither can serve a self-hosted Llama deployment. Hyperspace scales sub-linearly. Every new node that joins the network adds to the global cache. Every inference result enriches the cache for all future requests. The cache hit rate rises with network size because query distributions follow a power law - the most common questions are asked exponentially more often than rare ones. The implication is simple: as the network grows, the effective cost per inference drops. Not linearly. Logarithmically. At 10 million nodes, we estimate 75-90% of all inference requests can be served from cache, eliminating 400,000+ MWh of energy consumption per year and avoiding over 200,000 tons of CO2 emissions. The first person to ask a question pays the compute cost. Everyone after them gets the answer for free, with cryptographic proof that it's authentic. Training is competitive. Inference is shared Open-weight models are converging on quality with closed models. Labs will continue to differentiate on training - data curation, architecture innovation, RLHF tuning. That's where the real intellectual property lives. But inference is a commodity. Two copies of Qwen-32B running the same prompt produce the same KV state and the same response, byte for byte, regardless of whose GPU runs the matrix multiplication. There is no moat in multiplying matrices. The moat is in training the weights. A global distributed cache makes this separation explicit. It doesn't matter who trained the model. Once the weights are open, the inference cost approaches zero at scale - because the network remembers every answer and can prove it's correct. No lab, no matter how well-funded, can match this. They cannot share caches across competitors. They scale linearly. The network scales logarithmically. The marginal cost of intelligence approaches zero. That's the endgame.

Ethereum planned to cut L1 deposit wait times by 98%. Not with a hard fork. Not with a new chain. Not by breaking anything. Just by counting attestations differently. The Fast Confirmation Rule (FCR) is the upgrade shipping to @ethereum right now ↓

🚀 New Plonky3 release just dropped. This is probably our most impactful and ambitious release so far: - MUCH faster lookups - High-arity folding - N-ary Merkle trees + Merkle caps - Major Poseidon2 optimizations - Poseidon1 support - And many more… Let’s break it down 👇

hello world

Craaazy 365 days of @leanEthereum progress. Cheers to the builders. Cheers to the dreamers. Cheers to anti-fragility, too :) Devcon, Bangkok — Nov 12, 2024. The suspense is real. The room overflows; hundreds can't get in. An "announcement of an announcement" had sparked wild speculation about my "most ambitious initiative". Who knew the beam chain vision would evolve into lean Ethereum? Next-level ambition, seeping into all layers of L1. Snarks for consensus and execution. Fort mode and beast mode. What's new? zkEVMs. Real-time proving. Full validation in a tab, on a phone. Let's pump L1 gas with the exponential snark curve. Starting in months, not years. To me it all points to 10K TPS, the gigagas frontier. Dream bigger dreams for L1. Believe in something. ——— part 1—lean consensus devnets → clients: 4 new lean CL clients (Zeam, Ream, Qlean, Lantern) → languages: 3 new CL languages (Zig, C++, C) → specs: by @tcoratger + 14 others; 3SF-mini subspec by @vitalikbuterin → testing: revamped test framework by @fselmo2; @Sib_Katya metrics → devnets: multi-client 3SF with 4s slots and 12s finality; PQ soon™ coordination → hires: EF Protocol coordinators @corcoranwill and @ladislaus0x → CL teams: led by @Gajpower, @unnawut, @kamil_abiy, @mstore80 → 7 consensus calls: teams, PQ, p2p, exit queue, APS, 3SF, PQ specs → Cannes workshop: 1 day at EthCC in June; interop kicked off → 13 interop calls: by @corcoranwill on Wednesdays at 2pm UTC → Cambridge Oct workshops: 1 day leanVM, 3 days PQ, 3 days CL cryptography → leanSig: 3 papers on hash sigs by Benedikt, @khovr, @kudinov_mikhail → leanVM: fast minimal aggregation zkVM by Emile → WHIR: fast Plonky3 implementation by @tcoratger → optimisoors: @AngusGruen, @GiacomoFenzi, @lambdaclass, @kiliconu → Poseidon2: 4 cryptanalysis workshops by @khovr, @asanso → maths: $1M Millennium-like proximity prize; papers flowing → formal verification: ArkLib by @QuangVDao research → consensus team: hires @yannvon and lead @robsaltini join @luca_zanolini → faster finality: 1- or 2-round designs with Ethereum-grade liveness → 3sf-gold: new fast inclusion by @fradamt, @vitalikbuterin from Cambridge → p2p: @qdrvm_io simulator; @raulvk ethp2p; @soispoke leanp2p → rainbow staking: new Cambridge ideas; specs by Dan Goron & Alex Vlad ——— part 2—lean execution zkEVM tech → real-time proving: ~100 engineers pushing across ~10 zkVM teams → GPU proving: 16 5090s (10kW) proving mainnet; $0.01/block → guests: revm (Reth), levm (Ethrex), evmone (Zilkworm), ZKSync OS → more guest programs: Geth, Besu, Nethermind and others soon™ → RISC-V: de facto ISA of choice for zkEVM proving → Picus: prolific Veridise tool to identify under-constraints → formal verification: $4M across 40 grants by @alexanderlhicks Ethproofs community → zkVM integrations: Airbender, OpenVM, Pico, R0VM, SP1, Ziren, ZisK → other integrations: Cysic, Fermah, Marlin, Snarkify, Zilkworm, ZkCloud → website: driven by @fbwoolf under new EF Ethproofs team → 7 calls: zkVMs, RTP, gigagas, RISC-V, native rollups, proximity gaps → Ethproofs day: Nov 22 at Devconnect; register at ethproofs[.]day → zkAttester demo: my home validator on zkEVM proofs at Ethproofs day EF zkEVM team → new team: led by @kevaundray with Cody, Han, Ignacio, Radek, Sophia → EF blog post: real-time proving requirements by @_sophiagold_ → zkLighthouse: modified Lighthouse client by @kevaundray → zkEVM/acc: @ignaciohagopian benchmarks; @codytouchgrass tests → more zkEVM/acc: @kevaundray standardisation; Ere by @han__0110 future of EL → Fusaka: per-tx gas limit (EIP 7825); MODEXP killer (EIPs 7823, 7883) → EVM 2.0: @vitalikbuterin proposal to enshrine RISC-V under the EVM → native rollups: championed by @lucadonnoh; wrote book and draft EIP → gas auto-pumps: 3x/year gas pumps (EIP-7938 by @dankrad) → gigagas L1: champion wanted—reach out :) gigagas@ethereum.org

. @Lighter_xyz, a ZK rollup using Ethereum blobs, is doing 4k+ TPS posting just ~100MB per day, all thanks to state diffs and the app-specific architecture. for comparison: - @arbitrum: 37 TPS @ 436MiB/d - @base: 155 TPS @ 2.11GiB/d - @soneium: 39 TPS @ 241 MiB/d

Today, around 7:48 AM UTC, an exploit affected Balancer V2 Composable Stable Pools. Our team is working with leading security researchers to understand the issue and will share additional findings and a full post-mortem as soon as possible. Because these pools have been live onchain for several years, many were outside the pause window. Any pools that could be paused have been paused and are now in recovery mode. All other Balancer pools are unaffected. This issue is isolated to V2 Composable Stable Pools and does not impact Balancer V3 or other Balancer pools. Balancer is committed to operational security, has undergone extensive auditing by top firms, and had bug bounties running for a long time to incentivize independent auditors. We are working closely with our security and legal teams to ensure user safety and are conducting a swift & thorough investigation. We’re grateful to our partners and the broader DeFi community for their support. Security notice: Fraudulent messages claiming to be from the Balancer Security Team are circulating. These are not from us. Do not interact with unsolicited communications or click unknown links. Official updates will be posted only via: - This official Balancer account on X (Twitter) - Our official Discord server Be careful with communications from other sources, they can be fraudulent. We will provide a comprehensive update with more details as our investigation progresses. The Balancer Team.

programmable Ethereum part 1: programmable Ethereum, VMs, and ISAs part 2: zk everything (costarring: RISC-V, EVM) part 3: scaling lean Ethereum with leanVM and EVM 2.0 1/30

alright, after a week of work you can now fully simulate Safe transactions _locally_ before signing and verify the transaction hashes in parallel using my `safe-tx-hashes-util` with a single command (use the `--simulate flag` simply). This lets you check exactly how the transaction will execute before you sign. Everything is local. Everything what is executed is printed in the terminal. Transparency at all cost. You trust your RPC provider here so use a trusted endpoint or the preferred solution of running your own node. I know many still do not believe me but local-first, cli-based verification is the way to go. Not hosted UIs. My verification script is _one_ fucking Bash file. Everyone can audit it by looking at exactly one file. No dependency bloat.

@MikeIppolito_ @nickwh8te @jillgun @pumatheuma Wow I watch almost all you guys’ content but didn’t realize it even existed. The content looks interesting too rip

Hey all - got some bittersweet news to share. As of today we are formally announcing the end of the Expansion pod. Thank you so much to everyone who tuned in, as well as my cohosts @nickwh8te , @jillgun and @pumatheuma Very proud of the content that we created, long live the modular future 🫡

1/ Ethereum should replace the EVM with RISC-V by @VitalikButerin at @protocol_berg. Not tomorrow—but eventually. Why? Because ZK proving costs are dominated by EVM execution. And if Ethereum wants to scale seriously, replacing the EVM is the next big lever. Let’s break it down

2024 was in many ways the centralization cycle. Decentralized primitives lost out to centralized apps across the board. Ethereum decentralization -> L2 multisigs Uniswap immutability -> Jupiter frontends Seaport programmability -> Ordinals offchain orders Metam*sk selfcustody -> Privy app wallets dYdX cosmos chain -> Hyperliquid validator The one beautiful exception to this? LayerZero - immutable protocol with flexible DVNs - clear tokenholder governance and rights - dominating marketshare across bridges - sovereign universal token standard for OFTs - opensourcing better L1 tech than any 11-figure chain In the past few months alone, they've shipped: - QMDB, a better blockchain database which gives you 2.3 million state updates per second - FAFO, a transaction parallelization approach handling storage read/write conflicts that can process 1.1 million transactions per second - vApps, an architecture to fit any Rust application into a universal onchain proof framework Back in 2021 I would joke that the corollary to the blockchain trilemma was the bridge trilemma: if we have at least 3 live crosschain bridges, one of them is about to get hacked. Ronin (625m), Poly (610m), Binance (570m), Wormhole (325m), Nomad (190m), Harmony (100m), Multichain (125m) - all eviscerated user capital. We don't worry about that any more. It's so rare to see the team with the best cryptonative ethos also dominate on both tech and distribution. Kudos to @LayerZero_Core for all the quality opensource shipping.

@barnabemonnot No one brings up how this might affect real time proving efforts. With zk the gas limit can be increased significantly, so we’re trading that off with faster slots it seems to me

Shortening the Ethereum L1 slot time from 12 to 6 seconds has benefits across the stack: * Better UX * More value for users, with lower fees on DEXes * A better confirmation engine for interop protocols * More! We make the case for its headliner consideration in Glamsterdam.

@PaulRBerg 👀

What is the best solution for onchain escrow nowadays?

1/ Synchronous composability is often called the holy grail of interop, enabling rollups to read and react to each other’s state within the same slot. Until now, this was mostly reserved for based rollups. Can we extend it to the rest of the rollup ecosystem? 🧵

the eth core devs don’t tweet a lot about just how hard the work that they do is so let’s talk about it: 1. every line of code they merge can move more money than most banks process in a quarter. there is no staging server for that. 2. they swap consensus logic for a 400B + dollar economy without scheduling downtime. ever. 3. they coordinate hundreds of researchers, auditors, and client teams across time zones, cultures, and philosophies, yet ship like a single mind. 4. they do it all in public, with every decision dissected by the loudest peanut gallery on the internet, and still keep the vibe collaborative. 5. they design for attackers who have nine figure incentives and infinite patience. then they sleep anyway. 6. they keep six independent clients in perfect sync so the same block lives at the same height for every node in the world. 7. they turn bleeding edge research into production code while preserving backwards compatibility for machines that went online before defi even had a name. 8. they debug issues that only happen once a year on a single archive node because someone somewhere will rely on that edge case. 9. they write cryptography that must stay unbroken for decades while the math itself evolves beneath their feet. 10.when the upgrade lands smooth the outside world shrugs. inside ethereum we know it was a minor miracle. every successful fork proves that decentralized coordination can outperform the world’s best hierarchies and shows that open internet capital markets are now the default. thank you, truly. we owe you everything.

The point of blockchains is to scale social consensus. Cryptography > economic incentives > social trust in order of scalability. Crypto is an attempt to move everything into the leftmost bucket but I have a feeling not everything can be and the final boss is governance.

gm Ethereum ☀️ It is our great honor to announce the mainnet launch of Privacy Pools! ETH users can now achieve on-chain privacy, while still dissociating from illicit funds It is now up to all of us to Make Privacy Normal Again 🫡 More info in this thread 👇