RapidFireGames

chips get all the love but the interconnects across all levels from c2c to rack-to-rack are as important, and many chip makers are sleeping on it until very recently. Even most interconnects people just want it to be as transparent as possible, just send and receive the bits with lower error rates and lower energy per bit, wrong long-term direction imo. Interconnects are part of the living creature, so many things happening in your blood vessels in addition to just moving stuff, and your axons do much more than carrying spikes. People do not appreciate interconnects, smaller volume, poor margin, messy ecosystem, manual process, it's been a spiral of grinding, and it is largely invisible. How often do you see people tearing down transceivers and die shot of DSP chips vs logic chips? How often do you see high res pictures of all the connector's gold fingers on the NVL72 cartridge? Because it sounds boring, it's just making contacts, shoving electrons and photons, what a simple problem. But that is deceiving, and theres so much to it. You might want 576 to begin with, had to cut down to 288, then to 144, and finally to 72, and that barely worked first time. You are entering the domain of analog and mixed signaling, you are fighting copper real estate with power delivery, you are getting impedance mismatch and reflections and interference at every stupid interfaces, your optical components' and connectors' backreflection is making your laser mode hop.. And we are not even going into the thermal and strain-stress, the reliability of how many times you can actually mate your connector, the horrible jobs people are doing across the stack from science-project-originated photonic PDKs to hand cleaved laser dies to optical engines to rack manufacturing, on spec-ing out the requirements, the tolerances.. On top of all these, people thinking about where the bits should be going and people who know what the bits have to go thru are two totally different groups of people. But it is shifting, pluggable volumes shipped are doubling and tripling for scale-out, scale-up domain asks for much higher bw than scale-out, and interconnects are inevitable even if you cram as much compute and memory onto a single wafer. People will see it always has been interconnects, the chips people have already been doing it on the chips, that you can sort it out with your chip designers and foundries, and now you need to work with more people to sort it out from chips to boards/trays to racks to pods to data halls and data centers. These people speak very different languages and care about very different things, and it will take a lot of effort to pull order out of all the chaos. At the end of the day, it is such a crazy problem to work on, such a beautiful thing to make, millions of amps of current flipping 1e20 of flops, sextillions of photons carrying thousands of terabits per second, a few tons of copper, tens of thousands of fibers totally few hundred kilometers, one scale-up domain. You absolutely need a group of people that appreciate the beauty and care about the craft behind the grinding to make it together.

Memory cost and capacity are significant issues for AI accelerators. Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth. NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth. You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff. That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance. There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn. It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk. A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run. I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.

So following these principles here is the architecture that falls out and it’s quite different than what is thought to be optimal today. Starting with the memory: LPDDR6 tops out around 14.4 Gb/s per pin. We would want to as many channels as the die’s edge can carry. If we can fit 24 channels, that’s 576 data pins, and you get ~1 TB/s per chip with 192 GB of storage. Same as the b200 in size but about an a100 in bandwidth. But remember that LPDDR access latency is ~100 ns which is about the same as HBM. So streaming weights from LPDDR costs you nothing on latency and we just give up per package bandwidth, and remember our pipelining already collapsed that requirement. Now work backwards to compute. Tokens/sec = ~aggregate BW / bytes touched per token, and at batch 64 in FP4 every delivered byte wants ~256 FLOPs. 1 TB/s × 256 = ~260 TF of FP4. Size ~280 TF for some headroom and stop! That’s all we need, the rest of the alpha is in latency tuning not flops. Now process node, again using first principles thinking. On TSMC N6 that MAC array would be about 30 mm^2. The rest of the die goes to SRAM buffers, 24 LPDDR PHYs, and 56 lanes of plain 32 GT/s NRZ SerDes PCIe 5.0 transceivers, nothing exotic. Because we didn’t provision more flops we can provision what we really need which is ports. If we do 7 ports × x8 we can get a full mesh across 8 chips with ~224 GB/s any to any ops, sized for 8 KB activations instead of 100 MB gradients. Approximate Total Area: ~144 mm^2, ~180W with the DRAM. Fucking Mobile SoC level economics. And we get away without a crazy process node. N3 shrinks your logic, but that’s actually the one thing this die barely has. SRAM stopped scaling and the PHYs and SerDes are analog so they don’t shrink at all. An N3 port shrinks ~20% of the floorplan, and saves maybe 10-15W, but doubles the cost on a small die that already has beautiful *chefs kiss* 👨‍🍳💋 yield. Now do the node math: 8 chips (b/c full mesh w/ 7 ports) = 8 TB/s aggregate BW, 1.5 TB total of memory, and ~1.44 kW. A B200 inside an NVL72 is 8 TB/s, 192 GB, ~1.67 kW all in. So we are basically neck and neck on BW with a b200 with slightly lower power because we have more chips connected with quicker interconnects allowing them to act as a single unit. If we scale to the rack level at 72 nodes, that 576 chips, 576 TB/s which is again the same aggregate bandwidth as a GB200 NVL72, so approximately the same tokens/sec by construction but with ~104 kW of power against ~120-130 and 110 TB of memory against 13.8 TB of the NVL72. So in summary 4.5x less compute silicon, zero HBM, zero CoWoS, zero leading-edge nodes and still token/watt ~20% better. So what’s stopping everyone from doing this? The number I told you hold whether you want to believe it or not and get even more competitive with longer reasoning chains or agentic workloads because as you scale those the all reduces now sits on the critical path of every token and on a 30 ns link with a full mesh that’s orders of magnitudes of token throughput improvements over Nvidia’s stack. Counterintuitively the link latency, not the chip architecture, decides your memory, logic, and everything else! Thanks for reading if you made it this far.

FRANCE’S ATTACK AT THE WORLD CUP💀

Legendary.

🚨There is a ton of momentum building in photonics right now. $POET $MRVL $LITE $COHR $AAOI $TSEM $CIEN Every part of the stack is heating up, and management teams have basically spent the last month broadcasting the same story. Let’s dig into some of the recent bits I’ve been watching... $LITE Q1 revenue up 58% YoY to $533.8M with next quarter guided to ~$650M, hitting their mid-2026 target two quarters early. Management pointed to cloud-optics demand running well above supply with transceivers, optical circuit switches, and early co-packaged optics driving the next leg. $COHR “AI data centers and communications remain strong long-term growth drivers.” They just posted $1.58B in revenue, +17% YoY, and highlighted accelerating hyperscale DCI demand across their ZR / ZR+ lineup and their 400G / 800G ramps. $MRVL “AI infrastructure is transforming faster than ever.” “We’re going to have a silicon-photonics powerhouse at Marvell when this is all done.” The $3.25B Celestial AI acquisition comes with a modeled $500M run-rate by FY28 and $1B by FY29 Celestial AI “Marvell is the ideal home for our Photonic Fabric… the scale and customer reach to take this platform into high-volume production.” $POET Their optical-engine technology is already designed into Celestial AI’s Photonic Fabric, the same platform Marvell just paid $3.25B to acquire. As AI systems move toward denser, more integrated optical engines, POET sits directly inside one of the most advanced photonics architectures in the market - now part of Marvell’s silicon-photonics roadmap. $AAOI Revenue up 82% YoY. They’re on track to build what they believe will be the largest domestic production capacity for 800G and 1.6T transceivers by year-end, roughly 35k parts per month, all inside their Texas footprint. $TSEM Expecting silicon-photonics revenue to more than double off last year’s base. They’re investing $300M into photonics and AI-focused expansion and calling for ~75% growth in that segment. $CIEN “We delivered record orders. Cloud and AI providers continue to invest in high-capacity optical transport.” DCI and long-haul momentum continue to show up directly in the numbers. ... But you can’t just look at the transceiver and laser makers. Some of the clearest signals are coming from the companies building the architecture and the supply chain behind it. $AVGO AI revenue moving from ~$11B to $20B+. They’re doubling down on silicon photonics Controlling the switch silicon, the optical DSP, and the optical engines at the heart of modern AI racks. If you’re bullish on photonics, this is one of the companies defining the socket. $ANET Building the “AI spine.” Their Ethernet-based AI networking platform relies entirely on high-speed optics to stitch together massive GPU clusters 100k-GPU scale and up. As clusters get larger, copper falls out of the system and optics takes over. $FN Record ~$980M in revenue. They manufacture optical engines for some of the largest players in the space - $NVDA, $LITE, $COHR, and others. When Fabrinet says demand is exceptional, it means the orders are already in the building. $GLW Optical Communications revenue up 33% YoY. AI racks require roughly 10x the fiber density of legacy cloud racks. Corning is supplying the physical layer that makes 800G and 1.6T optics viable at scale. $SMTC Solid print tied to growing traction in Linear Pluggable Optics (LPO). They sit in the analog layer, drivers and TIAs that fire the lasers, and are positioned directly under the 800G / 1.6T cycle. $MTSI Revenue up 30% YoY. They build the high-speed analog components that sit behind next-gen optical engines and are essential for 1.6T designs. .... This is all widespread. It’s lasers, transceivers, optical switches, silicon photonics, scale-up fabrics, DCI, long-haul transport, Ethernet AI spines, fiber density, and the analog chips behind every laser all moving in the same direction. If you’re tracking AI infrastructure, photonics continues to gain strength week after week.

This chart is why I'm staying away from the SpaceX IPO. Five of the most hyped IPOs of the last 15 years, and every single one collapsed after listing. - UBER lost 70% of its IPO price. - META crashed 77% from its peak. - Robin Hood fell 92%. - Coinbase fell 93%. - Rivian fell 95%. The hype is always priced in on day one. The people who bought the hype got crushed. But look at where the real money was made. At the bottom, when nobody wanted these stocks. Robinhood went up 22x from its low. Meta went up 45x. Uber 7x. Patience beat hype in every single case and Rivian reminds us that even patience doesn't save every company. SpaceX will be the most hyped IPO of the decade. History tells me I don't need to be there on day one. If it's a great business, there will be a better price later. There almost always is. First they fall. Then they fly.