Edward Kmett
13.5K posts

Edward Kmett
@kmett
Founder/Chief Scientist @Positron_AI Haskell, category theory, AI, and safety. https://t.co/wYaTDISbEB https://t.co/XwbnuaOUzq 🦋 @kmett.ai


256 Tb/s data rates over 200 km distance have been demonstrated on single mode fiber optic, which works out to 32 GB of data in flight, “stored” in the fiber, with 32 TB/s bandwidth. Neural network inference and training can have deterministic weight reference patterns, so it is amusing to consider a system with no DRAM, and weights continuously streamed into an L2 cache by a recycling fiber loop. The modern equivalent of the ancient mercury echo tube memories. You would need to pipeline a bunch of them to implement modern trillion parameter models, but fiber transmission may have a better growth trajectory than DRAM does today, so it might someday become viable. Much more practically, you should be able to gang cheap flash memory together to provide almost any read bandwidth you require, as long as it is done a page at a time and pipelined well ahead. That should be viable for inference serving today if flash and accelerator vendors could agree on a high speed interface.



@ajeya_cotra So that's ~50 hours of work specific to this project; it's hard to say how long it would take unassisted but I'd guess months at least.

Excited to announce today that my startup, @positron_ai, has closed a $230M Series B financing round at an over $1B valuation, co-led by great folks at @jumptrading, Arena, Unless Ventures, and strategic backing by @Arm! bloomberg.com/news/articles/…



























