🌱Taka Yoshida◽

256 Tb/s data rates over 200 km distance have been demonstrated on single mode fiber optic, which works out to 32 GB of data in flight, “stored” in the fiber, with 32 TB/s bandwidth. Neural network inference and training can have deterministic weight reference patterns, so it is amusing to consider a system with no DRAM, and weights continuously streamed into an L2 cache by a recycling fiber loop. The modern equivalent of the ancient mercury echo tube memories. You would need to pipeline a bunch of them to implement modern trillion parameter models, but fiber transmission may have a better growth trajectory than DRAM does today, so it might someday become viable. Much more practically, you should be able to gang cheap flash memory together to provide almost any read bandwidth you require, as long as it is done a page at a time and pipelined well ahead. That should be viable for inference serving today if flash and accelerator vendors could agree on a high speed interface.

France is investing $30,000,000 in this new ai initiative That’s how much Google will spend in 90 minutes. No joke. Every 90 minutes Google will spend $30 million on capex this year. One company.

この教科書の「メッセージで操作を起動」だと、単なる「メソッド呼び出し」とイコールになりがちですね^^; 外から命令できず、送れるのは要求や質問だけ…というのがケイのOOPの原義の核で「オブジェクトは小さなコンピューターで、サービスを提供するサーバーと見做す」ことこそが「独特な考え」です