Less Wright

@SemiAnalysis_ ymmv, but I've run 2K scale H200 and B200 runs with 70B model, up to 3D parallel, with regional torch.compile with no issues. Compile is not distributed aware, so the better method imo is regional compile of the transformer blocks, not full model compile: #L345" target="_blank" rel="nofollow noopener">github.com/pytorch/torcht…

Torch compile is great for single node but horrible for any serious multi-node production training. It is a gaint footgun with 100+ mini footguns waiting to be stepped on

Our TorchTitan Paper has been Accepted to ICLR 2025! (lnkd.in/gw5gfVVA) From the paper chair: " I recommend Accept ...: (a) This is a production-grade framework that covers a wide range of parallelism method and .... is likely to have significant impact "

We've just released the first version of our Deep Learning Tuning Playbook! This is our attempt to distill our process for actually getting good results with deep learning. We emphasize hyperparameter tuning since it has been a large pain point. github.com/google-researc…

testing my new handle

Excellent article on vector processing on CPU and GPU. If you want to dive deeper into what's happening internally and the tradeoffs involved, this covers SIMD, SIMT, blocks, warps, and more: erik-engheim.medium.com/vector-process…

Trying out @zksync 2.0 zkEVM! uni.zksync.io Send some test tokens for 0x9495f4d2477511a8cf4e.

My trading strategy Software Stars is up +53% for the year so far! collective2.com/details/121783…

@IOHK_Charles This is a badge of honor. BitCoin went through the very same thing with Wikipedia in 2010 or so (for real!). Basically,imo this annoints ADA as the successor to BTC. That said, I'll go update the page tomorrow with some links/news to make sure it stays.

Verifying myself: I am silicontitan on Keybase.io. K_jka6m-OwMKZJX11cnIEsiEZzQb_OMyUTrZ / keybase.io/silicontitan/s…

#KarenTheSuperTrader = lost $50M in SPX Options, over-leveraged, and then used fraud? Over-leverage strikes again...bit.ly/2fOaiTL