the tiny corp

using tinygrad and have comments/questions/bugs? join our discord! almost all company stuff is public there too discord.com/invite/ZjZadyC…

@kneeanderthul How do you connect the Mac to the tinybox? 10 gigabit ethernet is an option, but it doesn't get those 120 Gbps speeds.

@__tinygrad__ You wouldn't need that: M3 Ultra M4 Pro / Max M5 Pro / Max All come with TB5. It's just a matter of ensuring you buy the right models at this point If other companies add TB5 to their devices, we could potentially have mixed share vram across multiple devices over TB5

If we can tunnel PCIe over USB3, why not also tunnel it over Ethernet? extra/remote/serve.py on remote, REMOTE=<ip> on local.

@kneeanderthul I wonder if you could plug one of these into a tinybox red? It has an open PCIe slot. Super high speed MacBook link! amazon.com/ASUS-Thunderbo…

@__tinygrad__ Phenomenal!!! I've been thinking of using a few M3 with 512gb of ram to load a huge model and getting a Red box v2 to pass over the computing over TB5 I think I saw someone do it with Exo Who knew Vram could be so fun

@AIFlow_ML ugh then I have to plug in a wire. this is Wi-Fi

@__tinygrad__ Why not Thunderbolt 5 ?

@andreiofstan github.com/tinygrad/tinyg…

@__tinygrad__ I have thought about this for a long time, what is it actually forwarding, the TLPs? How does BAR mapping work in this case

@kneeanderthul Oh we do that too. Once you have the driver, if you can tunnel PCIe through it, it just works!

@__tinygrad__ This whole time I thought y'all were running eGPUs over TB5 ☠️

People are too focused on trying to build God and not thinking about the unit economics 🤑

@fzimmermann89 @fw7th These are the internals. This is the entire IR across the whole lowering process.

@__tinygrad__ @fw7th Is there a design overview of the internals? The spec is great to use Tinygrad, but not to understand how it works compared to torch or jax, and where to look for stuff..

today, I wanted to contribute to tinygrad but got confused as to what the fuck I was reading. So, I embark on a project - will be documented and hopefully improve my low-level ML side;

@Jonathan_Blow lol sam altman posted an ad for his company. do you remember having to fill out your taxes line by line? I barely remember now that I have TurboTax!

This is such a *completely* different reality from where I live, at this point it's just difficult to say anything meaningful about it at all. x.com/sama/status/20…

@SemiAnalysis_ We love to see NVIDIA commoditizing the model tier!

The West is now more commie than even china. NVIDIA is creating an committee for pretraining open weight models meanwhile China has a flourishing amounts of competitive startups pretraining OSS models

@experimetal 405B *is* middle class

@__tinygrad__ Explain why this matters to the GPU middle class

Llama 405B is really only 10 Tensors

@LenSeaside this displeases Muon

@__tinygrad__ Which is one tensor really. Along the file dimension.

@norpadon oh cool, merging w1 and w3. benefit is fewer tensors to reason about. norm can't be fused during training, right?

@__tinygrad__ And rmsnorm scales can be fused into the weights of the next linear layer

@comma_ai comma 4 in stock spend-to-send?

Big announcement tomorrow. Rhymes with lend-to-lend.

@n6bbo SIP sorted for NVIDIA, for some reason they didn't give us the AMD one. It's a super annoying process but I think we'll get it. And our eGPU board (ships Q2) supports full power toggling to the GPU, so no reset issue there.

@__tinygrad__ Did you get the SIP sorted and what is the status of this NVIDIA 50xx reset bug?

Mac Mini + eGPU. Both NVIDIA and AMD supported.

@Thorium_Labs Yes, that card is supported, it's basically a 5090 with more RAM. Bandwidth is USB4, 40 Gbps.

@__tinygrad__ Wow! Very interesting. What about the bandwith? Any bottlenecks? Does it support a RTX 6000 Pro Blackwell?

@robertocarrizo @beffjezos umm in the repo ... and over the last 2 years

@__tinygrad__ @beffjezos where? when

@beffjezos We have full pure Python user space drivers for AMD and NVIDIA in tinygrad. USB4 devices can be mmaped like they are directly on the PCIe bus. This isn't hype, it all works today on any 3000-5000 series NVIDIA or RDNA3/RDNA4 AMD.

@__tinygrad__ Wait, how?