Eric Quinnell

Levels should end asking you “is it dark outside? No looking at your watch”, after cabling and manually PXE booting for 16 hours in a row

@0xmer_ Also, more FLOPs. Even if you can’t use them.

Top 6 things we need in 2026: 1. Another DSL for ML kernels 2. Another article explaining FlashAttention 3. Another Neural Network from scratch in C 4. Another Arena Allocator 5. Another Game Engine from scratch in C 6. Another 400 RVV instructions

@itsclivetime And after your light warmup, questioned simultaneously by a panel of experts, then normalized and softmax’d for a single answer before the 2nd lap

i wonder what it feels like to be quantized, hadamard rotated, and speculatively decoded

@HarshVamja Samsung ExynosM5, AMD K8, IntelP6, Tesla Dojo v1/2, ARM A9/A15…all these could open source at this point to train the LLMs, but said tribes would have to release them…

@divBy_zero 💯 Also hallucinations are expensive in hardware. AI-generated RTL still misses corner cases in complex designs, which can be catastrophic in silicon. How is AI ever going to mine decades of undocumented tribal knowledge. :)

AI sucks at RTL for two big reasons: 1. Trained on github open source technical prose pieces and not production code 2. Assumes the EDA toolchain supports the whole System Verilog spec

@bertverrycken Claude. Not passing the “it compiles” outside the trivial, so doesn’t get a “quite good” from me

@divBy_zero Which AI, deploying claude code rn and it writes correct RTL and does the verification. Touch wood but it looks quite good at it.

@FelixCLC_ Yes, but what type of flops?

The matrix decomposition will continue until peak flops improve

@Arronwei3n This. Can’t say it enough, thx @NuttyCLD PS GJ on citing the quantum qbits interconnect issue. Cmos has gain to clean up SNR, ppl forget it’s not just a switch PPS no PAM8/10 pls, I beg you

"Today, $NVDA's latest GPUs can theoretically perform thousands of trillions of operations per second, but actual utilization in AI inference workloads is only 30-40%. The rest of the time? Waiting for data to arrive."

@mfbalin @SemiAnalysis_ @tri_dao Credit equally to @mfbalin specifically plus the Trainium and Neuron team at large, it required the whole stack to solve. (And really fun to fix.)

@SemiAnalysis_ @tri_dao We were pressed to address this and we ended up with a custom designed solution for it with @divBy_zero.

IMPORTANT: Blackwell Ultra solves an attention operation performance issue found in Blackwell. As @tri_dao presented at the SemiAnalysis Hackathon, one of the biggest bottlenecks in the core attention operation is not GEMMs but the softmax (exponential)  part. In Hopper & Blackwell, the exponential operation's throughput was too slow to effectively overlap with the 2 GEMMs in attention. Blackwell Ultra solves this by having 2x higher exp throughput than Blackwell  Note that other chips have this performance mismatch issue too such as Trainium3 having to increase their exponential function throughput by 4x compared to Trainium2.

@trishume Love the Zachtronics reference. TIS-100 style as a performance predictor. Really clever

I wrote a blog post on my journey trying to make a take-home that increasingly powerful models couldn't beat. We also released the original take-home for you to try, people tend to find it really fun! It's optimizing a parallel tree traversal on a simulated SIMD VLIW machine.

If the liquid boils somewhere along the way I suppose that clears out the biological contamination? But if liquid internal convection/conduction is used somewhere along the way internally, it’s a perfect biome for things to grow. Deployed datacenter liquid cooling has a while sub field on organics growing in the tubes, especially pure water. I’ve smelled it myself, it’s very gross. Said space plan needs some verbiage on the organics in liquid problem imo

don't know how much liquid would be needed but surely they're not going to grow too much in space, sort of a materials problem to find the best one for it. IIRC for radiation shielding they intended to have liquid water cause it's great at that, I'd assume whatever fluid you have for that is what you would use to distribute it to the radiation surface but that's not necessarily needed. Thermal capacity and boiling point vs pressure is probably the things you'd look through carefully to determine which. If the liquid boils you get a large increase in pressure.

Dojo1 was ultra ambitious and really pushed a lot of technologies (packaging, power delivery, system design, clock distribution, even number formats), but execution and focus could've been better. If Dojo3 is anything like the ideas floated around the office years ago it'll be nuts. Will be interesting to see if things move faster when Elon's focus is on the chips - gogogo and good luck!

Addendum: Transformers and the self attention architectures are notable and clever, but still mix may older ideas. My original point is that our infrastructure is truly new, yet breakthrough ideas are rare and many stem from the ages before us. Check the hubris of “new” AI

McCulloch-Pitts neuron 1943. Rosenblatt’s Perceptron 1958. Back propagation 1986. The ideas are old, the compute and the data are new.

@FelixCLC_ I.e. passed the “engineering” threshold and not the “scientific” one.

the problem with git (as deployed and used within most interfaces) is that it's clearly well passed the "good enough" threshold, but not clearly in the "great" threshold :/

@yacineMTB Nobody mentioned the most important part: 100pF is pronounced “One Hundred Puff Capacitor” Not joking, really truly that’s the lingo. Ask about resistor color bands sometime, it’s NSFW

is the capacitors in series (is that the right word) basically just there to smooth out any upstream spikes/brownouts? why is the ferrite bead there? why is the ferrite bead followed by a resistor? why those two resistors? can someone explain

@sleep_deprivado @itsclivetime @Adam_and_EVs @rzidane360 @soumyarooproy @ezhang887 @timzaman @Starcloud_ Read through it, thx, seems just solve with outrageous surface area. Brute force method. I saw no note of flushing the liquid cooling. Stuff grows in those pipes, it’s pretty awful. Maybe in sporadic maintenance they’ll flush it? But that requires carrying a ton of liquid

@itsclivetime @divBy_zero @Adam_and_EVs @rzidane360 @soumyarooproy @ezhang887 @timzaman @Starcloud_ Either way it's a problem of surface area. GPU/TPUs would be centralized probably... closed fluid lines move their concentrated heat out probably. Heat from panels can be efficiently transmitted to radiators thru thermal conductivity.. aluminum is great for that.

@itsclivetime @Adam_and_EVs @rzidane360 @soumyarooproy @ezhang887 @timzaman We don’t need (nor want) the attention. Will be thrilled to see Dojo-in-space and have taken some small part in the early, yet flawed, 1st genesis. Someone will have to explain to me the cooling in space part someday

@Adam_and_EVs damn we didn't make the list 😭 @divBy_zero @rzidane360 @soumyarooproy @ezhang887 @timzaman

@HellPrize @mu_chrinovic This. Only past 512 is it net benefit, which is not most processors. AVX did it right.

@mu_chrinovic However, there is no clear advantage compared to fixed-length SIMD instructions like AVX. The benefits aren't really apparent...

Scalable Vector Extension