Daniel Nishball

Github search needs some nudging. github.com/pytorch/pytorc… github.com/pytorch/pytorc…

The Nvidia Tensor Core is the most important evolution of computer architecture in the last decade We explain why / how it's evolved Shout out to collaborators @bfspector @tri_dao @colfaxintl @charles_irl @ia_buck Neil Movva Jonah Alben esp @simonguozirui for the cutest cover pic

Full Speaker List: Mark Saroufim of GPU Mode @marksaroufim Vijay Thakkar of Nvidia CUTLASS @__tensorcore__ Horace He of Thinking Machines @cHHillee Philippe Tillet of OpenAI Tri Dao of TogetherAI @tri_dao

Speakers include Philippe Tillet of OpenAI, @tri_dao of TogetherAI, @cHHillee of Thinking Machines, and more. Sponsors: @togethercompute , @LambdaAPI , @googlecloud, @nvidia, @GPU_MODE, @thinkymachines, @OpenAI, @PyTorch, @CoreWeave, @nebiusai. Apply now! semianalysis.com/hackathon-2025/

SemiAnalysis is hosting an Nvidia Blackwell GPU Hackathon on Sunday March 16th. It is the ultimate playground for Blackwell PTX tech enthusiasts, offering hands-on exploration of Blackwell & PTX infrastructure while collaborating on open-source projects.

Here's the links for my 5-hour conversation on the future of AI with @dylan522p and @natolambert: YouTube: youtube.com/watch?v=_1f-o0… Spotify: open.spotify.com/show/2MAi0BvDc… Podcast: lexfridman.com/podcast

@thoefler @SEC_ETH @NUSingapore @NTUsg @ASTARsg @sgSMU @SwisschamSG I'm in Singapore - want to connect?

Just set up office near CREATE @SEC_ETH in Singapore to edit the Ultra Ethernet spec 😀 I'll be here the next three weeks to establish new local contacts. So far I have 32 meetings and 9 talks scheduled, including @NUSingapore, @NTUsg, @ASTARsg, @sgSMU, @SwisschamSG, and others

BG2. AI Compute Landscape ‘25-26, scaling intelligence, chip competition, memory tech, inference time reasoning & more. 👊💥@altcap @bgurley feat @dylan522p

Met with @LisaSu today for 1.5 hours as we went through everything She acknowledged the gaps in AMD software stack She took our specific recommendations seriously She asked her team and us a lot of questions Many changes are in flight already! Excited to see improvements coming

MI300X vs H100 vs H200 Benchmark Part 1: Training CUDA Moat Still Alive Our 5 month journey conducting independent analysis + benchmarking User Experience, Usability GEMM + attention InfiniBand vs Spectrum-X vs RoCEv2 Ethernet SHARP Total Cost of Ownership semianalysis.com/2024/12/22/mi3…

Happening now - the Thrilla on Chinchilla Latent Space LIVE! - Best of 2024: Startups, Vision, Open Src, Reasoning... youtube.com/live/wT636THdZ… via @YouTube

5/5 - Read all about AI Neocloud economics, anatomy and cluster setup in our AI Neocloud Playbook and Anatomy Article: semianalysis.com/p/ai-neocloud-…

4/5 - Blackwell's impact is another important factor. How many B100+B200s hit the Neocloud market? Will GB200 NVL36/72s be readily available in 2025 given liquid cooling tightness? How will Neocloud Giants set GB200 NVL72 debut rental price?

1/5 - H100 SXM on-demand rental prices cut by 30% in the last two months. Today, you can rent H100 SXMs on-demand for as low as $2.99 per hour. What's going on?

1/4 - Chevron deference has just been struck down by the US Supreme Court. What is it and how does it affect semiconductor companies? ⬇️ Under that 40-yr-old legal doctrine, US federal agencies had the power to create their own rules & regulations when a law is ambiguous. In our industry, this is particularly relevant for technology export controls – agencies were in the driver’s seat and didn’t have to worry about being challenged in Court. This is now over, and the power has been handed back to the Court system after the Supreme Court’s ruling in Loper Bridge Enterprises v Raimondo.

@r_bjerg @FredaDuan Freda's math is correct. H100s can't just run on their own - they need to be in a server with a CPU and various networking chips which all take up energy. Back end networking for GPU clusters - switches, optical transceivers etc also add power requirements.

@FredaDuan One H100 has a peak power usage of 700W=0.7KW=0.0007MW. Therefore 600,000 H100 chips use around 600,000 x 0.0007 = 420MW, so not 1MW as you wrote.

The AI-driven energy bottleneck seems real: US data center power usage accounts for 22GW, or 4.5% of the nation's power consumption today. According to SemiAnalysis, it is projected to reach 100GW, or nearly 20% by 2030 due to AI buildout. Interesting data points from the interview: "Right now, the AI data centers are 50MW or 100MW. Then you get into building a data center that's…1GW, but I don't think it (1GW) will happen next year" To put into perspective, 100MW dc = ~60k H100 1MW dc = ~600k H100 Assuming each new generation of the AI model is ~10x more FLOPs (assuming a 3x increase in parameters and a 3x increase in training tokens). This scaling suggests that reaching a 1GW data center capacity may actually occur very soon (within one or two model gen, or 1-2 years). Potential Solutions and Implications: Nuclear Power Co-location: The most straightforward solution. One plant is ~1GW. It still doesn't solve the problem for subsequent model generations (potentially 10x power needs). Gas Plants: Viable option given abundant supply, but the lead time can be 4 years, with transmission capacity being a significant limitation. Rising Electricity Prices: According to Morgan Stanley, for every additional 1GW of load, power prices increase by $4/MWh. For ERCOT North ($45/MWh), a 1GW increase could raise prices by about 9%. Long-term solutions: Distributed cross-datacenter training? Nuclear fusion? Paradigm change in GPU energy efficiency?? h/t SemiAnalysis:

@FredaDuan Other solutions could include seeking out stranded renewable resources - wind farms and such, repurposing inefficient old enterprise data centers > 1.6 PUE into AI DCs at 1.3 PUE and below.

Peeling away the layers of the 30x Performance Gains Onion semianalysis.com/p/nvidia-black…