William Chen

I love inference! Tap in🧑🏻‍💻

duhhhh

people are underestimating the efficiency innovation in kimi k3's scale-up to 3T: percent of experts activated Nemotron 3 Ultra = 4.3% MiniMax M3 = 3.1% Inkling = 3.1% Kimi K2 = 2.1% Kimi K3 = 16 activated / 896 experts = ONLY 1.8% stable latentmoe down-projects tokens into a compressed latent space before routing. the router chooses 16 experts from the latent rep, plus 2 always-on shared experts, which means less data overhead. at 1.8%, quantile balancing counters the sparse expert activation by deriving allocation straight from router-score quantiles without tuning a sensitive hyperparameter this new architecture is converting each unit of compute into more intelligence rather than throwing hardware at the problem K3 scales ~2.5x more efficiently than K2. can’t wait for this to be applied to pretraining everywhere. exciting stuff!

Very interesting

We're excited to open-source AgentENV! Built for large-scale Agentic RL, AgentENV provides secure Firecracker-based execution environments while reducing infrastructure costs by up to 96.8%. Already powering the Agentic RL training of advanced models, including Kimi K3 from @Kimi_Moonshot. 📖 Learn more: kvcache.ai/blog/agentenv-… ⭐ Try it on GitHub: github.com/kvcache-ai/Age… #OpenSource #AI #AgenticRL #LLM

communication and networking is more important than kernels must read!!!! thanks @Kimi_Moonshot

@VoidAsuka taking bets

im so curious to see

i think we finally have enough clues to reverse-engineer ilya sutskever’s secret SSI research. my highest-probability guess: SSI has found a brain-inspired way to make an AI continually learn. today’s frontier models learn mostly during training. afterward, their core knowledge is largely frozen. they consume enormous datasets and still fail strangely when a problem falls outside their training. SSI may have an early system that can: • learn a new skill from very few experiences • recognize when an approach is failing before reaching the final answer • update itself without erasing old skills • transfer one lesson into completely different problems • keep learning after deployment think of it like a gifted teenager instead of a finished encyclopedia. it may not begin knowing every profession, but it could rapidly learn any profession. the evidence lines up almost too cleanly: ilya called poor generalization the fundamental limitation of current AI. models “generalize dramatically worse than people.” he described SSI’s target as a “superintelligent 15-year-old” capable of learning any job. when asked how to create human-like learning, he said there is a machine-learning principle he has opinions about but cannot discuss publicly. WSJ now reports that SSI’s secret research focuses on “overlooked aspects of how the human brain functions.” Nvidia received rare access to the research, made a substantial investment, gave SSI 10x more compute, and agreed to let SSI help shape future computing platforms. my technical guess: experience → internal judgment → self-correction → durable learning → transfer → repeat the internal judgment may be the brain-inspired component. humans do not wait until the end of a 10,000-step task to know they are failing. emotions, intuition and judgment provide constant feedback. ilya has argued that AI needs an equivalent internal “value function.” this could also explain SSI’s central promise: capability and safety trained together. the same mechanism that teaches the AI what works may also teach it what it should care about. my confidence is roughly 70% on human-like generalization plus continual learning, and 40% that an internal value system is the central mechanism. the simplest description: an AI whose intelligence compounds from experience, with its values learning inside the same loop.

!!!!

@adityazero_ great read

XLA’s strengths and its limitations are the same design decision. Freeze the shapes. See the whole graph. Allocate every buffer before the program runs. Fuse against a global cost model. That is exactly why XLA saturates real hardware and why it recompiles on every new sequence length, refuses data-dependent shapes, and gives you no easy way to say "fuse these two, not those." I wrote up a deep dive into what XLA does, using live IR dumps and source analysis. A few things I measured that surprised me: - The Compile Time: A 24-layer dense tower (360 HLO instructions) compiles on CPU in ~0.03s. A single 4096×4096 matmul takes ~2s on an A100. Because the backend is autotuning it. "XLA is slow to compile" is almost always a story about autotuning and per-shape specialization, not graph size. - The CI Gap: XLA ships Blackwell codegen (sm_100, sm_103), yet the only NVIDIA hardware in public CI is an L4 and an 8×H100 node. Meanwhile, AMD self-hosts presubmit CI on current-gen CDNA4 (gfx950) for every PR. The multi-device H100 workflow? It’s cron-only, and failed 52% of the time in its last 100 runs. The compiler is excellent, but the bargain is real. If you depend on XLA, you need to pin versions, run your own tests, and learn to read HLO. hiraditya.github.io/posts/xla-revi…

@aleks_sharik jealoussss

I’ve added the next generation of Instinct to my office display MI100 thru LEGO Helios 😁😅

Time to scale that SSI:

published a blog that covers HBM meant to be foundational lmk if theres any inaccuracies! I WILL EDIT I want this to be useful for anyone touchdown-labs.com/blog/how-hbm-w…

Jensen Huang: “The semiconductor industry is probably going to have to be 10x larger than it is today.”

Looks at all of those decoupling caps! So dense? They need an entire reel of 0201 caps to assemble just one board I think.

Around 1500 MLCCs in this photo

딥시크가 신모델을 내면서 엔비디아만 쏙 빼놓고 알려줬음 세미애널리시스가 공개 첫날 무슨 일이 있었는지 짚었음. 딥시크는 이번 V4 프로에서 모델 안쪽 치수 하나를 예전과 다르게 바꿨음. 업계가 당연히 고정값이라 여기던 자리였음. 그래서 공개 당일 엔비디아 쪽에서 제대로 안 돌아갔음. 이유가 허탈함. vLLM이나 SGLang 같은 오픈소스 실행 도구들은 비밀유지 조건으로 모델을 미리 받아봤음. 엔비디아는 못 받았음. 출발선이 달랐던 거임. 같은 날 화웨이는 반대로 움직였음. 자기네 칩에서 딥시크를 어떻게 빠르게 돌리는지 중국어 영상으로 당일에 풀어버림. 패널들이 그걸 보고 "화웨이가 한 대로 따라 하면 첫날부터 성능 나온다"고 함. 딥시크가 같이 공개한 고속화 코드가 검증된 하드웨어도 딱 둘, 엔비디아 GPU와 화웨이 어센드임. 칩을 판다고 그 위에서 뭐가 돌아갈지까지 정하는 시대는 지났음. 이제는 모델 만드는 쪽이 누구를 먼저 부를지 고른다는 뜻임.

@nikita_builds They all do Neoclouds

why isnt there an american company that just takes the chinese open source models and hosts them as a US company?

I’m going to modernize manufacturing in the US I’m going to make STEM fun in the US

Nah build a foundry

@brandonbbao 😂

Houston we might have a problem