Ashish Tuli

The useful signal here is not one packaging winner. It is that AI silicon customers are starting to value optionality at the package layer.

TokenSpeed reporting up to 580 tok/s on Qwen3.5-397B-A17B is a useful signal, but not just because the number is high. The interesting part is how the speed was achieved: less copying, more kernel fusion, and better overlap between the CPU and GPU so the accelerator spends less time waiting. Agentic workloads change the inference problem. Multi-turn context, tool histories, and long state make latency a systems question, not just a model or accelerator question.

OpenAI's Tax AI case study is easy to read as a tax automation story. I think the more important signal is the operating loop. Every expert correction becomes a production trace. Recurring traces become evals. Evals become bounded engineering tasks. Those tasks ship only after regression checks. That is a very different pattern from “AI gives an answer and a human checks it.” It is a way to turn expert judgment into compounding product improvement. For vertical AI, the first model answer matters. The learning loop around the model may matter more.

Agent UX is becoming an inference systems problem. Context ingestion, cache reuse, and decode, each have different bottlenecks.

The HBM bottleneck is no longer just bandwidth or supply. It is the heat path. SK hynix’s iHBM puts integrated cooling elements inside the HBM package, directly in the D2D PHY region where HBM connects to the accelerator, and says the design reduces thermal resistance by 30%. That matters because HBM scaling is moving in two directions at once: taller stacks and faster interfaces. Both increase power density in places traditional system-level cooling does not reach cleanly. The interesting shift is where the fix now lives: not around the rack, but inside the memory package.

The goal in AI infrastructure is tokens per dollar. One of the constraints now determining it is tokens per watt. As racks move from tens of kilowatts toward hundreds, 54V power delivery stops being a background choice. Lower voltage means much higher current, and current is what drives copper mass, heat, voltage drop and wasted space. Moving from 54V to 800V cuts current by roughly 15x. Because resistive losses scale with current squared, the power-chain math changes fast. That is why 800VDC is not just an electrical upgrade. It is a way to move the power-delivery ceiling higher so dense AI racks can keep scaling. Really good @SemiAnalysis_ article on this: x.com/SemiAnalysis_/…

Agent safety will not be solved only in the prompt layer. As agents start taking actions, the control plane moves to policy, identity, auditability, and tool execution.

UPDATE: Came up with an even better version of this prompt after the feedback Ask Codex to look across your sessions, Memories, and Chronicle, identify patterns, reuse what already exists, and only create the smallest useful skill, subagent, or automation. "Look back over my recent work from the last 30 days, or all available history if shorter, and identify repeated manual workflows worth packaging. Use available evidence in this order: - Recent Codex sessions and task summaries. - Codex Memories and rollout summaries to find patterns repeated across sessions. - Chronicle, if enabled, to spot repeated work outside Codex. Use Chronicle for discovery only; confirm important details in the relevant source system when possible. - Existing skills, custom agents, and automations, so you reuse or extend what already exists instead of duplicating it. Look broadly for work that is repeated, time-consuming, error-prone, context-heavy, or benefits from a consistent process. Include workflows across coding, research, writing, planning, communication, operations, analysis, and personal administration. Only act on a candidate when it: - occurred at least twice, or is clearly likely to recur and costly to repeat; - has stable inputs, a repeatable procedure, and a clear output or stopping condition; - would materially improve speed, quality, consistency, or reliability; - is not already adequately covered. Choose the smallest appropriate form: - Skill: a reusable workflow or playbook. - Custom subagent: a bounded specialist role or investigation task suitable for delegation. - Automation: a scheduled or recurring check, report, reminder, or monitor. - Skip: work that is too one-off, ambiguous, sensitive, or poorly evidenced to package. First produce a compact shortlist with: - repeated workflow - supporting evidence and dates - frequency/confidence - recommended form: skill, subagent, automation, extend existing, or skip - why it is or is not worth creating Then create only the high-confidence missing items. Keep them narrow, practical, source-aware, and easy to validate. Do not create speculative, overlapping, or overly broad assets. Finish with: - what you created or extended - what you deliberately skipped - what needs more evidence before packaging"

@cytrusf1 We need a downvote button for a post, not just comments.

He can't deal with adversity. When things don't go his way he lashes out with unnecessary anger and borderline violence.

True!

@F1 What a race indeed!

KIMI ANTONELLI WINS IN MONTREAL!! 👏🏆 WHAT A RACE! 🤯 #F1 #CanadianGP

Coding agents are exposing a critical infrastructure sizing problem. @SemiAnalysis_ measured 432k real coding-agent requests. Median input was 96k tokens, roughly 3x what much of today's AI infrastructure was sized for. Nearly half already exceed 128k. The important part is what those tokens are. Not users writing longer prompts. The agent harness is assembling a working set: system prompts, tool definitions, MCP schemas, skills, prior turns, repo structure, file contents, retrieval, and tool state. That changes the workload. 3x the context is not a 3x cost problem. In long-context inference, prefill becomes the bottleneck, and standard dense attention compute scales quadratically with sequence length. That is the accelerator-side problem. The CPU-side problem is everything around the model call: tokenization, request scheduling, cache management, tool execution, API calls, permission checks, and validation loops. A chat request is mostly prompt in, tokens out. A coding-agent request is a distributed systems workflow wrapped around inference. That is why the CPU:GPU ratio debate matters. Not because GPUs matter less. Because agents increase the CPU, memory, and orchestration required per GPU. The real issue is not that agents are "a little more expensive" than chat. It is that infrastructure built for chat is structurally undersized for agents.

The interesting question in EMIB vs. CoWoS is not which packaging technology is “better.” It is what each architecture makes easier or harder as AI accelerator packages get larger. Once a single die is no longer enough, packaging stops being back-end assembly and becomes system architecture. That is why the comparison is not just about interconnect density. It is about how multi-die systems scale around reticle limits, warpage, yield, utilization, and cost. Must read primer from @austinsemis: chipstrat.com/p/advanced-pac…

AI credits are becoming a go-to-market strategy. OpenAI’s YC offer made the pattern explicit: tokens for equity. Hyperagent’s $10M Founding 500 program is the lighter version: credits to seed the next wave of agent-first builders. This is not just “startup support.” It is distribution. Cloud credits helped shape where startups built their infrastructure. Token credits may do the same for AI platforms: get founders building, testing, debugging, and scaling inside your stack before defaults harden. In AI, usage is the new land grab.

@awaiting_ai Yes.

@ashishtuli +

“Fabless” used to mean asset-light. Now it increasingly means prepaying, co-investing, and locking up years of wafer and advanced packaging capacity just to secure supply. You may not own the fab, but you are still paying to make sure it exists for you. The label is still “fabless.” But it is not truly asset-light anymore.

@kinsonwu08 Exactly, and this exposure isn’t just an AMD problem, it’s the reality for every major fabless company.

@ashishtuli Correct. If AMD do not establish its own robust supply chain now, AMD will be dependent on others, especially by overly relying on Taiwan's industrial chain, which could become a fatal problem in the coming years.

@Buvisk007777 💯

@ashishtuli Real men have fabs

@Alex_Intel_ It’s just the beginning…

@ashishtuli 💯 CPU TAM growing so large, all of Intel's excess clean room space looks like a great decision now too