Alex

Agentic Confidence Calibration 📄 Paper: bit.ly/4kbjgs3 AI agents are overconfident when they fail. Existing calibration methods assess only the final output, but agent failures often stem from earlier missteps masked by high confidence at the end. Holistic Trajectory Calibration (HTC) diagnoses the full execution path, extracting 48 process-level features across four categories: cross-step dynamics, intra-step stability, positional indicators, and structural attributes. A lightweight linear model maps these to calibrated confidence scores. 🔑 Key results across 8 benchmarks, multiple LLMs, and diverse agent frameworks: → HTC-Reduced achieves 0.031 ECE on HLE, down from 0.656 (verbalized confidence) → Outperforms LSTM, Transformer, XGBoost, and Gaussian Process baselines while showing dramatically lower variance in small-data regimes → Works across GPT-4.1, GPT-4o, Deepseek-v3.1, Qwen3-235B, and open-source alternatives → Architecture-agnostic: consistent gains on both smolagents and OAgents frameworks 🔍 The most predictive failure signals are task-dependent. For knowledge QA, features distribute across dynamics, stability, and position. For complex reasoning, positional features (first/last step confidence) dominate. Across all tasks, a diagnostic hierarchy emerges: positional signals serve as primary alerts, while stability and dynamics features complete the picture. 🔄 Transferability: a calibrator trained on SimpleQA transfers to HotpotQA and StrategyQA without retraining. A General Agent Calibrator (GAC) pretrained on 7 diverse datasets achieves the best calibration (lowest ECE of 0.118) on the held-out GAIA benchmark, zero-shot. Authors: Jiaxin Zhang @jxzhangjhu, Caiming Xiong @CaimingXiong, Chien-Sheng Wu @jasonwu0731 #FutureOfAI #EnterpriseAI #AgenticAI

$MU AMA: The Best Two Questions (Related) "How could Micron avoid the age old cycle of capacity expansion-over supply-ASP collapsing?" @yizheng95 "It seems natural that acceptable P/E ratios will expand. What are your thoughts on this?" @melone3710 My Refined Answer: Micron is transitioning from cyclical commodity memory (DRAM/NAND) to a premium, AI-centric platform with HBM, CXL, and SOCAMM, brand new products that never existed before. This gives far more wafer-allocation flexibility, flattening booms/busts and justifying PE expansion beyond historical foward single-digit valuations. AI are much more memory hungry than any other tech invention and right now it's mostly enterprise demand. The consumer demand hasn't even kicked in, YET and with rate cuts in the next few years, the consumer demand will also go up. Why Now Different? More Robust Porfolio Legacy: DRAM + NAND → Consumer + Enterprise mix → Limited flexibility, sharp cycles. New Reality: HBM (high-bandwidth for AI GPUs) + CXL (memory pooling/expansion) + SOCAMM (efficient AI-server modules) → Heavy enterprise/AI focus → Much higher margins and agility to shift production. Broader portfolio lets Micron redirect wafers from softening areas (e.g., consumer NAND) to Regular Enterprise NAND or HPC HBM AI without big ASP hits → More stable earnings. Thesis: Cycle Flattening = Valuation Upgrade: More memory for you but also for 8 billion people on earth: Your lifelong AI companions or agents will need to remember decades of inquiries so when you ask about a travel or a problem, they should be able to access your travel inquiry made 15 or 30 years ago in a similar location and formulate the best suggestion based on all the context available (memory). Now multiply that by 8 billion people on earth. AI Inference will not just demand more memory but it will demand a unprecedented transformation in the memory industry overall. What's up with HBM? -HBM TAM is exploding: ~$35B in 2025 → ~$100B by 2028 (40% CAGR, two years faster than prior forecasts). -Micron's entire 2026 HBM supply is sold out (including HBM4 in volume production early), with multi-year contracts locking in pricing/visibility. -HBM consumes 3-4x times more wafer per bit. -Memory is hardware and not software so we can't make more memory in just weeks. We have to build new fabs and that can take 5 to 10 years. So good luck waiting for 3-4x more fabs "all of the sudden." Like every real AI enablers, 20–25x+ forward P/E isn’t just acceptable; it’s inevitable.