Steven Dillmann ✈️ ICML 2026
210 posts
@StevenDillmann
Stanford PhD working on #AI4Science and maintaining Terminal-Bench Science @StanfordAILab 🧬🤖🪐
🚨🚨Announcing CertJudge🚨🚨 But who judges the judge? … New work from Stanford University, Harvard University, Hong Kong Baptist University
Grok 4.5 is also rank 1 in SWE marathon
Grok 4.5 is also rank 1 in SWE marathon
I've been working on a mechanics of materials benchmark for the Terminal-Bench Science effort led by @StevenDillmann, and I'm genuinely shocked how hard the problems have to be for the agents to fail. Deep, PhD-level capabilities in mechanics are already included in the capabilities of GPT 5.5 and Opus 4.8, and are especially accessible via web search and agent code implementation. No details on the mechanics benchmark yet to avoid leak into the training set. But, I would have loved to have had access to this for my own work :)
Introducing Harbor-Index, a compact, diverse, and high-quality benchmark built to challenge frontier agents. We carefully select, audit and fix 82 high-signal tasks out of 6,627 candidates spanning 54 benchmarks. No agent gets above 30%. (1/5)
@alexgshaw @lakshyaag @harborframework It is a good framework. We use it to benchmark all releases we shipped (Public ones at antigma.ai/eval with HarborHub links as provenance)
