Spent most of my time @icmlconf 2026 in beautiful Seoul, attending talks on evaluation of LLMs and LLM agents. Here are a few papers and talks that I found particularly interesting:
Evaluating Robustness of Reasoning Models on Parameterized Logical Problems
arxiv.org/abs/2602.12665
→ Evaluating reasoning beyond static benchmarks.
The Obfuscation Atlas: Mapping Where Honesty Emerges in RLVR with Deception Probes
arxiv.org/abs/2602.15515
→ Understanding honesty and deception in reinforcement learning.
Understanding Reasoning Collapse in LLM Agent Reinforcement Learning
openreview.net/forum?id=4U7oG…
→ Why RL-trained agents can lose reasoning ability.
Causal Inference with Transformer Models (Invited Talk by Susan Athey)
icml.cc/virtual/2026/i…
→ A thought-provoking perspective on transformers for causal inference.
Rare Event Analysis of Large Language Models
arxiv.org/abs/2602.06791
→ Focusing on rare failures instead of average-case performance.
My main takeaway is that evaluation is steadily moving beyond leaderboard metrics toward understanding when, why, and how models fail in realistic settings.
#ICML2026
English
