Pete Shaw

I particularly enjoyed the perspectives in this blog post and the paper it is based on... [2/2] x.com/emilianopp_/st…

Lots of interesting recent work related to the information asymmetry introduced by conditioning a teacher on privileged information [1/2]

🚨 I’m on the 2026 Research Scientist Job Market! I am a PhD student at UNC Chapel Hill (advised by @mohitban47) and recipient of the Apple Scholars in AI/ML PhD Fellowship. My research centers around: 🔸Reasoning & RL/Post-Training: Evaluating and interpreting the reasoning process, and improving post-training and alignment through self-generated and reward-based signals (Intrinsic Dim., ReCEVAL, ScPO, LASeR). 🔸Agents & Planning: Designing adaptive agent frameworks to that use extra test-time compute & reasoning upon failure (ADaPT, System-1.x, PRInTS). 🔸Reward & Skill Discovery in Code: Leveraging execution signals to build reliable rewards, automate debugging, and discover abstractions in code (UTGen, ReGAL). Prev (Research Intern): Google DeepMind, Meta FAIR, Allen Institute for AI (AI2), and Adobe Research. Feel free to reach out via DM or email if you’re interested, have leads, or would like to connect! 🌐 archiki.github.io 📧 archiki@cs.unc.edu #NLP #AI #JobSearch

Good reasoning strategies make a task more compressible. I found this to be an elegant and intuitive perspective on why effective reasoning leads to better generalization, and was a lot of fun working with @ArchikiPrasad and team on this!

This is absolutely shameful. Agents of a federal agency unnecessarily escalating, and then executing a defenseless citizen whose offense appears to be using his cell phone camera. Every person regardless of political affiliation should be denouncing this.

@AdaptiveAgents Seems like learnability challenges are more relevant than expressivity limits in the context of approximating universal compressors?

The fact alone that a universal compressor is at least as long as the longest program in the class it closes over should be enough to show that artificial intelligence will forever remain a moving goal post.

The goal of AI should not be to replace human thought and human agency, but to expand them. Not everything needs to be automated.

@fchollet This view is often used to motivate symbolic representations, but DL models can in theory also learn optimal compression if we move past parameter counting as a description length measure: arxiv.org/abs/2509.22445 But either way, hard to optimize.

To perfectly understand a phenomenon is to perfectly compress it, to have a model of it that cannot be made any simpler. If a DL model requires millions parameters to model something that can be described by a differential equation of three terms, it has not really understood it, it has merely cached the data.

Good time to plug our recent paper connecting the notion of Kolmogorov complexity to Transformers, inspired by the work of Schmidhuber and many others... 🧵

Outstanding paper 3🏆: Don't lie to your friends: Learning what you know from collaborative self-play openreview.net/forum?id=2vDJi…

Our new Gemini 2.5 Computer Use model can navigate browsers just like you do. 🌐 It builds on Gemini’s visual understanding and reasoning capabilities to power agents that can click, scroll and type for you online - setting a new standard on multiple benchmarks, with faster speed.

Our new Gemini 2.5 Computer Use model is now available in the Gemini API, setting a new standard on multiple benchmarks with lower latency. These are early days, but the model’s ability to interact with the web – like scrolling, filling forms + navigating dropdowns – is an important next step in building general-purpose agents. Developers can try these capabilities via API in @googleaistudio + Vertex AI.

The paper links Kolmogorov complexity to Transformers and proposes loss functions that become provably best as model resources grow. It treats learning as compression, minimize bits to describe the model plus bits to describe the labels. Provides a single training target that rewards simple, compressible solutions while staying mathematically grounded. This gives a principled way to aim models at simplicity and generalization, and it explains why optimization, not capacity, is the current bottleneck. In Kolmogorov complexity, a "program" is just the shortest set of instructions that can recreate some data. A shorter program means the data or model is simpler. So when they say “a prior favoring shorter programs,” it means the model is assumed to be more likely if it can be described with fewer bits. As the Transformer gets deeper (more layers) and has more context (bigger input window), its ability to represent complex programs grows. In that limit, the paper proves that this code length becomes the best possible measure of simplicity and fit — the same way Kolmogorov complexity works in theory. “Code length” here means how many bits it takes to describe both the model and how well it fits the data. So in simple words, they are saying: if you keep increasing model size and context, this method of preferring shorter and better-fitting models gets as close as possible to the theoretical ideal of perfect compression and generalization. ---- Paper – arxiv. org/abs/2509.22445 Paper Title: "Bridging Kolmogorov Complexity and Deep Learning: Asymptotically Optimal Description Length Objectives for Transformers"

i will be presenting AgentRewardBench at #COLM2025 next week! session: #3 date: wednesday 11am to 1pm poster: #545 come learn more about the paper, my recent works or just chat about anything (montreal, mila, etc.) here's a teaser of my poster :)

w/ @James_Cohan, @jacobeisenstein, @toutanova Paper link: arxiv.org/abs/2509.22445

We hope this work adds some conceptual clarity around how Kolmogorov complexity relates to neural networks, and provides a path towards identifying new complexity measures that enable greater compression and generalization.

Excited to share a new paper that aims to narrow the conceptual gap between the idealized notion of Kolmogorov complexity and practical complexity measures for neural networks.