George Morgan

We set a new ARC-AGI-2 SotA: 85.28% using an Agentica agent (~350 lines) that writes and runs code.

This is a really thoughtful reflection. I didn’t intend to watch the whole thing, but I ended up doing it anyway. AI is like playing a hard game you can’t beat with cheat codes on. It’s amazing at first, but it becomes boring very quickly. But worse than that, it does something to your brain that ruins the game. If you turn the cheat codes off, you become acutely aware that you’re now struggling unnecessarily. You can’t forget how easy it was, but you don’t want it to be that easy because it takes all the fun out of it, but now the inability to unsee what you’ve seen creates a tension that causes you to lose interest in even continuing to play. The magic is gone. You’ve broken the spell. AI is doing this to life. And the societal consequences are going to be enormous.

1. Yes it sees each game only once. 2. It is not yet human level. We observe that it performs fairly close to human baseline on almost all of the levels but it seems to get stuck on a few of them (as shown in this video) before eventually recovering. This totally blows the action budget. Our scores are below. We haven't yet optimized the harness. I am confident that we can drastically improve the performance before the rest of the puzzles come out. We will be sure to rerun it and release our official final scores publicly when they do! • ft09, 344 actions, 39.15% • vc33, 2092 actions, 42.87% – L5: 1604 actions vs 92 baseline • ls20, 3703 actions, 69.77% – L7: 3240 actions vs 82 baseline

We have now solved all publicly available ARC-AGI-3 puzzles.🧩

👀

for some reason this image keeps going viral, the good news is that it is in a *much* better shape right now, it will (probably?) not catch fire, and it will be serving something really cool for you really soon™ (no it is not for LLMs; matmuls are banned from my cluster)