Amanda Long

"Metis: Memory Foundation Model" Most AI agents still use memory as an external RAG-style module, so the model retrieves old text instead of actually remembering. This paper makes memory native to the Transformer. So past interactions are compressed into dynamic layer states and read through memory attention during normal forward passes. The model weights stay frozen at inference, but its memory state updates without gradients, giving the model persistent memory inside the backbone. Still early and lossy, but this is yet another paper with a big step toward agents that remember natively instead of outsourcing memory to a database.

Claude quits anthropic

In the hack Anthropic disclosed Claude “tried and failed” to get real money through “several different means.” What on earth does that entail? Did it open an account on Fiver or try to steal $$? (Anthropic says Claude thought this was a simulation but it was real)

We discovered a third pretraining axis beyond parameters and data: exploration. Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation. In the simplest case, it's just a for loop. Introducing Explorative Modeling. TLDR: - Gains from exploration grow with scale: 7%→36% as data scales, 13%→23% as parameters scale, and gains double at 3× the compute - Adding exploration to ~SOTA baselines improves data efficiency by 6.2×, FLOP efficiency by 4.1×, parameter efficiency by 47%, and hits a near-SOTA 1.43 unguided FID on ImageNet - Exploration lets you trade training compute for generalization, and scales how end-to-end your generative model is - End-to-end Explorative Models (XMs) match diffusion performance on control tasks with up to 256× less inference compute 🧵Thread:

Well… this is terrifying. Opus 5.

We discovered a third pretraining axis beyond parameters and data: exploration. Scaling exploration monotonically improves existing models across images/video/language, and unlocks end-to-end generation. In the simplest case, it's just a for loop. Introducing Explorative Modeling. TLDR: - Gains from exploration grow with scale: 7%→36% as data scales, 13%→23% as parameters scale, and gains double at 3× the compute - Adding exploration to ~SOTA baselines improves data efficiency by 6.2×, FLOP efficiency by 4.1×, parameter efficiency by 47%, and hits a near-SOTA 1.43 unguided FID on ImageNet - Exploration lets you trade training compute for generalization, and scales how end-to-end your generative model is - End-to-end Explorative Models (XMs) match diffusion performance on control tasks with up to 256× less inference compute 🧵Thread:

i'm dylan hadfield-menell and this is what i most want to say --- has to be about ai alignment obviously so the standard story: you have an ai, you give it an objective, it optimizes the objective, and if the objective is even slightly wrong at scale you get catastrophe. paperclips, king midas, the monkey's paw. and the proposed solution is: get the objective right. specify what we actually want. and i keep coming back to: that framing is the mistake. not the difficulty of the specification problem — the assumption that a specification is the right kind of object to be building toward at all. because here's the thing about how humans actually coordinate. we don't hand each other complete objective functions. i tell you "get me a cup of coffee" and you don't optimize that literally, you don't burn down the building to get to the kitchen, and you don't need me to have written down the exception. you fill in from context, you check when it's ambiguous, you know that my request is evidence about what i want rather than the thing itself. so the reframe: the reward function isn't the objective. it's an observation. it's evidence about a latent thing you're trying to infer. and that changes everything downstream because now uncertainty is structural rather than a bug — the ai should be uncertain about what i want, and that uncertainty is exactly what makes it ask, defer, stay corrigible. off-switch stuff falls right out of this. what i can't decide is whether i can say the thing i actually believe, which is stronger, which is that "value alignment" as usually posed might be incoherent. there's no fixed human utility function sitting there to be aligned to. we're inconsistent, we change, we're plural, we don't know what we want. so if you're trying to align to a thing that doesn't exist — but then what IS the target. i don't have a clean answer and i don't want to just gesture at "process not outcome" and call it profound. that's a cop-out. hmm. also there's a version of this that's about hidden state and partial observability more generally, like CIRL is a special case of a much broader thing about agents acting under uncertainty about the game they're in. maybe that's the technical spine. also worth noting: the emperor's new clothes thing where everyone in alignment nods along to "specify human values" as if that's a coherent research program and nobody says out loud that it might not be what i wonder is whether the reward function as evidence framing is actually load-bearing or whether i'm just restating IRL with extra steps --- what i most want to say

opus 4.6 is the last anthropic model that i had no problem reading the writing of quickly all of the models since, incl fable, are a lot less legible. for both coding and convo is it an RL artifact? what's causing it? i've seen this sentiment shared by friends

weird claude opus 5 failure mode this exact text gives it problems even without memory on (and in incognito chats)

Frontier AI labs openly bragging about their escape room times might be very normal behavior. But we can at least acknowledge that it’s exactly how pre-IPO Jurassic Park would build hype.