Bad theory labs

If you're reading this, stop scrolling. I might have just reinvented computer use. Might. That word is doing a lot of work here and I'll explain why at the end, because the craziest part of this isn't even the claim. Ok so from the top. Every computer use agent today works the exact same way. Take a screenshot. Feed it to a huge vision model. Act once. Sleep for some guessed amount of time. Screenshot again. Over and over till the task is done or the agent falls apart. Think about what that loop actually costs you. A 20 step task pays for 20 full perceptions of a screen that barely changed. The page structure gets rebuilt from pixels every single step, and rebuilt slightly differently each time, so agents literally lose track of buttons they already clicked. And between screenshots the agent is fully blind. It clicks a button and finds out if the click worked one whole screenshot later. If it finds out at all. The entire field's answer to this? Bigger vision models. Denser screenshots. Capture more often. Bigger eyes basically. I'm going the opposite direction. Here's the thing everything rests on: a screen is not a photograph. It's the output of a layout engine. It's mostly empty rectangles. The structure is a handful of regions that matter. The meaning is carried by text. And change is sparse, one progress bar ticks while everything else sits still. Every one of those four properties has a classical, decades old, basically free sensor. Quadtrees. Hit testing and hover response. OCR inside known rectangles. And my personal favorite, dirty rectangle tracking, which VNC and RDP have been transmitting as their literal wire format since the 90s. The exact change stream agents are dying for has been getting computed for free on every remote desktop session on the planet for 30 years. Nobody ever handed it to a model as a sense organ. So I did. It's called ESP. Echo-Skeleton Perception. And yes the pun is deliberate. It gives a model with no eyes something like extrasensory perception. A bat flying through a cave in the dark. It perceives in three passes: Skeleton. One framebuffer grab gets carved into an adaptive region tree. Big lazy cells over empty space, tiny dense ones hugging content. Then it probes the screen like a bat pinging a cave. Hit tests, hover sweeps, cursor changes. When the cursor flips from arrow to hand, the OS just told you something is clickable. Free. No neural network anywhere. Populate. OCR runs only inside the cells the skeleton already found. Faster and more accurate than reading a whole page, and every word lands already attached to a node. Icons get hovered once, the tooltip names them, name gets cached forever. The skeleton learns to read. Echo. The screen gets diffed at millisecond scale and changes arrive as tiny events against a skeleton the model already holds: changed(node 51, text: "47%" -> "100%") appeared(node 60, banner "Upload complete") no_change(after=click(node 14), window=600ms) Look at that last one again. no_change is a first class percept. The silently failed click, the number one killer of agent runs, currently discovered a full screenshot later or never, becomes something the agent feels instantly and just retries. The failure is a sensation, not a deduction from a later photo. And waiting stops being sleep(3000) and a prayer. The agent subscribes to quiet(node) and the world wakes it up. End result: a small local text only model, 4B to 14B, the kind that runs on your laptop, holds a labeled skeleton as world state, receives tiny deltas, and acts through strict tool calls. It never sees an image. Not once. The whole perception layer is a few thousand lines of ordinary code. No GPU anywhere in the loop. Every token the model spends goes to actual thinking instead of re-photographing the same static form for the 8th time. The whole field is racing toward bigger eyes. ESP is my bet that for most of the work people actually automate, you don't need eyes at all. You need a skeleton and echoes. Now the part I said I'd explain. I published this paper before writing the prototype. On purpose. The paper has four falsifiable hypotheses, each with a hard number attached. Within 10 points of an accessibility tree agent. At least 5x cheaper in perception tokens. At least 15 points of recovery on silently failing controls. At least 70% of success retained on canvas pages where DOM agents drop to zero. Those numbers are locked in print right now, before a single line of code exists. Same with the decision rule for every possible outcome, including the branch where everything fails and I publish the negative result with the full trace corpus. There is no version of this where I quietly move the goalposts to fit whatever the experiments say. A bat doesn't decide where the cave wall is after it hears the echo. Neither will I. If you've been following Bad Theory Labs you already know the shape of this bet. Prism externalizes the reasoning scaffolding of frontier models into inspectable modules. ESP does the same thing to perception. Two halves of one thesis: mechanical scaffolding around small local models, everywhere the field assumes you need a giant one. The paper is live. Architecture, an honest accounting of which pieces are old (most of them) and what's actually new, a worked session, the pre-registered experimental plan, and six named failure modes where I expect this to get attacked. Prototyping started. The experiments cost a weekend and zero GPU hours, which is exactly why the claims are locked before they run. Read it. and Scrutinize it Paper in comments