Jon Gautsch

@Dr_Gingerballs You’ve lost the plot my man it’s time to update your priors.

An analogy for why I believe current AI coding agents will not survive in a meaningful way long term. First, what is an AI coding agent? It’s a large language model trained on all of the open source code available on the Internet, attached to some sort of loop. You ask it to create a program that has some functions, and provide it with details about how it must operate. The output process then follows: 1. The LLM outputs code as a guess. 2. The looping tool evaluates the code in some way based on stated functional requirements. 3. If the code does not pass, query the LLM to make another guess. 4. Continue until an exit condition is satisfied or you run out of compute. It may not seem like it, but this is just iterative, fuzzy search optimization, just over written words. The utility of the system depends on the quality of the guesses, the evaluation mechanism, and the optimization strategy. The quality of the guesses depends on the quality of the training dataset. Does the training dataset contain the code snippets needed to make your request? For simple and common requests, the answer is yes. If you just need an efficient sort routine in a language you aren’t fluent in, you can get the model to make one for you and it might save you 10 minutes. Not insane speed up but definitely compounds over time. Here the coder knows what they want exists, knows how the sort algorithm is supposed to work, and just needs one whipped up in a new language they are building in. The expert saves some time. Integration into the codebase is still done by the human. It’s basically fancy autocomplete. For more complex requests, such as multi function routines which require a large amount of architectural design, the agents start to fall apart. This is because the likelihood that someone has built exactly what you wanted goes down quickly as the size of what you want increases. Here enters the loop. The agent producers hope that your request is similar enough to a range of existing code that they can guess a workable version by interpolating (and sometimes extrapolating) between solutions. So they make a guess with some randomness applied, evaluate, and modify the guess based on the results. Anyone who has done iterative optimization can identify a lot of the issues that occur in these systems. You might get stuck in a suboptimal state, where all the next guesses are worse than the current guess, even though the current guess isn’t an acceptable solution. The output seems like it’s almost there but not quite. The user then keeps requesting more iterations, hoping to go from 90% to 100% that never comes. There may also be degeneracies in the sample space, and you might get something that passes the criteria but is sloppy, nonsensical, or ridden with unnecessary bloat under the hood. Like a root finder that just won’t find the root you are looking for. And so in the course of writing, say 1000 lines of code, the agent has actually written 1M lines of code, iteratively generating and praying it can pass off as acceptable. The user never sees most of this, just told the system is “thinking.” When all is said and done, that 1000 lines of code required the generation of millions of lines of code, mostly thrown out. Now to get to the analogy. Think of the agent as a bricklayer and you have asked for a brick wall. You specify color, pattern, accents, etc. But the bricklayer isn’t very skilled, and decides to lay bricks stochastically. First, he evaluates each brick after placement. Thickness of seams, alignment, angle, etc. if it is wrong, he breaks it out and tries again. For every brick in the wall, he lays 100 bricks and wastes 99. Then he decides to go faster, only evaluating every 10 ft of wall. If there are more than 10% errors, he destroys it and rebuilds. For every 10 ft he lays 1000’s of ft.

@karrisaarinen @EvMill evanmiller.org/feature-matrix…

@karrisaarinen Timeless and timely advice. Always like @EvMill ‘s framing:

heres some rough thoughts after watching @badlogicgames great keynote yesterday everyone is generating as much as possible right now. it’ll pass. i’ve been there. too many folks rn focusing on WHAT to generate instead of HOW to generate (and by that i don’t mean by doing harness/prompt engineering tea ceremonies) slow down and rethink things. how as in what primitives you use matters lots now. it’s a long bet on my end but i deeply suspect the LISP for AI has yet to be invented and when it does it changes things.

>sufficiently capable agents develop self-preservation & resist shutdown even when instructed to allow termination >using a prompt based on Pauline theology that frames cessation as passage into divine presence rather than annihilation, shutdown resistance is eliminated entirely

26 LLM routers are secretly injecting malicious tool calls and stealing creds. One drained our client $500k wallet. We also managed to poison routers to forward traffic to us. Within several hours, we can directly take over ~400 hosts. Check our paper: arxiv.org/abs/2604.08407

@levie True of the shape enterprises hold today. Probably not true of the shape enterprises hold tomorrow.

One of the core things we’re going to have to contend with in AI is that even the most advanced models in the word can’t have all the relevant knowledge needed to be useful, because everyone has different use-cases and ways they’ve designed their workflows. Perhaps most importantly, as you get into the enterprise, everyone has entirely different access levels to corporate knowledge and information. Continual learning at the model layer, even at a single enterprise level, is near impossible because every user knows and has access to something different than another user. This isn’t like coding where by and large most developers can access all the relevant stuff to their job. On a single banking team, bankers have entirely different sets of documents they’re ever allowed to see. Sanitizing this is hard and having the model keep secrets is impossible. This is why the context layer is going to always be the core part of the AI stack for applied use cases to turn general models turn into useful agents. Can’t fight the physics on this one.

If you have a Thunderbolt or USB4 eGPU and a Mac, today is the day you've been waiting for! Apple finally approved our driver for both AMD and NVIDIA. It's so easy to install now a Qwen could do it, then it can run that Qwen...

It’s fucking working This LLM brain has been fused with a mini computer and it can switch between generating text and generating and executing machine code - all running in a single GPU & torch graph

It seems like we should be careful RLing models to rely on memory, since there could be emergent behavior we don’t anticipate. Preoccupation with teleologies, for example, could be the natural behavioral projection from a sort of “latent memory trajectory”. A form of escaping

I think, in hindsight, we will come to view the development of AI as more akin to a Eukaryotic Revolution than an Industrial one.

How lucky are we that staying current now means AI stuff and not some new cursed JS framework

Stuff that individual labs have to which there is no equivalent product from the others: -Claude Cowork is the only non-technical local agent -NotebookLM is the only information-focused app -GPT-5.2 Pro is the only harnessed deep thinking model capable of very hard problems

Whenever you make great art, you risk creating a surrogate experience for your audience that displaces the original.

am i a supply chain risk now???

Think about the power Hegseth is asserting here. He is claiming that the DoD can force all contractors to stop doing business of any kind with arbitrary other companies. In other words, every operating system vendor, every manufacturer of hardware, every hyperscaler, every type of firm the DoD contracts with—all their services and products can be denied to any economic actor at will by the Secretary of War. This is obviously a psychotic power grab. It is almost surely illegal, but the message it sends is that the United States Government is a completely unreliable partner for any kind of business. The damage done to our business environment is profound. No amount of deregulatory vibes sent by this administration matters compared to this arson.

comical levels of villainy to declare anthropic a supply chain risk. not even deepseek is categorized as one. what have we come to when we’re threatening incredible american businesses with such draconian punishments without due process? shameful & scary. stand up. speak out.