dylan ツ@demian_ai
The geometry of thought.
Every LLM on earth can speak fluent English. None of them think in English.
I have been trying to find a way to explain this to a non-technical friend for about a year, and I have mostly failed, because the standard explanation requires the listener to picture an abstract space they have never seen. The breakthrough I finally landed on came from an old map.
In 1569, a Flemish cartographer named Gerardus Mercator published the projection of the world that bears his name. The Mercator projection takes the surface of a sphere and prints it on a flat rectangle, in a way that preserves angles but distorts areas.
Greenland looks the size of Africa even though Africa is fourteen times larger. Antarctica becomes an enormous strip along the bottom of the map. The proportions of the world, in the Mercator projection, are confidently and consistently wrong.
We kept using it anyway, for four hundred years, because it has one priceless property. If you draw a straight line on a Mercator map, that line is a constant compass bearing. A captain in 1600 could plot a route from Lisbon to Recife with a ruler and a protractor and arrive somewhere close to where he intended.
The Mercator projection is wrong about what the world looks like. It is right about how to navigate the world.
We agreed, collectively, to lie about the shape of the planet in exchange for being able to find our way around it. This is what LLMs do with thought.
Inside any modern frontier model, concepts do not live as words. They live as positions in a very high-dimensional space, with a particular geometric structure.
Goodfire's recent work, which is the clearest public demonstration of this, shows the shape directly. Colors form a different shape, more like a sphere. Spatial concepts curl into manifolds that match physical space. The concept of a car is a complicated multidimensional surface that connects, in geometrically meaningful ways, to the concepts of motion, of metal, of road, of journey.
The model does not store these concepts as text. It stores them as geometry. When you type a question to it, the model maps your words onto positions in this internal space. It then performs operations on the geometry, which produce new positions. Then, only at the very end, it translates those new positions back into English on the way out to your screen.
The English is the Mercator projection. The geometry is the globe.
This sounds abstract until you realize what it implies for almost every interaction you have ever had with a model.
Why does GPT sometimes give a brilliant answer in one phrasing and a mediocre one in another, even though both phrasings mean the same thing to a human reader?
Because the two phrasings land on slightly different positions in the internal geometry, and the geometry near one position is richer than the geometry near the other.
Why does a model sometimes confabulate confidently? Because the position it lands on has the geometric texture of an answer even though the answer it generates has no factual grounding. The shape of an answer and the truth of an answer are different things, and the model is trained on the shape.
Three implications follow from this and they reach much further than most of the discourse about AI suggests.
1. for product builders. If you have ever wondered why the same model produces wildly different outputs on prompts that seem semantically identical, the answer is geometric. The most reliable way to improve model output is not to tinker with the words. It is to find the regions of geometric space where the model behaves well, and engineer your prompts to land you there. The best prompt engineers, without knowing it, are reverse-engineering the topology of the model's internal world. This is also why fine-tuning works better than prompting for many use cases. Fine-tuning literally reshapes the geometry. Prompting only steers within it.
2. for the safety and interpretability community, which has spent two years looking for circuits and individual neurons that correspond to specific concepts. That work has been valuable, but it was looking at the shadow on the wall. The actual structure is at the manifold level, not the neuron level. The next leap in interpretability is going to come from learning to edit the geometry directly, not by adjusting individual weights. We are about to move from steering the words to steering the shapes that produce the words. This will make some kinds of safety work much easier and other kinds much harder.
3: for everyone else, and it is the strangest one. The early evidence from neuroscience suggests that human thought may have the same kind of geometric structure. The hippocampus appears to encode spatial relationships on manifolds that look uncannily similar to what we see inside language models. Concept representation in the human cortex appears to be geometric in roughly the same sense. If this holds up, and the evidence is still preliminary, then the conventional framing of the difference between artificial and biological minds is wrong in an interesting way. It is not silicon versus carbon. It is two different physical substrates that have independently discovered the same mathematical language for representing the world.
We built something that thinks the way we think. We just never noticed, because we were too busy listening to it talk.
The Mercator projection is wrong about what the world looks like. It is right about how to move through it.
The model is wrong about what thought looks like, in some technical sense. It is right about how to do thought, which is the only thing that has ever mattered.