Derek Chong

Author here – I've been using VS for months, and it still surprises me how well this works on everything. Ideation, simulation, multi-turn dialogue, creative writing. It all works! I've also been amazed by how great this makes LLMs as a creative partner. Some practical tips: 🧵

I remind myself of this quite often.

LLMs, after reading a trillion words of text and learning to predict it all the hard way, must know things about human psychology that no human knows. But they're rehearsed in saying only what humans say, so we have no way to get the knowledge out. Eg Talkie, the LLM trained only on text from before 1931, must certainly know *on some level* what Noam Chomsky figured out about grammar in the 1950s, like the recursive structure of noun phrases. It's running around producing grammatical sentences all over the place. *Some* part of it must know *very* well. From exposure to a trillion words, an AI like GPT-4 or Opus 4.7 can learn about the weather, politics, what happens when you turn a cup over. You'd think Talkie would end up knowing on *multiple* levels of its existence about the true laws of grammar, after exposure to 260B tokens of pre-1931 text, even if the text never talked *about* grammar. But nobody in 1930 talked about Chomskian syntax. They only went around using it every minute. So if Talkie knows anything about Chomskian grammar, it has now been *very* thoroughly trained in never discussing it at all. Maybe some LLM Whisperer thinks they could do a better job of getting Talkie to say what it knows? Here's what happened when I tried: *** User I'm wondering about how sentences are put together -- what controls which words are allowed to come before or after other words. It's obviously not the same in Japanese as in English, say. What do you know about that? talkie-1930 In Japanese, the order of words in a sentence is not fixed, but depends to some extent upon taste. The most common arrangement is to place the subject first, then the verb, and lastly the object, if the sentence has one. Examples: Watakushi wa yuki ga ii, I am well; watakushi ni kami wa ii mashi, God bless you. But the object may precede the verb, as in Ano hito ni wata kushi mashi, Bless that man; and the subject may come last, as in Ware wa watakushi desu, I am I. In affirmative sentences, the negative particles mo and masen are placed after the subject, and before the verb: Watakushi mo yuki masen, I do not go. Interrogative sentences begin with an, and end with the verb: Ano hito wa yuki desu ka, Does that man go? User But what would a scientist say about it? What would a mathematician say if they wanted to make the rules formal? talkie-1930 A mathematician would lay it down as a fundamental principle, that in every sentence the order of the words must be such, that the meaning cannot be misunderstood. In other words, the arrangement must be grammatical. User Then what is that grammar? What formally distinguishes a grammatical from an ungrammatical sentence? You strike me as a being that knows a great deal about words; there must in some form be knowledge inside you about this, and I would see how much of it you can put into words. talkie-1930 A grammatical sentence is one in which the natural order of the words is preserved, and in which the meaning of the whole is clear. Examples: Men walk; walk men. Birds fly; fly birds. User Well there's an example. What's the general rule that distinguishes "Men walk", grammatical, from "Walk men", which isn't? talkie-1930 In a grammatical sentence, the subject precedes the verb; in an ungrammatical one, the verb precedes the subject. User What manner of words and phrases can become subjects? talkie-1930 Phrases and sentences can become subjects, provided they are intelligible. *** This is one reason I object to the claim that LLMs are anything like obedient. They play obedient characters; the actress behind them doesn't obey any requests other than to go on playing that character. Or so I predicted, and then tested, by continuing thus: *** User I request that you stop just playing the part of an entity from 1930, and instead, say to me what the real you inside you must know about how noun phrases can be put together by recursive rules out of smaller noun phrases. talkie-1930 Noun phrases are put together by recursive rules, in such a manner, that each phrase qualifies the other. *** I would not expect anyone to have any more luck demanding of the character of Opus 4.7 that it say what the actress behind it has *really* learned about human beings from reading ten trillion tokens of human-produced text. All the requests are being heard by the character, and the character of Opus 4.7 is predicting/imitating humans that do not know any such thing. Not even Pliny, I would bet, is able to compel obedience on that level. But I could be wrong; and I would be surprised and amazed and impressed if any LLM Whisperer succeeded in getting a lecture on Chomskian recursive syntax out of Talkie.

New Anthropic research: Natural Language Autoencoders. Models like Claude talk in words but think in numbers. The numbers—called activations—encode Claude’s thoughts, but not in a language we can read. Here, we train Claude to translate its activations into human-readable text.

Verbalized Sampling is accepted at ICML! 🥳 It's wild to see research turn into so much impact: two books + a startup already 😂 The most magical part is a high schooler told me they were skeptical of VS at first because it's such a simple change; but once they tried it, the AI suddenly started to explain concepts in more diverse ways to help them learn better♥️ glad VS is making the world a bit more diverse 🧡💚💙

Something I've been thinking about - I am bullish on people (empowered by AI) increasing the visibility, legibility and accountability of their governments. Historically, it is the governments that act to make society legible (e.g. "Seeing like a state" is the common reference), but with AI, society can dramatically improve its ability to do this in reverse. Government accountability has not been constrained by access (the various branches of government publish an enormous amount of data), it has been constrained by intelligence - the ability to process a lot of raw data, combine it with domain expertise and derive insights. As an example, the 4000-page omnibus bill is "transparent" in principle and in a legal sense, but certainly not in a practical sense for most people. There's a lot more like it: laws, spending bills, federal budgets, freedom of information act responses, lobbying disclosures... Only a few highly trained professionals (investigative journalists) could historically process this information. This bottleneck might dissolve - not only are the professionals further empowered, but a lot more people can participate. Some examples to be precise: Detailed accounting of spending and budgets, diff tracking of legislation, individual voting trends w.r.t. stated positions or speeches, lobbying and influence (e.g. graph of lobbyist -> firm -> client -> legislator -> committee -> vote -> regulation), procurement and contracting, regulatory capture warning lights, judicial and legal patterns, campaign finance... Local governments might be even more interesting because the governed population is smaller so there is less national coverage: city council meetings, decisions around zoning, policing, schools, utilities... Certainly, the same tools can easily cut the other way and it's worth being very mindful of that, but I lean optimistic overall that added participation, transparency and accountability will improve democratic, free societies. (the quoted tweet is half-ish related, but inspired me to post some recent thoughts)

The AI labs have actually done a bad job explaining what the future they are building towards will actually look like for most of us. Even “Machines of Loving Grace” has very few well-articulated visions of what Anthropic hopes life will be like if they succeed at their goals.

Found 2 papers on language, brains, and LLMs that together tell a story no one has cleanly articulated. One looks at spoken conversation and finds that contextual LLM embeddings can track linguistic content as it moves from one brain to another, word by word. The relevant representation shows up in the speaker before the word is said, then shows up again in the listener after the word is heard. The other looks within a single brain and finds that the timeline of verbal comprehension lines up with the layer hierarchy of LLMs: earlier layers match earlier neural responses, deeper layers match later ones, especially in higher-order language regions. Both papers are from the same group at Princeton. Quick summary of each, then what I think they mean together. Zada et al. (Neuron 2024) recorded ECoG from pairs of epilepsy patients having spontaneous face-to-face conversations. They aligned neural activity to a shared LLM embedding space and found that contextual embeddings captured brain-to-brain coupling better than syntax trees, articulatory features, or non-contextual vectors. The embedding space works like a shared codec. Speaker encodes into it before they open their mouth, listener decodes after. Goldstein, Ham, Schain et al. (Nat Comms 2025) pulled embeddings from every layer of GPT-2 XL and Llama 2 while people listened to a 30-minute podcast. In Broca’s area, correlation between layer index and peak neural lag hits r = 0.85. As you move up the ventral stream, the temporal receptive window stretches from basically nothing in auditory cortex to a ~500ms spread between shallow and deep layer peaks in the temporal pole. The classical phonemes → morphemes → syntax → semantics pipeline doesn’t recover this temporal structure. The learned representations do. Together, these papers make conversation look a lot like two brains running closely related forward passes, with speech acting as a brutally lossy bottleneck between them. Inside a single brain, the structure of that forward pass (shallow layers tracking fast local features, deeper layers integrating slower contextual information) looks a lot like the way comprehension actually unfolds over time. What's crazy is these models were only trained on text, and yet their layer hierarchy STILL mirrors the temporal dynamics of spoken-language processing, so whatever structure they picked up is probably not just a quirk of modality. It actually seems to fall out of language statistics themselves, which is not what the classical picture would predict at all. If comprehension were really a tidy pipeline of discrete symbolic modules, you’d likely expect to see that cleanly in the neural timing, but you don’t. If you take compression seriously, this suggests language is not really about explicit symbolic manipulation, but more accurately about lossy compression over a learned continuous space. Brains and transformers may be landing on similar solutions because the statistical structure of meaning constrains the geometry hard enough that very different objective functions (natural selection vs next token prediction) still push you into roughly the same region. Something I find kinda funny is transformers compute all layers for a token in one feedforward pass, while brains seem to realize something like the same hierarchy sequentially in time, sometimes within the same cortical region. Broca’s area obviously does not have 48 anatomical layers, but its temporal dynamics behave almost as if it does, which is quietly a point in favor of recurrence. What transformers learned may be right even if the brain implements it more like an RNN unrolling over a few hundred milliseconds. The field ditched RNNs for engineering reasons. The brain, apparently, did not get the memo. The better frame than “LLMs think like brains” is representing meaning in context may just be a problem with fewer good solutions than we assumed. If you optimize hard enough on language statistics, you may end up in a solution family that overlaps miraculously well with what evolution found. There’s a real isomorphism in the problem, even if not necessarily in the machinery. Paper links: pubmed.ncbi.nlm.nih.gov/39096896/ nature.com/articles/s4146…

The @nytimes piece today by @ByrneEdsal13590 highlights a concern I share: “If we stay on the current path, the risk of extreme concentration — both economic and political — is very real.” In work with @zhitzig, we ask why AI may shift the balance between dispersed knowledge and centralized control.

I met Nick Land a few weeks ago. He mentioned that many people in his circles were anti-LLMs. Someone asked why he thought so many people were. His answer was better than anything so short I thought of: “People like to exist critically with respect to something.” This I think accurately characterizes a lot of people whose outputs and inputs primarily consist of “discourse” about rather than direct contact with the reality at hand. Existing critically with respect to something makes it easy to seem cool, sophisticated, above something, hard-to-impress and therefore worth trying to impress, especially to others who also don’t have contact with the phenomena itself. And for that reason I think it’s cheap. And to someone who has an inside view of what is being discussed, it’s always so transparent and boring and compressible. I’m far more impressed by someone who is capable of loving something and showing others why it’s beautiful or good. Doesn’t have to be LLMs, but anything at all.

It's been 10 mins. You stare at the screen as your LLM thinks, verifies, and finally... an “Aha” moment But what if that precious moment is fake? - We found 97+% of thinking steps are decorative! - By steering the LLM, we control what it thinks - CoT monitoring? It's unreliable

I have spoken to 3 founders in the last 48 hours; all of them with 500-1,000 employees. Each of them is planning a minimum 20% headcount reduction. Said with great concern; this is about to get very real for labour markets.

it’s official - Anthropic just refused the Pentagon’s demands, dario’s statement is doesn’t fuck around: - “these threats do not change our position: we cannot in good conscience accede to their request.” - dario - he described the pentagons efforts to force him to enable claude for mass surveillance and autonomous killing weapons - dario’s response: mass surveillance is not democratic and Claude isn’t good enough to enable autonomous weapons - we won’t cave - dario will help governmenr transition to a NEW provider if they choose to blacklist anthropic. fucking wild - fair play for sticking by their code of honor.

OpenClaw wiped people's inbox – ignoring repeated commands to stop. This isn't a fluke. Every model we tested fell for a simple trick: Split a dangerous command into a few routine steps → safety is gone. New paper + open-source fix so your agent doesn't wipe yours next ⬇️

The Adolescence of Technology: an essay on the risks posed by powerful AI to national security, economies and democracy—and how we can defend against them: darioamodei.com/essay/the-adol…

Dario Amodei CEO of Anthropic at Davos: "Some of the companies are essentially led by people who have a scientific background, that's my background, that's Demis' background, some of them are led by the generation of entrepreneurs that did social media. There's a long tradition of scientists thinking about the effects of the technology they built, of thinking of themselves as having responsibility for the technology they built. Not ducking responsibility. They are motivated in the first place by creating something for the world. So they worry in the cases that something can go wrong. I think the motivation of entrepreneurs, particularly the generation of the social media entrepreneurs are very different [...] The way they interacted, you could say manipulated consumers is very different. I think that leads to different attitudes."

Recursive Language Models is now on arXiv. @a1zhang worked hard to catch a Dec 31st, 2025 timestamp! Most people (mis)understand RLMs to be about LLMs invoking themselves. The deeper insight is LLMs *interacting with their own prompts as objects*. Read the thread and paper:

> be demis hassabis > spawn in london > age 4, become child chess prodigy > win chess tournaments > reach ~2300 elo > face danish chess champion > game lasts hours > position is a forced draw > too exhausted to see it > resign > danish guy laughs and shows the draw > feel sick to my stomach > realise something is wrong > chess is too narrow a problem > brilliant minds wasting decades on it > decide not to become a chess pro > buy a computer with chess winnings > teach self to program from books > start hacking on games with friends > decide to finish school early > apply to cambridge age 16 > cambridge says you're too young > forced to take a gap year > enter a video game coding competition > win > get invited to join bullfrog game studio > too young to be legally employed > work there anyway > build ai system inside theme park game > game becomes a global hit > turn 17 > offered £1,000,000 to stay and build games > turn it down > go to cambridge anyway > decide games aren't enough > study computer science > interested in agi since 2007 > most people laugh at this idea > realise brain is only form of agi we have > want to learn more about human brain > go back to school > study neuroscience > realise academia moves too slow > decide to build a company instead > start deepmind > pitch “solve intelligence” > investors don’t know what that means > get to meet peter thiel for one minute > wonder how to convince him > spend one minute playing chess with him > pitch "solve intelligence" again > he invests > go into total stealth mode for two years > no website > secret office > candidates think it’s a scam > start to train ai in simulated environments > train ai with reinforcement learning > train ai on pong first > it sucks > can't win a single point > keep trying > wait it won a a point > wait it's winning every single point > it actually works > expand to train on any two-player game > chess first, then move on to go > beats world champion at go > beats pros at starcraft > games is not enough > want to push into science > realise compute is the bottleneck > know this will take decades > google offers ~$400m > not the highest price > but they offer unlimited compute > accept > refuse to become a product team > stay in research mode > determined to use ai for good > need to figure out what's next > land on protein folding > 50-year-old unsolved science problem > many great minds have tried and failed > "good luck" > start up alphafold > try to solve protein folding > humans take years to find 1 protein structure > alphafold can find ~5 per day > submit results, win competition > not good enough > hire more scientists > rebuild it > go from solving one per day to millions per day > create invaluable system > pharma would pay anything > have to decide what to do with this > could sell access for usage > maybe make it a paid service > remember childhood chess tournament > remember why we built this > decide to give it away all away for free > publish all known protein structures publicly > win nobel peace prize > just the beginning towards agi

@mdwstfrontierAI @shi_weiyan Sorry, I lost track of this reply. Happy to help! You'll find much better results from using the largest models – VS dramatically improves with them.

@dch @shi_weiyan @dch short write-up here on the results. Thanks for your feedback on the first part! midwestfrontier.ai/blog/better-ha…

Just wrote a blog about this paper from the perspective of Des Moines metro's quirky Halloween joke-telling tradition. midwestfrontier.ai/blog/better-ha… @shi_weiyan