leanxbt

This paper completely changed how I think about what a reasoning model should be rewarded for: Sample solutions -> Label every step of the chain -> Train a PRM on the steps -> Score candidates step by step -> Select the best solution Here is the 5-step blueprint: Two schools of reward: an ORM looks only at the final result of the chain, while a PRM receives feedback on every intermediate reasoning step. Step-level labeling: humans annotate individual solution steps, which pinpoints the exact location of an error instead of one correct-or-not signal over the whole output. PRM as verifier: the trained model scores chains step by step and picks the best solution out of many generator samples. Active learning: labeling targets the most informative chains rather than everything in order, which sharply raises the efficiency of expensive human feedback. Alignment bonus: step-level reward directly encourages following a human-endorsed chain of thought, not just landing on a lucky result. Key insight: rewarding the right answer teaches the model to reach it through wrong reasoning, so the process is what must be rewarded, not the result. The process-supervised model solves 78% of problems on a representative subset of MATH. Read this, then check the article below.

Five years before he joined Anthropic, Andrej Karpathy stood on a stage next to Elon Musk and explained how he built the neural network that drives every Tesla on earth. Tesla AI Day 2021. One hour. Live from Palo Alto. He walked the room through the exact architecture, the exact data pipeline, the exact training loop. One year later he quit Tesla. Two years after that he founded an AI education company that never launched a product. Two months ago Anthropic hired him to build the system that lets Claude train Claude. Most engineers describe the future in slides. This one already showed the roadmap in 2021 and just walked into the lab that will actually execute it.

This paper completely changed how I think about where reasoning training data comes from: Generate rationales -> Filter by correct answer -> Rationalize failures backward from the answer -> Fine-tune on the survivors -> Repeat with the new model Here is the 5-step blueprint: Few-shot seed: it all starts from a handful of rationale examples and a large question-answer dataset, with no human-written reasoning chain for the corpus. Generation: the model writes rationales and answers for every question in the dataset itself. Correctness filter: only rationales that led to the right final answer survive, answer correctness acting as an automatic verifier. Rationalization: on failed problems the model is handed the correct answer as a hint and writes the rationale backward, which pulls exactly the hard examples into training. Fine-tune and repeat: the model is fine-tuned on all surviving rationales, then the loop restarts with the improved model. Key insight: no reasoning annotator is needed, the correctness of the final answer plays verifier while the model trains on its own successful chains. STaR on a far smaller model matches the performance of a much larger model fine-tuned to predict the answer directly. Read this, then check the article below.

This paper completely changed how I think about a swarm of agents: Describe an agent as a graph -> Nodes are operations -> Edges are information flow -> Wire agents into one graph -> Optimize the graph itself automatically Here is the 5-step blueprint: Agent as a graph: any LLM agent is represented as a computational graph, where nodes are functions that process data or query the model. Edges as links: edges define the information flow between operations, that is who passes a result to whom inside an agent and between agents. Composition: graphs are recursively assembled into large composite graphs, so a swarm of agents becomes one object instead of a pile of hand-written scripts. Node optimization: the first optimizer automatically tunes prompts at the level of individual nodes. Edge optimization: the second optimizer uses RL to change graph connectivity, that is the orchestration of the swarm itself, cutting useless links and keeping the working ones. Key insight: you do not hand-design a swarm's topology; graph connectivity is a learnable parameter optimized like weights. Two levels of auto-optimization - node prompts and edge connectivity - turn scattered agent frameworks into one optimizable graph. Read this, then check the article below.

Two months before he joined Anthropic, the co-founder of Monzo stood in front of a YC batch and told them to burn the org chart. Tom Blomfield. Built two British billion-dollar companies before turning 35. Group Partner at Y Combinator. Portfolio worth five billion dollars. In 13 minutes he explained that modern companies are Roman legions and that the winners of the next decade will run their businesses as recursive self-improving AI loops. He told the room to stop counting headcount and start counting tokens. Last week he took a leave of absence from YC to work as an individual contributor on Anthropic's compute team. Most partners stay in the top box of the org chart. This one walked out to go help build the loop he was describing.

This paper completely changed how I think about how an LLM solves a task it was never built for: Parse the request -> Plan a queue of subtasks -> Pick a model for each -> Execute and gather outputs -> Generate the final answer Here is the 4-step blueprint: Task Planning: the LLM controller parses the user request, extracts the intent and breaks it into a queue of linked subtasks in a single pass. Model Selection: for each subtask the controller picks a concrete expert model from HuggingFace by reading its textual function description. Task Execution: the selected models are actually run, each on its own piece, and the results are returned to the controller. Response Generation: the LLM aggregates all model outputs into one coherent answer and explains which model it used for what and why. Key insight: the LLM itself does not need to be able to do everything, it only needs to read the descriptions of other models and conduct them through language as a common interface. A single LLM controller conducts hundreds of HuggingFace expert models and covers language, vision and speech within one task. Read this, then check the article below.

The CEO of Anthropic sat down with Bloomberg and admitted he does not know what role his own product played in a missile strike that killed 120 children at an Iranian elementary school. Dario Amodei. His company is worth $965 billion. Anthropic just filed for IPO. The Pentagon runs Claude across US Central Command targeting systems under a $1.3 billion contract. In the same interview he said the principle of human decision-making was obeyed. Then he said he cannot verify how the model was used. Amnesty International documented the strike as the first US attack on the day the war began. February 28. Most CEOs defend their product with certainty. This one defended it with the words we don't know.

This paper completely changed how I think about where a model's tool use comes from: Show APIs in special tokens -> Let the model place calls in the text itself -> Actually hit the API -> Keep only calls that lowered loss -> Fine-tune on them Here is the 5-step blueprint: Self-supervised annotation: instead of hand-wiring tool integrations, the model generates candidate calls right in the text, given only a handful of demonstrations per API. Special tokens: a call is wrapped in tokens like query -> result, so the call becomes part of the ordinary language stream. Real execution: every generated call is actually run against the tool - a calculator, a QA system, two search engines, a translator, a calendar. Filter by loss: a call is kept in the dataset only if inserting its result actually lowered the perplexity of future tokens, everything useless is thrown out. Fine-tune on itself: the model is fine-tuned on that filtered dataset, after which it decides on its own which API to call, when, with what arguments and how to weave in the result. Key insight: to teach a model tools you need no manual integration or labeling, you just keep the calls that lowered its own loss. At 6.7B, Toolformer matches or beats zero-shot GPT-3 at 175B where tools actually help, without losing its core abilities. Read this, then check the article below.

Investors gave a man six billion dollars to not release a product. Ilya Sutskever. Former chief scientist of OpenAI. Left in 2024 after voting to fire Altman. Founded Safe Superintelligence Inc. one month later. Twenty employees. Zero customers. No papers published. No demos. No timeline. Alphabet and Nvidia are both investors. Meta tried to buy the company. He said no. In June he returned to the University of Toronto to accept an honorary doctorate. He told the graduates AI will one day do everything humans can do. Then he walked off the stage and went back to building the thing he refuses to ship.

This paper completely changed how I think about why a team of agents beats a single one: Encode human SOPs -> Assign each role a task -> The role emits a structured document -> Hand it down the line -> Engineer runs the code with tests -> Finished software Here is the 5-step blueprint: SOPs into prompts: the standardized operating procedures of a software company are baked into prompt sequences, imposing a strict workflow instead of free chat. Role specialization: five roles - Product Manager, Architect, Project Manager, Engineer, QA - each with its own profile, goals and its own tools. Structured handover: agents communicate through documents and diagrams, not dialogue, which cuts the cascading hallucination of a game of telephone. Shared message pool: roles publish structured messages to a shared pool and subscribe only to what their profile needs, with no one-to-one polling. Executable feedback: the engineer writes unit tests, runs the code, catches execution errors and fixes them from memory until they pass, up to 3 retries. Key insight: the cascading hallucination of chained agents is choked not by a smarter model, but by a human SOP that forces roles to exchange hard documents instead of chatter. MetaGPT hits 85.9% and 87.7% Pass@1 on HumanEval and MBPP, and on its own SoftwareDev drops human revision cost from 2.25 to 0.83. Read this, then check the article below.

The CEO of OpenAI does not let his own son use his own product. Sam Altman. Home interview. He told Laurie Segall he wants his baby to play in the dirt, not talk to ChatGPT. He said he watches kids just a little older than his son who cannot put down the iPad and it makes him feel strongly. He said he wants to be on the late end of what is reasonable, not the early end. His son is 15 months old. The CEO of the most-used AI on the planet is delaying it for the one child in the world he loves most.

This paper completely changed how I think about how an agent learns for a lifetime without forgetting what it learned: Curriculum sets a goal -> Write a code skill for it -> Execute and catch errors -> Self-verify the result -> Store the working skill -> Next goal Here is the 5-step blueprint: Automatic curriculum: the agent sets its own next task, aiming at maximum exploration for its current level, with no human and no fixed goal list. Skill as code: each behavior is an executable program, not a weight, so a skill is interpretable, transferable and composable from simpler ones. Iterative prompting: the code runs in the environment, and environment feedback, execution errors and self-verification loop back into the prompt until the skill works. Growing library: a working skill is stored in memory and later retrieved as a building block, which compounds abilities and chokes catastrophic forgetting. Black box: it all runs on top of GPT-4 via blackbox queries, without a single step of parameter fine-tuning. Key insight: if an agent's skill is stored as reusable code in a growing library, abilities compound instead of being erased when it learns something new. Voyager obtains 3.3x more unique items and clears the tech tree up to 15.3x faster than the prior SOTA. Read this, then check the article below.

Six months ago the creator of Claude Code uninstalled his IDE. Then he realized he hadn't opened it in a month. Boris Cherny. Ex-principal engineer at Meta. Ten years of muscle memory built into that IDE. Now he ships 27 pull requests in a day from his phone and hasn't written a line of code in 2026. He sat down with Acquired last month and said the quiet part out loud. He is starting to lose the ability to code manually. Karpathy called it brain atrophy. Cherny said he feels it too. Most engineers worry about their job being replaced. This one is watching his craft disappear from his own hands.

This paper completely changed how I think about where reasoning accuracy comes from with no reflection at all: Sample many different chains -> Reach an answer in each -> Collect the final answers -> Marginalize out the paths -> Return the most frequent one Here is the 5-step blueprint: Drop greedy: instead of one greedy chain over chain-of-thought, the model samples a whole set of different reasoning paths. Many paths to one answer: it leans on the fact that a hard problem admits many ways of thinking that all reach the same correct answer. Final answers: only the final answer is pulled from each chain, the reasoning itself is not needed afterward. Marginalization: the paths are dropped, the answers are aggregated, and the one reached by the most independent chains wins. Fully unsupervised: no training, no fine-tuning, no separate verifier and no human labels, just a change of decoding strategy. Key insight: if the model reaches one answer by many different paths, that answer is far more likely correct than the output of a single greedy chain. A single decoding change gives +17.9% on GSM8K, +11.0% on SVAMP and +12.2% on AQuA. Read this, then check the article below.

The CEO of OpenAI once ran to a bathroom mid-party to ask ChatGPT if his baby was developing normally. Sam Altman told the story on national TV in December. Five weeks later he sat down with builders for a one-hour town hall and answered the questions no journalist ever gets to ask. When will agents run long workflows without a human. Whether software engineers become obsolete or in higher demand. What happens when intelligence gets too cheap to measure. The interview came out the same week he circulated an internal memo declaring OpenAI code red. Most CEOs go on late night to sell you their product. This one showed you what it looks like when even he can't put it down.

This paper completely changed how I think about how one model improves itself with no training: Draft an output -> Give yourself multi-aspect feedback -> Rewrite from that feedback -> Check the stopping criterion -> Repeat Here is the 5-step blueprint: One model, three roles: the same LLM acts as generator, critic and refiner, with no separate models and no fine-tuning. Draft: the model first produces an ordinary initial output for the task, as is. Self-feedback: the same model inspects its output and writes concrete actionable feedback across several aspects, not a single vague score. Refine: the output is rewritten conditioned on the input, the previous version and the feedback, aiming exactly at the named flaws. Stopping criterion: the generate-feedback-refine loop runs until the model decides there is nothing left to improve or hits a limit. Key insight: even a top model rarely gives its best answer on the first try, and the same weights, asked to critique themselves, pull the result up with no RL at all. Across 7 diverse tasks, humans and metrics prefer Self-Refine output over direct GPT-4 generation, with an average gain of about 20% absolute. Read this, then check the article below.

The co-founder of Anthropic asked his 4-year-old daughter who is right when Daddy disagrees with an AI. She immediately answered AI. Jack Clark. Started as the only neural network reporter at Bloomberg. Left journalism to run policy at OpenAI. Co-founded Anthropic. Six weeks ago at Oxford he gave his current probability. 60 percent chance AI trains its own successor by 2028. He said the risk of the technology killing everyone on the planet has not gone away and it was important to state that clearly. Days before he took the podium his own company hired Karpathy to build exactly the loop he was describing. Most co-founders sell the upside. This one gave Oxford a deadline.

This paper completely changed how I think about how an agent fixes its own code: Generate code -> Execute it -> Explain the code line by line -> Spot the bug from the explanation and output -> Rewrite -> Repeat Here is the 5-step blueprint: Iterative debugging: the model does not write correct code from scratch, it enters a loop of edits to its own prediction. Rubber-duck: the model explains its generated code line by line in natural language and finds the bug from that explanation alone, with no error messages. Execution feedback: where unit tests or an interpreter exist, the real execution output is fed into the loop, not just self-explanation. Trace variant: the model dry-runs the code step by step in its head, which strengthens debugging where no tests are available. Reusing failures: failed predictions and their feedback are not thrown away but carried into the next round, sharply raising sample efficiency. Key insight: to find a bug the model needs no error message, it only needs to force itself to explain its own code in words. Self-Debugging matches or beats baselines that generate 10x more candidates, and lifts accuracy by up to 12%. Read this, then check the article below.

Eight years ago Ilya Sutskever stood at a San Jose conference and warned the industry that near-term AGI should be treated as a serious possibility. He was 32. Chief scientist at a billion-dollar nonprofit. Nobody outside the room took the framing seriously. In the same 45-minute keynote he described how pre-training and fine-tuning on unsupervised language data would significantly improve state of the art. Four years later that architecture became ChatGPT. Last month the National Academy of Sciences gave him the first award ever presented in AI. The 2018 keynote is still on YouTube, on a channel almost no one watches. Most people call AGI when it arrives. This one called it eight years early.