Burny - Effective Curiosity

Hey! Follow me on the quest to understand the fundamental mathematics of intelligence and of the universe with curiosity! I explore how everything works, artificial intelligence, intelligence, mathematics, machine learning, physics, science, technology, engineering, understanding AI using mathematical theory from the perspective of physics or pure mathematics or other sciences and by reverse engineering using mechanistic interpretability from the perspective of neuroscience or other sciences, AGI, superintelligence, LLMs, reinforcement learning, AI beyond LLMs and autoregression and transformers and pure deep learning like neurosymbolic AI or physics inspired AI like diffusion models and liquid neural networks or biology inspired AI like evolutionary AI and selforganizing AI or open-ended novelty search, AI for science like physics and biology, comparing AI and biological intelligence, diversity of minds or intelligences or information processing systems, making AI do what we want, AI engineering, creativity, curiosity, neuroscience, fundamental physics, theoretical mechanics, statistical mechanics, quantum mechanics, biology, foundations of everything, philosophy, philosophy of science, bayesianism, ontology, cognitive science, consciousness, philosophy of mind, foundations of mathematics, theories of everything, interdisciplinarity, transdisciplinarity, multidisciplinary, omnidisciplinarity, grokking, AI for engineering like software engineering and programming, AI for good like healthcare, AI agents, future of humanity and AI, futurology, politics, geopolitics of AI, transhumanism, world views like Effective Altruism and Effective Accelerationism, potential benefits and risks of advanced technologies like AI and everything, forecasting, open science, open source, democratizing technology, concentration vs decentralization of power, applied mathematics, pure mathematics, metamathematics, category theory, probability theory, linear algebra, real analysis, optimization theory, statistics, information theory, systems theory, complexity science, emergence, selforganization, dynamical systems theory, open systems, art, logic, formal ethics, cybernetics, computational neuroscience, qualia, meditation, psychology, mathematical psychology, mathematical psychonautics, identity, wellbeing, memetics, epistemic humility, morphological freedom, cosmology, collective intelligence, and so on. Questions I explore the most - How does the world work? How does everything work? - What is the fundamental mathematics of intelligence? What are all the different types of all the possible current and future intelligent systems? - How does artificial intelligence work? What's the current state of empirical research and mathematical theory in artificial intelligence? What is the state of the art in artificial intelligence engineering practice? - What is the fundamental mathematics of the universe? What are all the equations, and mathematical structures more generally, governing reality across all scales in physics, and in all natural science more generally? - How to apply AI for reverse engineering mathematics behind everything? How to apply AI for good as ideally as possible as much as possible? - How to define and understand artificial general intelligence and superintelligence? How to make it do what we want? - What is the fundamental mathematics of the brain? How to upgrade human intelligence? - How does AI and biological intelligence compare? How can humans and AIs form even greater collective intelligence? - How to connect all sciences, formal and natural? What is the fundamental mathematics behind emergence and complexity? How does biology and other scientific fields emerge from physics and chemistry? - What is the fundamental mathematics of creativity in science and art? How to make machines creative beyond human limitations and comprehension for scientific discovery, physics, mathematics, art, philosophy? - What are all the concepts in mathematics? What are all the possible foundations and mathematics with all sorts of mathematical universes and which ones are the best in what contexts? - What is the fundamental mathematics of consciousness? - What is the fundamental mathematics of building a great future for all where everyone flourishes? How to make the world better for all? How to maximize the benefits, and minimize the disadvantages, of technologies and political systems? What is and what will be the geopolitics of AI? What are the probabilities of different future scenarios? - What are the answers to the problems in philosophy? Artificial intelligence research and engineering - How does artificial intelligence (and intelligence more generally) that might soon become very general and superintelligent, works from an interdisciplinary lens using all sorts of mathematics and methodologies from computer science, statistics, information theory, physics, mechanistic interpretability, cognitive science, geometry, pure mathematics and so on, what is the mathematical theory of artificial intelligence? How to mathematically and empirically understand current and future AI systems, why and how they work, and how to make them much more reliable, robust, steerable, creative, intelligent, safe etc. across all levels of their development, such as with formal verification? What are the current state of the art results and methods in AI engineering and research in academia and industry, such as mathematical results, empirical results, and tools used like PyTorch? What will be future developments in scaling, data, algorithms, architecture, hardware, wrappers, agents, multiagent systems, etc.? How good in what usecases are various AI paradigms such as statistical AI, connectionist AI with deep learning, symbolic AI, neurosymbolic AI, evolutionary AI, cognitive AI, bayesian AI, biologically-inspired AI such as neuromorphic AI, quantum AI, embodied AI, distributed AI, etc.? How can we merge different paradigms? What would be considered as Artificial General Intelligence and Superintelligence? How to define, measure and build intelligence? How much does AI safety play a role? What is the intersection of artificial intelligence x biological intelligence x collective intelligence? How to create theory of everything in intelligence? State of the art of AI - I am interested in the current state of the art top artificial Intelligence systems (machine learning, data science, statistics, deep learning, generative AI (large language models, image/sound/video models, multimodal models), reinforcement learning models, expert systems, neurosymbolic AI, etc. I want to use them in practice for the benefit of others, such as for automating mundane tasks (dishes, laundry), healthcare (AMIE, AlphaFold, SLIViT), programming (coding AI copilots such as Claude Code, Codex), science (autonomous science such as AI scientist), physics (FermiNet), mathematics (AlphaProof), technology development (AlphaChip, virtual reality), chatbot assistants grounded in reality, education, information searching, minimizing various risks and crises, transportation, manufacturing, security, cybersecurity, energy optimization, supply chain optimization, weather forecasting, agriculture, translation, recommendations, finance, call centers, entertainment, legal services, games, robotics for good, altruism, etc. by predicting, forecasting, generating, classification, analysis, clustering, segmentating etc., with AI engineering methods by building and training models, finetuning, prompt engineering, retrieval augmented generation, agent and multiagent frameworks, etc. using PyTorch, Keras, Scikit-learn, FastAI, OpenAI or Anthropic API, Llama locally or deployed, Llamaindex, Langchain, Autogen, LangGraph, vector databases, etc. Mathematical and other fundamentals, steerability of AI - I am interested in trying to mathematically and empirically understand current and future AI systems, why and how they work, and how to make them much more reliable, robust, steerable, creative, intelligent, safe etc. across all levels of their development! Better steering wheel for AI systems would be great! RLHF, prompt engineering, systems made of LLMs, and current reverse engineering methods don't seem to be enough! Mechanistic interpretability, neurosymbolic AI, weak to strong generalization paradigm, and formal verification sound promising! I'm curious about the mathematical theory of artificial intelligence! Big picture of artificial intelligence - I love AI for science like biology and physics, mathematics, healthcare, education, technology development for good, understanding the nature of intelligence, increasing the standards of living for all, progress of civilization and so on. I want to see more of that please! I want to see AI applied much more in science, technology, engineering, mathematics, healthcare, altruistic usecases, etc. I want to see it as a tool that generates abundance for everyone. I want the technology to build better future for all. I want the technology to fight poverty and other world problems and risks. I want the research to help understand the nature of intelligence. I want the technology to empower all humans that don't want to see the world burn or are not dictators. I want the power of it be used for good. I want the power to not be concentrated. I want to see it developed safely and ethically in steerable way. I want people to get compensated properly. I'm trying to push that and help to work towards these goals more! AI can be used for both bad, good, and neutral things. Let's maximize the good usecases! - What are the benefits, risks, impact and future of artificial intelligence? How will the current artificial intelligence revolution transform humanity technologically, economically, culturally, governmentaly? How to make sure that AI benefits everyone, such as by automating mundane tasks (dishes, laundry), science (AlphaFold in biology), physics (FermiNet), mathematics (AlphaProof), healthcare (diagnosis), technology development (recursive self-improvement), programming (copilots, autonomous software engineers), preventing various risks (biorisks), useful chatbot assistants and robots factually grounded in reality etc.? How does existing technology already make us cyborgs? - Is artificial intelligence a tool like scissors, or like internet, or like electricity, or as powerful as nuclear weapons, or even more powerful and AI systems will populate the whole galaxy, or are we growing new species that will require moral rights? How can we collectively create optimistic stories about our future and build that great future together? Will there be post-scarcity economy where technology generates abundance for all, not just for select few? How to make sure that people in the AI and in general the fourth industrial revolution with exponential automation don't suffer? Maybe something like universal basic income or services will be needed to catch up with lob loss with increasing automation? Is universal basic income or services realistic? How to minimize power concentration in the hands of the few? How to prevent realistic risks? Is rogue superintelligent AI likely? How to prevent regulatory capture? Is singularity near? How will singularity look like? Science, Technology, Engineering, Mathematics, Physics, Biology, STEM - How does reality, science, technology, engineering, mathematics work? What is the structure of everything, what equations govern everything across all scales, what is the source code of our reality? How does astrophysics, celestial mechanics, etc. emerge from lower scales? How does sociology emerge from neuroscience and biology, and how that emerges from chemistry, and that from physics? What are the answers to the questions in cosmology? What is the best simplest most predictive and explanatory, most useful, integrating, unifying model in all natural sciences, using all its applied mathematics methods, like linear algebra, calculus, differential equations, geometry, topology, discrete mathematics, probability theory, statistics, graph theory, etc. with the help of pure mathematics? How can we use the methods of physics in as many fields as possible? How to effectively map all of knowledge and follow state of the art in many fields at once? How to most effectively create a generalist synthesis, but also narrowly model reality concretely on each level of abstraction on all scales, scruting the seemingly inscrutable reality's quantum fields with it's emergent laws? How do we turn the seeming alchemy of empirical sciences into deep understanding of its underlying mechanisms by the most optimal mathematical compression? How can AI accelerate this process? How can we infer the best tools for different domains for different usecases? How can we understand and unify all mathematical subfields using abstract mathematics such as category theory? How can we integrate together all methods from different physics subfields, such as tools from classical mechanics, statistical mechanics, quantum mechanics, quantum field theory, and so on? How to solve quantum gravity? Are string theory and loop quantum gravity good solutions to quantum gravity? How to create a theory of everything, theory of everything in fundamental physics, theory of everything in all of physics, theory of everything in natural sciences, theory of everything in all of science? Why did big bang happen? What if alternatives to big bang like big crunch happened instead? Did it actually happen? Why is universe governed by few fundamental forces between tens of elementary particles? Why is the standard model and general relativity the best current description of it that we have so far? Why do we struggle with unifying quantum mechanics and general relativity so much? Is theory of everything even possible? What even is space? What even is time? Is there such thing as "before the big bang" if time might not have existed before it? Why and how did chemical elements exactly emerge? Why and how did life exactly emerge and how does it work? Why is evolution such unreasonably effective algorithm? Why and how exactly is there such mindblowing specialized diversity of life? Brain, mind, and body - What is life? How does the brain work? How to integrate the lenses of biology, neuroscience, chemistry, computational neuroscience, statistics, probability theory, physics, machine learning, systems theory or other mathematics in terms of understanding the brain? How does learning work, what kind of intelligence is biological intelligence compared to artificial Intelligence, what is the intersection between biological and artificial intelligence? How to reverse engineer and amplify human intelligence, agency, wellbeing and longevity to catch up to exponentially increasing machine intelligence using neurotechnology, biotechnology, and other methods? How does wellbeing, agency, drive, productivity, meaning etc. work on the level of the brain and the whole society according to cognitive science? On what level are factors influencing these phenomena genetic and on what level are they environmental? What is the physical substrate of experience? How does experience arise? What's the best philosophy of mind position? What are the best psychotechnologies and neurotechnologies strenghtening or transforming the neuroscience and software of the brain, for example intelligence, wellbeing and longevity, like brain computer interfaces, meditation, philosophy, psychotherapy, selfhelp, culture, substances, cold exposure, or just healthy lifestyle, what are the realistic limits? How can we upgrade our sentient substrate collectively? How to achieve longevity? Is immortality solvable engineering problem and how? Technorealism, future of humanity, AI, sentience, futurology, politics - How to be not naively technooptimistic or technopessimistic when it comes to technology, but something in the middle, technorealistic? How can we gather the benefits of technology and minimize the risks of artificial intelligence and technology in general? How can we merge the ideas of Effective Altruism and Effective Accelerationism? How can we the most effectively use science, technology and other methods to adapt or solve the biggest world problems and prevent risks, from suffering from for example poverty, wars, injustice, crises? How to prevent existential risks such as natural or engineered pandemics, nuclear war, environmental collapse? How is polycrisis actually real and how do we solve it? How to use AI and other technology to solve and prevent all these problems? How to prevent technology itself becoming too powerful or in the hands of the wrong people like dictators? Should it be open source? What is the best political system? Is it liberalism, collectivism, individualism, global governance, benevolent dictators, AI assisted governance, global AGI governance, decentralization, democracy, anarchism, minimal state, and so on? Should we regulate more, or less, and regulate what and when? - How do we collectively steer all of sentience not into oppressive dystopia without democracy, or complete extinction, or plateau without progress, but into collective protopia, collective growth of science, technology, intelligence, wellbeing, connection, knowledge, drive, motivation, freedom, agency, survival, longterm adaptability, stability, sustainability, love, peace, safety, meaning, fulfillment, selfactualization? Metamathematics and philosophy - What is the most useful logic and foundations of mathematics and metamathematics like set theory, homotopy type theory or category theory? What are the most beautiful, novel and exotic parts of formal sciences? Can we mathematize ethics? What is the best positions in metaphysics with ontology? What is the best epistemology? What is the best interpretation of quantum mechanics? How does identity work? What are the best positions in philosophy for science, meaning, wellbeing and freedom? Why is there something rather than nothing? Why can we ask this question? Does asking this even make sense? How to create theory of everything, theory of everything in philosophy, theory of everything in culture, meta theory of everything? Beyond polarization - How can we collectively effectively communicate and understand what is empirically true by steelmanning eachother, accelerating omniperspectivity, bridging between eachother, instead of polarizing tribalism more disconnected from reality, leading to improved collective decision making? How do we understand our limited computational power of our brain, limited data and perspective, when we consider ourselves as information processing agents that model the world's enormous growing complexity together to collectively flourish in the future, future of humanity, AI, sentience, futurology, politics, and future of universe? How can artificial intelligence help us in this process? The ultimate existential challenge - How to beat the ultimate existential challenge, the second law of thermodynamics, how to survive the death of the universe? How can we together achieve resistance to entropy, optional transhumanistic merging with eachother or machines or the universe, and so on, for every being, using the most optimal collective emergent selforganized sentient coordinated thermodynamic cybernetic architecture that might expand into the whole universe and become a beautiful cosmic constellation of linked posthumanist clusters of sentient matter of any form it wishes to shapeshift into, such as raw computronium? Infinite curiosity - Why is there something rather than nothing? Why can we ask this question? Does asking this even make sense? Why did big bang happen? What if alternatives to big bang like big crunch happened instead? Did it actually happen? Why is universe governed by few fundamental forces between tens of elementary particles? Why is the standard model of particle physicsand general relativity the best current description of it that we have so far? Why do we struggle with unifying quantum mechanics and general relativity so much? Is theory of everything even possible? What even is space? What even is time? Is there such thing as "before the big bang" if time might not have existed before it? Why and how did chemical elements exactly emerge? Why and how did life exactly emerge and how does it work? Why is evolution such unreasonably effective algorithm? Why and how exactly is there such mindblowing specialized diversity of life? Why and how did intelligence emerge and how does it work? What are the best definitions of intelligence? Why are brains and artificial intelligence systems so unreasonably effective in different complementary ways? How can they be upgraded? What happens to consciousness after death? Why and how did consciousness and experience emerge and how does it work? What are the best definitions of consciousness? What is the solution to the hard problem of consciousness? Does this question even make sense? What even is consciousness in the first place? Why are be able to design so many technologies that allow us to manipulate the universe to such degree? Why does emergence happen in the first place? How will the universe end? Is there such a thing as end of the universe? Is the multiverse theory true? Why is mathematics so unreasonably effective at describing and predicting nature? Is there a better mathematical foundation than set theory, type theory or category theory? Is mathematics invented or discovered? Is mathematics fundamental language of reality or just our mental tool to survive? What even is reality? What is being? Why can we even ask all of these questions? Do many of these questions even make sense and are they any final answers to them, or answers we get are just getting closer to to us incomprehensible "truth", or they have many parallel answers, or many answers are differently relatively valid depending on the assumptions we start with, or are they fundamentally unanswerable? Effective Curiosity, Effective Omni - Effective Curiosity = Maximizing the total understanding of reality by building models of as many physical phenomena as possible across as many scales of the universe as possible, that are as comprehensive, unified, simple, and empirically predictive as possible! Intelligence and fundamental physics, which are subsets of this, are the most fascinating to me! - Effective Omni = Steelman and if possible verify all models from all disciplines, all theoretical, applied, natural, formal, social sciences, all movements with worldviews shaping the future, all overall philosophical or other perspectives, and synthesize them all on a higher level of complexity as compatibly as possible for collective survival, wellbeing, flourishing and growth of all of sentience, increasing intelligence and ascending the Kardashev scale! Let's integrate it all into one useful framework! Reduce suffering in the universe! Increase prosperity in the universe! Increase understanding in the universe! The best way to do that is with AI and other technologies from the fourth industrial revolution! Trying to understand the equations of intelligence, our world, and the universe, and applying them to build technology for the benefit of all! Effective Curiousity (Supercuriousity)! Effective Understanding (Superunderstanding)! Effective Intelligence (Superintelligence)! Effective Longevity (Superlongevity)! Effective Wellbeing (Superwellbeing)! Effective Flourishing (Superflourishing)! Effective Omni (Superomni)! Effective Omni! - Effective Curiousity (Supercuriousity), Effective Understanding (Superunderstanding) = The quest to uncover all mathematical patterns of the universe and all of physical and platonic reality! - Effective Intelligence (Superintelligence) = Intelligence so advanced in every dimension that it surpasses human comprehension, vastly more advanced than any currently existing physical system! - Effective Longevity (Superlongevity) = Extending lifespan, achieving immortality, and overcoming the heat death of the universe! - Effective Wellbeing (Superwellbeing) = Complete fulfillment of Maslow’s hierarchy of needs, and similar models of wellbeing, at their highest possible levels! - Effective Flourishing (Superflourishing) = Collective superwellbeing! Contact and links - My website with my exocortex in wiki format: burnyverse.com, burnyverse.com/Exocortex , burnyverse.com/Home - Substack: substack.com/profile/160971… - BlueSky: burnytech.bsky.social - Youtube channel: @burnytech" target="_blank" rel="nofollow noopener">youtube.com/@burnytech - Discord: burnytech - Discord server: discord.gg/2vyjxYTDMU - Patreon: patreon.com/BurnyTech - LinkedIn: linkedin.com/in/libor-buria… - Telegram: @burnytech - Mastodon: @Burny" target="_blank" rel="nofollow noopener">mathstodon.xyz/@Burny - Facebook: facebook.com/burian.libor/ - Gmail: burian.lib@gmail.com - Github: github.com/BurnyCoder - HuggingFace: huggingface.co/BurnyCoder

After 100 iterations, the outer loop discovered seven improvements over the baseline. Including a new search policy, a memory system that compresses prompt by 16x, and a layered defense against reward hacking. (3/7)

What do you think? ✍️

“Proxy Exploration and Reusable Guidance” Post-training usually forces the large model to do expensive RL exploration itself. This paper, PUST, moves that exploration to a smaller proxy model, then transfers only the learned update direction, not the proxy’s final distribution. This lets small models search once, cache the signal, and reuse it to improve bigger models across math and code, with Qwen3 4B signals giving strong gains on Qwen3 8B.

[arXiv] Beyond Cosine Similarity arxiv.org/abs/2602.05266 I almost always see cosine, and it was interesting to read about similarity metrics as classes of more and less restrictive similarity measures.

Highly-recommended overview of metacognition in LLMs. (bookmark it) Interesting behaviors in LLMs like confidence calibration, self-verification, knowing when to stop, and knowing what you do not know have mostly been studied in isolation. This survey argues they are facets of one thing, metacognition, and proposes a comprehensive map of it. The authors taxonomize methods and benchmarks for measuring and evaluating metacognitive abilities in LLMs, then connect those abilities to capability, reliability, and transparency. As agents take on longer horizons, the ability to monitor and regulate their own reasoning becomes an important way to measure reliability. Paper: arxiv.org/abs/2607.11881 Learn to build effective AI agents in our academy: academy.dair.ai

@eshear Ha

@burny_tech Overgeneralizing you might even say

i feel like each few months there is some new "we solved how generalization works in deep learning!" paper which makes me think they're all overselling

@prajdabre Someone activate the signal!

Our team spent months developing RLFR, our method which uses probes on a model's internals as reward signals for RL. Silico reproduced it in 2 days, reducing hallucinations in Qwen3-8B by 37% without capability loss. (3/6)

Babe wake up, someone reinvented the LSTM again.

> "a pivotal moment in human history" - Hassabis The Overton window has shifted massively in AI discourse in the last 6 weeks. We might be languishing in the clutches of AI psychosis, but given the caveat emptor, this is ringing less idiosyncratic with every passing day. We live in strange and fascinating times... I suspect both worlds are possible simultaneously, it is a pivotal moment, and also, lacking in important and unexpected ways. I don't know what to make of the proposed standards body, Prima facie; it reads like cynical corporate- and US-capture (to the obvious detriment of the open weights and China ecosystem). But given recent progress, I believe it's coming from a genuine place. > "working with appropriate federal agencies and the US National Labs to conduct testing in areas relevant to national security ... model would qualify as ‘Frontier-class’ if it meets certain thresholds on a set of benchmarks." It reads very US-centric, and US-controlled. After the recent Fable wake-up call, it will be an uphill struggle getting the rest of the western world to sign up to that existential dependency.

A Research Scientist at Google DeepMind just dropped a 58 page paper on building agents that specialize in game theory. Here are the most important parts:

“Deep learning is alchemy” may be the most repeated criticism in AI. It also misses the mark. Alchemy failed to deliver results. Deep learning, by contrast, has produced transformative technologies. And fields like medicine are only partially understood without being deemed alchemical. So calling AI “alchemy” captures part of the problem, but not all of it. Modern AI is not simply undisciplined experimentation. It contains significant amounts of rigor. But we still struggle to answer basic questions: • Do models understand? • Why do they generalize? • When will they fail? The deeper issue is that rigor takes different forms—and in AI, those forms are unevenly developed. My new paper distinguishes three: • Conceptual rigor: coherent terminology and paradigms • Epistemic rigor: reliable scientific understanding • Operational rigor: reliable performance and deployment This framework helps explain both the extraordinary progress of modern AI and the uncertainty surrounding it. Conceptual rigor asks whether the field knows what it's talking about. • What exactly is intelligence? • What qualifies as AGI? • What does it mean for a system to be aligned? Consider the debate over whether current models are intelligent. One person points to their breadth of performance. Another points to weak planning. Another emphasizes sample inefficiency. Another asks whether it has a grounded model of the world. They appear to disagree about one property. Often, they are evaluating four. This is why conceptual clarity matters in practice. Questions about intelligence, understanding, AGI, and alignment do not remain confined to philosophy: they shape how things are measured, optimized, and built. Epistemic rigor asks whether empirical success has become scientific understanding. The paper focuses on three criteria: • Can findings be reproduced? • Can behavior be predicted in advance? • Can success and failure be explained? AI experiments are unusually reproducible in principle: code, data, and models can be copied. But conclusions may still depend heavily on random seeds, hyperparameters, implementation choices, benchmark selection, and compute budgets. Reproducing a number is not always the same as reproducing the conclusion drawn from it. Prediction is harder. Scaling laws can forecast some training outcomes. Infinite-width theory can lead to more tractable settings. Classical learning theory explains important pieces. But we still lack broad principles telling us when a model will generalize, fail under distribution shift, or remain robust under adversarial perturbations. Explanation is harder still. Neural networks are mathematically specified, yet their learned features resist human interpretation. A behavior may arise from training data, optimization dynamics, internal representations, or interactions among all of them. The system is transparent in code but opaque in meaning. Operational rigor is where modern AI is strongest: benchmarks, evaluations, monitoring, red-teaming, and deployment controls. The field has become highly effective at improving systems without first obtaining a scientific theory of them. Benchmarks turn capabilities into measurable targets. Post-training shapes behavior. Tools and scaffolding compensate for model weaknesses. Operational rigor can therefore partially substitute for scientific understanding. That imbalance defines the deep-learning era: • Capabilities rise rapidly. • Explanations lag behind. • Benchmarks become optimization targets. • New systems generate new phenomena faster than theory can absorb them. AI is advancing while continually changing the object that science must explain. For AI to mature as both a science and a technology, it will require all three forms of rigor: • Clearer concepts to define our goals. • Stronger science to predict and explain system behavior. • Better engineering to make systems genuinely reliable. The future of AI depends not simply on demanding “more rigor,” but on identifying which kind is missing—and understanding how the imbalance shapes what we can build, know, and control.

Moving from conventional ML to continual learning requires revisiting even the fundamental concepts such as “test”/“train” time. LLMs Need Sleep and Dreaming! We introduce a phase, where the model consolidates its fragile short-term memories into stable long-term memories, and then dreams to recursively self-improve over time. For memory consolidation, we introduce a new form of distillation, called Knowledge Seeding (KS), where a small model(s) distills its knowledge to a larger model. Our experiments on continual learning and reasoning tasks show that this new phase can help the model to perform better and relatively better mitigates catastrophic forgetting.

Overall I think 5.6 Sol Pro/Ultra etc. seems to be a substantial step up from 5.5 for math. That said, common interaction pattern is: I ask a question. It thinks for ~100+ minutes and returns a largely inscrutable response. I ask it to explain. It thinks for 20 minutes and says:

All abstractions and analogies are leaky

I'm so excited about this new work. We introduce requential coding, pushing compression to the absolute limits! We show larger neural networks and ensembles are much stronger compressors, even in an absolute sense, with better generalization bounds.

How do physical systems achieve collective intelligence and self-repair without a central brain? A new paper published today in Nature Communications by my Sakana AI colleague Sebastian Risi (@risi1979), along with co-authors from IT University of Copenhagen and Autodesk Research, presents a beautiful realization of biologically inspired robotics: Smart Cellular Bricks. The team built a system of physical 3D cubic units that can collectively infer their global shape and autonomously guide their own damage recovery using purely local interactions. Here is a deep dive into the paper’s key contributions: 1/ Neural Cellular Automata-based Architecture: Modular robots usually rely on central processors. This system flips that paradigm. Every block independently runs the exact same neural network on local microcontrollers. With no master plan or global coordinates, they communicate only with immediate neighbors. By passing continuous state vectors, hundreds of bricks achieve global consensus on their shape in under 3 minutes. 2/ Emergent Biological Morphogens: How does a block know it is part of a chair, not a table? The network’s internal memory automatically learns to establish continuous gradients across the structure. This beautifully mirrors how biological morphogens give positional info to developing cells. The bricks naturally form left-right, radial, and head-to-tail axes to align their identity. 3/ Performance and Generalization: Validated in large-scale simulations, the networks transferred seamlessly to nearly 200 physical hardware bricks, achieving a 100% convergence rate. Instead of rigid template-matching, the system infers broad categories. Even when tested on unseen variations, like an asymmetric table with five random legs, the collective correctly classified the structure. 4/ Fault Tolerance and Autonomous Damage Recovery: Hardware fails in the real world. This system easily tolerates up to 15% module failure without losing accuracy. By predicting spatial damage directions, the cells pinpointed missing components with 95% accuracy. They actively use these local signals to guide a self-repair process, regenerating back into the intended morphology. I believe this is a significant piece of research, bridging collective intelligence and Physical AI. This work represents the first successful physical realization of large-scale, decentralized 3D self-recognition and damage detection. By moving away from centralized control, this architecture paves the way for highly adaptive smart materials and resilient robotics that can survive and repair themselves. Read the full open-access paper: nature.com/articles/s4146… Congratulations to the team on this achievement!

@zhengyaojiang Impressive! And now also self-improve the architecture and weights of the underlying model powering the agent harness.

The first experimental evidence of recursive self-improvement (RSI). Autoresearching the autoresearch agent for eight days. The result beats the harness we hand-tuned for two years, on held-out benchmarks: 🧵(1/7)

Impressive! And now also self-improve the architecture and weights of the underlying model powering the agent harness.