Luke Darlow

Whereas before it wouldn't be unheard of for me to "understanding while coding". Several months ago I was convinced that part of my design and articulation process was enveloped in the actual coding. I think I was wrong.

Conversations about the impact of agentic work on humans is largely doomerist, and I dislike that. An unexpected positive is that I'm now required to design, think deeply, and articulate clearly what I want a system to do. This feels a lot like studying and learning.

"That feeling" of typing out a plan prompt for Claude that you couldn't fall asleep with last night because it was rattling around in your head. Several paragraphs, many enumerations, verbose instructions. Finally hitting enter and sipping on coffee feels so much like sci-fi.

@MLStreetTalk I think that another way of phrasing "laziness" is a hunger for bootstrapping. Building a function, particularly one that can compose with other functions, is addictive. It's kinda deep how closely linked this is with program synthesis, or even computational life (DNA bootstraps)

The number one virtue of a programmer is laziness. Larry Wall nailed it in 1991 in Programming Perl. Every dev knows the feeling: you'd rather spend ten hours writing a script than ten minutes doing the boring thing by hand. The other day I pointed Claude Code at my receipt backlog in emails/vendor sites. Used QuickBooks CLI, Google Workspace CLI, browser automation etc. It uploaded 300+ receipts and categorised them all, and the cherry on the cake -- a summary email fired off to my accountant! I actually enjoyed it, it felt like I had dev-ified the task. It still took ages and was an iterative/interactive process (like all good AI actually is and always will be), but it was actually fun! Agentic AI is a developer's wet dream. AI makes miserable tasks genuinely fun because you're solving an interesting novel orchestration problem every time instead of manually clicking the download button on a hundred PDFs. Still a tonne of tacit technical knowledge needed though, accountants are safe for a while 😃

@xandykati98 The solution to speeding up is JAX, without a doubt. I've recoded and improved the CTM V2 substationially on JAX. Honestly, I suggest just trying to set some CC agents on it, converting the CTM repo to Jax, and experiment from there. I think I got about 10x speedup.

@LearningLukeD hey could we do multi-task learning with continuous thought machines by wiring up different neuron pairs sets for each task? I did some experiments on my implementation but never got far because of compute, it seemed to work but performance overall got a hit.

Here's another with a different set of hyper-params. Again, super interesting how the different functional layers of the Continuous Thought Machine tend to occupy different spatial structures. Mostly, though, I'm showing off the fun and useful visualization we've built. Pretty!

My updated Continuous Thought Machine v2's neural activation patterns can be so mesmerizing. This CTM has 4 separate dedicated layers (input, attention, output, and motor control). It turns out that they learn to occupy local spaces in 2D UMAP space, just like... brain regions??

Robert Lange @RobertTLange from @SakanaAILabs on ShinkaEvolve -- an open-source framework combining LLMs with evolutionary algorithms for scientific discovery, with insane sample efficiency. His thesis that current systems optimise solutions to fixed problems. Going forwards -- real scientific discovery requires co-evolving the actual problems. By the way - NVIDIA GTC is coming and will showcase breakthroughs in physical AI, AI factories, agentic AI, and inference. Register for virtual GTC for free using my link: nvda.ws/4qQ0LMg and enter raffle to win a DGX Spark 😈

Things you should use LLMs for on the regular as an AI Research Scientist: - Upgrade your experiment visualization tactic (I do this once every 2 weeks). See more. Know more. Do more. - Squeeze those GPUs. Evaluate. Streamline. Speed. - Rethink assumptions. Unearth. Validate.

For me, 2026 LLMs let me: - rethink what "difficult" means - do the boring stuff nearly perfectly and rapidly - hold much more complex and unwieldy NN architectures in my mind - build insanely dynamic portable experiment visualizations - brainstorm without the lag - create more

@Offshore_369 Never seen it in my life! What on earth?!

@LearningLukeD You created this, right?? Ca: JAQcEYUi8fSizsFDACbJKyNFu2QKbphxndgshWMqBAGS

If you’re interested in learning about Continuous Thought Machines (sakana.ai/ctm/), we made interactive notebook tutorials so you can hack around with CTMs ImageNet: github.com/SakanaAI/conti… MNIST Tutorial: github.com/SakanaAI/conti… Let me know if you have any feedback!

Sakana AI共同創業者 Llion Jones（@YesThisIsLion）のTED AIトークが公開されました。目標を定めすぎないオープンエンドな研究がブレークスルーを生む理由、Transformerの成功が業界にもたらした状況、それを乗り越える次の構想と成果を語りました。 ted.com/talks/llion_jo…

We are thrilled to announce a strategic partnership with Google! Google is also making a financial investment in Sakana AI to strengthen this collaboration. This underscores their recognition of our technical depth and our mission to advance AI in Japan. We are combining Google’s world-class products with our agile R&D to tackle complex challenges. By leveraging models like Gemini and Gemma, we will accelerate our breakthroughs in automated scientific discovery. Our work on The AI Scientist and ALE-Agent has already demonstrated the power of these models. Now we are going further. We are scaling our deployment of reliable AI in mission-critical sectors. We are working with financial institutions and government organizations to deliver solutions that meet the highest standards of security and data sovereignty. We are excited to drive the widespread adoption of reliable AI and advance Japan’s AI ecosystem together!

@cloneofsimo M-Layers! We talk about the spiral problem and how it reveals issues in how NNs currently do things in our MLST interview (youtube.com/watch?v=DtePic…). M-Layers actually fit spiral data much better than standard MLPs. Nice.

What are some interesting, unconventional idea you have seen recently? I will go first.

@JonathanRoseD Sure thing, but could you elaborate which bit you're talking about? The MLST interview?

@LearningLukeD Brother, I just saw your bit about the transformer using leap-frog style pathfinding. I have a side obsession with pathfinding stuff and was blown away. Where can I find more details? Like, what strategy is used for the 'leap'?

We worked on a guide for researchers applying to Sakana AI. I highly recommend reading this if you're either currently going through our process or plan to in the future.

Introducing DroPE: Extending the Context of Pretrained LLMs by Dropping Their Positional Embeddings pub.sakana.ai/DroPE/ We are releasing a new method called DroPE to extend the context length of pretrained LLMs without the massive compute costs usually associated with long-context fine-tuning. The core insight of this work challenges a fundamental assumption in Transformer architecture. We discovered that explicit positional embeddings like RoPE are critical for training convergence but eventually become the primary bottleneck preventing models from generalizing to longer sequences. Our solution is radically simple: We treat positional embeddings as a temporary training scaffold rather than a permanent architectural necessity. Real-world workflows like reviewing massive code diffs or analyzing legal contracts require context windows that break standard pretrained models. While models without positional embeddings (NoPE) generalize better to these unseen lengths, they are notoriously unstable to train from scratch. Here, we achieve the best of both worlds by using embeddings to ensure stability during pretraining and then dropping them to unlock length extrapolation during inference. Our approach unlocks seamless zero-shot context extension without any expensive long-context training. We demonstrated this on a range of off-the-shelf open-source LLMs. In our tests, recalibrating any model with DroPE requires less than 1% of the original pretraining budget, yet it significantly outperforms established methods on challenging benchmarks like LongBench and RULER. We have released the code and the full paper to encourage the community to rethink the role of positional encodings in modern LLMs. Paper: arxiv.org/abs/2512.12167 Code: github.com/SakanaAI/DroPE

Introducing Digital Red Queen (DRQ): Adversarial Program Evolution in Core War with LLMs Blog: sakana.ai/drq Core War is a programming game where self-replicating assembly programs, called warriors, compete for control of a virtual machine. In this dynamic environment, where there is no distinction between code and data, warriors must crash opponents while defending themselves to survive. In this work, we explore how LLMs can drive open-ended adversarial evolution of these programs within Core War. Our approach is inspired by the Red Queen Hypothesis from evolutionary biology: the principle that species must continually adapt and evolve simply to survive against ever-changing competitors. We found that running our DRQ algorithm for longer durations produces warriors that become more generally robust. Most notably, we observed an emergent pressure towards convergent evolution. Independent runs, starting from completely different initial conditions, evolved toward similar general-purpose behaviors—mirroring how distinct species in nature often evolve similar traits to solve the same problems. Simulating these adversarial dynamics in an isolated sandbox offers a glimpse into the future, where deployed LLM systems might eventually compete against one another for computational or physical resources in the real world. This project is a collaboration between MIT and Sakana AI led by @akarshkumar0101 Full Paper (Website): pub.sakana.ai/drq/ Full Paper (arxiv): arxiv.org/abs/2601.03335 Code: github.com/SakanaAI/drq/

What's missing from modern AI, you ask? Sometimes turning to sci-fi helps to raise the right questions. "... a computer's speed and ability to fine -grind the details, a sentient mind's eye for what actually mattered." From Eyes of the Void by Adrian Tchaikovsky, @aptshadow

Naming your outputs 'model_final.py' or 'model_solved_final.py' does not an LLM right make.