alex zook retweetledi
At FutureHouse, we’ve noticed scientific agents are good at applying average intelligence across tasks. They always seem to make the obvious choices, which is good, but discovery sometimes requires more intuition and insight than average.
We’ve made the first step today towards superhuman insight by training a reasoning model for a specific domain of science: designing drug-like molecules.
We’re releasing a 24B open-weights reasoning model called 𝚎𝚝𝚑𝚎𝚛𝟶. 𝚎𝚝𝚑𝚎𝚛𝟶 has been trained with reinforcement learning to exceed frontier and human experts across a range of molecular design tasks. 𝚎𝚝𝚑𝚎𝚛𝟶 takes in natural language, reasons in English, and outputs a new molecule.
𝚎𝚝𝚑𝚎𝚛𝟶 is now a tool for our chemistry design agent, Phoenix, which can call upon it to design molecules.
Training a reasoning model for a scientific domain like chemistry, rather than math or programming, required a number of small technical advances. For example, we developed an iterative method of split specialist models and aggregation of reasoning traces. Another example is we used LLMs to rewrite questions that were partially solved.
A major finding from this work is that we can train with >10x efficiency per experimental measurement when using a reasoning model, rather than fine-tuning. We also found that reasoning models can learn new tasks, developed specifically for this paper and not in pretraining corpora. We even saw a task have 0% performance until 100 steps into RL, at which it randomly solved once. This, along with our change in modality from natural language to molecules, bodes well for applying reasoning models far from natural language.
Reasoning models in science are the future. Scientific tasks are naturally verifiable rewards: the physical world is the ultimate arbiter of accuracy, rather than human contractors. The data efficiency gain and ability to exceed frontier models with relatively few parameters/compute mean that we should expect more scientific reasoning models soon.
Congrats to team @SidN137, James, @Ryan__Rhys, Albert, @GWellawatte , @maykcaldas , @ludomitch , and @SGRodriques. Thanks to @VoltagePark @nvidia and @huggingface for supporting us, and huge thanks to @ericschmidt for funding @FutureHouseSF
The model weights, reward model, and new benchmark are open source. You can also read more about scientific reasoning models in our exclusive with Nature.
English
