Adam Hunt
3.8K posts
@RealAdamHunt
Researcher @Cambridge_Uni. PhD in evolutionary psychiatry. Explaining neurodiversity, improving methods & stigma. 'Evolving Psychiatry' podcast host.
I’m reluctant to get into an argument with @profbriancox, who is a great advocate for science. But I would like to stand up for UKRI’s strategy here, which I think is trying to do something really important – both for curiosity-driven research and for the wider benefits that research and innovation can bring to everyone in the UK. (Fair warning: if you’re expecting a snappy 280-character tweet, this is not one.) Everyone knows that Britain has a proud track record of amazing basic research; we’re 0.8% the world’s population and 8.4% of the academic citations. If scientometric stats don’t do it for you, consider some of the greatest hits. The Human Genome Project, the RECOVERY trial, gravitational waves, AlphaFold, graphene – all significant breakthroughs of recent decades with a big UK contribution. But the UK’s track record of successful innovation – of translating research into real-world impact, or of doing applied R&D for practical benefit – is more mixed. I’d like to see more R&D-intensive businesses in the UK creating good jobs, drawing on the UK’s scientific expertise. And a stronger connection between research and its application, to make the most of the potential of great research to improve people’s lives. The point of UKRI’s strategy is to invest in curiosity-driven research, while also doing a better job of backing translational R&D to improve people’s lives and drive economic growth. Prof Cox argues this is ‘short sighted’ and ‘short term’. I strongly disagree. Take our investment in Quantum Research Hubs and the wider quantum research strategy. These build on 20+ years of research council investment in quantum research, and combine pure scientific research with application, and also links to quantum start-ups, established businesses, public procurement, investors, and the Government’s wider industrial strategy. So I think UKRI’s strategy is the right thing to do in the national interest. I also think it’s the right thing to do pragmatically, even if all you care about is curiosity-driven research. To explain why, we need to talk about the process how the government allocates investment, including for scientific research. I hope I’m not giving away anything I shouldn’t. At fiscal events like the 2025 Spending Review that determine UKRI’s current budget, organisations like UKRI aren’t given a pot of money and told to spend it how they like. The spending negotiation is about “how” as well as “how much”. My reading of last summer’s negotiations is that the financial settlement for R&D – which as Prof Cox says was ‘quite generous’, especially at a time of straightened public finances – would have been significantly less generous if UKRI and DSIT had proposed to focus spending purely on curiosity-driven research, and that had we done that, curiosity-driven research would now be in a worse position, not a better one. I don’t say that because the last Government (or the previous Government for that matter) are against curiosity-driven research; on the contrary, they’ve been great advocates for it. But there are many other demands on the public finances, and all the research I’ve seen (and indeed, that I’ve done) on voters’ feelings about science is that application – and the practical benefits science brings – really matter to a lot of people. What happens in future fiscal events is obviously the new Government’s choice, but I’d personally be surprised if they see this issue radically differently, and as Prof Richard Jones has observed, no-one should take generous research budgets for granted. So my case is that the UKRI strategy, and its focus on advancing knowledge, improving lives and driving growth, is the right thing to do from a national perspective, *and* would also be the right thing to do even if all you are interested in is curiosity-driven research. If you’ve made it this far, thanks for reading.
1/ As AI agents become increasingly capable, what must *inevitably* emerge inside them? We prove selection theorems: strong task performance forces world models, belief-like memory and—under task mixtures—persistent variables resembling core primitives associated with emotion.
Recently I've flipped from being bullish to being bearish about AI. I think I'm updating my bearishness to be more solidly bearish. Early thoughts (which I hope to be disproven in the next year or so, I would prefer progress) and my reasoning: The whole 'it turns out if you keep training and scaling the models more they develop broad new capabilities in lots of domains' thesis is wrong (sorry Demis). The recent batch of models haven't got more general, they've got less general. This is most obvious in the fact that their language outputs have got much worse in comparison to e.g. o3. If they were gaining generalist capacities we would expect them to be describing their work in ever more graceful and comprehensive prose! The image that was being shared as the AGI thesis (November 2025, Tomas Pueyo) was the spiky bubble that has a current spike or two out past human capabilities (e.g. on coding or math) but below human on other capabilities on the other spikes - the future prediction was that as the models scale/advance, every spike would grow bit by bit until the whole center encompasses the human capabilities, with super-superhuman on some spikes. I think it seems like what's actually happened in the last few models has been that the coding/math spike has grown, but leaving behind or even at the cost of the other spikes. The models are no better at some simple logic, language (and sometimes worse!). This makes sense from a simple RL perspective; you can't RL something endlessly on one domain of tasks and expect it to improve on the other tasks. The fact that early LLMs did seem to improve generally was a byproduct of the written language corpus covering everything - that corpus is general, so training it on that gave the appearance of something generally intelligent and becoming more generally intelligent as it got better at replicating that corpus. But the actual logic and underlying ground truths behind the language aren't captured efficiently enough and weren't effectively RLd in - they top out at some point (I guess this happened around the time that there was the 'has scaling hit a wall' discussion in late 2024). Chain of thought was then a genuine breakthrough, along with web search, which plugged into that general LLM global-corpus intelligence to lead to post 2024 gains. The AI companies have since worked out that coding works (and pays) really well (basically this is because the entire job is nearly perfectly recorded and exists as training data, and you can set up clear benchmarks and rewards). The recent models (and benchmarks) have been maxxing that and we've seen degradation on normal English use for that reason. This could still be transformative, leading to extremely powerful (and potentially dangerous, particularly in cyber security) models but it's not a pathway to AGI. I'm probably at about 40% confidence about this. It fits my current observations of AI progress and has a basic explanatory model. It doesn't account for potential breakthroughs, which is a major reason for discounting. To make some predictions, I guess if I'm right this will become broadly apparent and more widely acknowledged in the next year or two, as we see how the spikiness of models that keep getting released develops. Maybe there will be efforts to concentrate on specific spikes e.g. health or law which require going back to earlier models and RLing on a different data set/with different rewards/benchmarks. Maybe those separate models can be linked together to give a more apparently general model. How capital intensive that is/the potential profitability will be a defining question. But I just don't see general abilities emerging atm, and I don't think we will any time soon. Good news - a whole industry of tackling important specific problems/sectors can open up!
