Hunter Jay

Superintelligent AI is possible in the 2020s -- progress continues to outpace predictions, and the trends in benchmarks and compute are unwavering.

@MKinniment I'm working on a paper in this direction right now, should have a draft out within a week or so.

I wonder what would happen if we let the models apply steering vectors to themselves?

We release a new application of the METR time-horizon methodology to offensive cybersecurity, grounded in a new human expert study with 10 professional security practitioners. Offensive cyber capability has been doubling every 9.8 months since 2019. Accelerating to every 5.7 months on a 2024+ fit. Opus 4.6 and GPT-5.3 Codex sit well above both trendlines again, reaching 50% success on tasks that take human experts ~3 hours. Furthermore, our 2M-token evaluations materially understate current frontier capability. Recent progress has likely moved faster than these numbers suggest.

@cauliflwr_human You reminded me of something John Cleese said about David Frost that was like this. Paraphrasing, "He was the opposite of paranoid. A pronoid." "He was just completely convinced everyone was out to help him." Seemed to work well enough for Frost!

thinking about super-cooperators -- seems important how do you identify super-cooperators with enough confidence to form useful networks between them? how do you cultivate super-cooperators emotional wellbeing and agency? how do you protect super-cooperators from exploitation?

Again I want to stress these are vibes, not a considered opinion. I expect to change my mind quickly once challenged with evidence my guesses are wrong.

It also lets you give clear, recordable, updatable beliefs. So, spitballing: Anthropic -- Leader Deepmind -- +2 years for equal safety OpenAI -- +2.5 years SSI -- +2.5 years Deepseek -- +3 years Zai -- +4 years Xai -- +5 years Alibaba -- +5 years Meta -- +6 years

Regarding AGI race dynamics -- I wonder if there's an intuition pump for 'time vs competitor' preference? For example, to me, based on my current knowledge, I think Anthropic reaching RSI before the next best company (Deepmind, maybe?) is worth about two years of time.

A good post on what Claude's Constitution *is* from Zack M Davis. zackmdavis.net/blog/2026/03/p…

11/n Key finding: duplicating a SINGLE middle layer almost never helps. Usually makes things worse. But duplicating a BLOCK of ~7 layers? Big boost. The middle layers aren't doing independent iterative refinement. They're circuits — multi-step recipes that work best as units.

I built a little tool to fiddle around with visualising progress speeding up or slowing down within each category of tasks as well: hunterjayperson.github.io/ai-s-curve-vis… lesswrong.com/posts/ZJm7AxEn…

Wrote up some thoughts on near-term AI self-improvement -- basically thinking through some possible ways current or near future AIs can contribute to the next generation of AIs, and guessing where we are on the S curve of that category.

@HackingButLegal You want to clearly inform people by tricking them about what the General Counsel has said?

@HunterJayPerson No, thanks. That would cancel out my intent to clearly inform individuals as to the risk and illegality of Sec. Hegseth's actions. Who are you, again?

👀 justsecurity.org/133970/legal-a…

@cauliflwr_human @chrislakin lesswrong.com/posts/2HHymvHB…

@cauliflwr_human @chrislakin Expanded context windows with high quality summaries (and regularly updating frontier models in the usual fashion) actually looks almost identical (for practical purposes) as weights-update based continual learning. Wrote about it recently!

this is the first step to solving continual learning btw

I really hope AI decides to treat us better than we treat these animals.

@nssharpe @deanwball @JoinFAI @Allinallnotbad Same vibes! I have to imagine it's the former + employees of competing labs, else it would be an open letter with anyone about to sign themselves.

@deanwball @JoinFAI @Allinallnotbad Are you looking for just relevant/important public figure or just anyone willing to sign on in support? I’m happy to sign if it’s the latter, can’t imagine I qualify as the former 😅

If any people or organizations want to sign on to @JoinFAI’s amicus brief in support of Anthropic, please reach out to me or @Allinallnotbad. You better believe I will be signing.

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

@alexalbert__ Will it be rolled out to Max users soon? Looks like Teams / Enterprise only rn, unless I'm missing something?

This has been a game changer for our internal eng and research teams. Rare to see a product get this much praise from some of the top engineers I know.