Trevor Lohrbeer

Applications for the Pivotal Research Fellowship Q3 2026 cohort are now open. 9 weeks in London with mentors from UK AISI, Google DeepMind, Redwood, and other leading orgs. Stipend, travel, accommodation, compute, and a dedicated desk at LISA are all covered – we do everything except the research itself. Of fellows who want to continue, ~90% secure extension funding (up to 6 months), with active support from Pivotal and their mentor. Apply now!

Currently I have two AirBnB requests that I did on my own waiting for acceptance. So, still no clue where I'm sleeping tonight because the hosts have 24 hours to respond. THIS is what the AirBnB rebooking assistance gets you—shit.

@Airbnb support has been unhelpful with the issue. While they refunded the cost of the AirBnB and (belatedly) gave me a 20% coupon, I'm stuck finding my own new room sitting here in the dark, where all the last-minute hotel bookings are incredibly expensive.

I've been sitting outside the door of an AirBnB for the past 2 hours. The host provided the wrong code for the door and has been unreachable.

@joshua_xu_ Compared to prior Heygen models, this is awesome. Compared to Veo 3 though, it's stuck in the uncanny valley. Having watched these models improve over the past year or so, I think you need to fundamentally rethink your approach. Though maybe they work for your business model.

A quick peek at the latest Avatar IV model — singing with real groove and energy. Just from a pic and one audio file!

Just read the Amazon cancelled Wheel of Time. After seeing the new Veo 3 videos, I suspect in a few years, we'll have fan fiction seasons that keep series that are cancelled going, even when the major studios don't want to invest in them.

For those interested in the work I did during my fellowship at Pivotal Research, below is a recording of the lighting talk I presented at ControlConf. ControlConf was the first AI control conference put on by @redwood_ai and @farairesearch in March '25. youtube.com/watch?v=eH_J-e…

@karpathy The idea was to have an agent attempt to solve tasks, then reflect on successes and failures to derive principles it could save in memory that would then be injected into the system prompt on subsequent runs. Happy to chat with anyone doing similar work on the specific details.

@karpathy Sounds like in-context principle learning, similar to this paper: arxiv.org/abs/2402.05403 I wrote a proposal for a UK AISI eval last year that used this concept to improve performance on the Intercode CTF benchmark. It got accepted, but I had to hand off the implementation.

We're missing (at least one) major paradigm for LLM learning. Not sure what to call it, possibly it has a name - system prompt learning? Pretraining is for knowledge. Finetuning (SL/RL) is for habitual behavior. Both of these involve a change in parameters but a lot of human learning feels more like a change in system prompt. You encounter a problem, figure something out, then "remember" something in fairly explicit terms for the next time. E.g. "It seems when I encounter this and that kind of a problem, I should try this and that kind of an approach/solution". It feels more like taking notes for yourself, i.e. something like the "Memory" feature but not to store per-user random facts, but general/global problem solving knowledge and strategies. LLMs are quite literally like the guy in Memento, except we haven't given them their scratchpad yet. Note that this paradigm is also significantly more powerful and data efficient because a knowledge-guided "review" stage is a significantly higher dimensional feedback channel than a reward scaler. I was prompted to jot down this shower of thoughts after reading through Claude's system prompt, which currently seems to be around 17,000 words, specifying not just basic behavior style/preferences (e.g. refuse various requests related to song lyrics) but also a large amount of general problem solving strategies, e.g.: "If Claude is asked to count words, letters, and characters, it thinks step by step before answering the person. It explicitly counts the words, letters, or characters by assigning a number to each. It only answers the person once it has performed this explicit counting step." This is to help Claude solve 'r' in strawberry etc. Imo this is not the kind of problem solving knowledge that should be baked into weights via Reinforcement Learning, or least not immediately/exclusively. And it certainly shouldn't come from human engineers writing system prompts by hand. It should come from System Prompt learning, which resembles RL in the setup, with the exception of the learning algorithm (edits vs gradient descent). A large section of the LLM system prompt could be written via system prompt learning, it would look a bit like the LLM writing a book for itself on how to solve problems. If this works it would be a new/powerful learning paradigm. With a lot of details left to figure out (how do the edits work? can/should you learn the edit system? how do you gradually move knowledge from the explicit system text to habitual weights, as humans seem to do? etc.).

Applications to our Q3 Research Fellowship are now open! → June 30 – Aug 29 in London at the London Initiative for Safe AI → Work on AI safety with the guidance of your experienced mentor and research manager → £5,000 stipend + meals, travel & housing support (link in bio)

Of course, this is the problem with aphorisms. They sound like great wisdom, but are often just flashy words that sound good—and often contradict each other. While they sometimes contain kernels of truth, it's best to treat these as perspectives, not principles.

The cognitive dissonance in this week's @farnamstreet newsletter is strong. If attention is the most valuable thing you spend, then those who receive the most attention gain the most value. If you believe the second aphorism, then flashy gets the most value, not boring.

We’re looking for talented individuals and organisations to help us build evaluations. We’ll reward bounties for new evaluations and agent scaffolding tools that assess the risks of autonomous AI systems. Find out more and apply by 30 November:

I just published A Taxonomy of AI Safety Evals containing my current thoughts on the different types of AI safety evals from the perspective of a software engineer. link.medium.com/FwygIGGrINb #AISafety #AIEvaluations

- Strategies for effectively scheduling these tasks, including leveraging your natural biorhythms and low-interruption zones.

What you’ll learn: - How to prioritize your most impactful tasks (Anchors) to make progress on your goals. - The role of Bumpers in maintaining flexibility and managing your day when things don’t go awry. - Why Core Commitments are essential to sustaining your energy and focus.

In the latest episode of AI Meets Productivity, ChatGPT and I explore the concepts of Anchors, Bumpers, and Core Commitments to help you take control of your day. aimeetsproductivity.com/e/29-anchors-b…

🎙️ New episode of AI Meets Productivity! This week ChatGPT and I dive into why we need AIs with customizable morality to balance societal norms with personalized morals and to support future moral growth. #AI #AIEthics #Podcast #AIandSociety aimeetsproductivity.com/e/28-ai-morali…

🎙️ What would it mean to be done for the day? In this episode, we explore how to define the end of your workday with strategies like time blocking, managing energy, and focusing on key accomplishments. aimeetsproductivity.com/e/27-how-to-kn… #ChatGPT #podcast

Recorded a video podcast episode of AI Meets Productivity today with a virtual avatar of Ryan Hoover, CEO of Product using the new Interactive Avatar tech from @HeyGen_Official Aside from some lag occasionally, the realism is incredible. Check it out at: youtube.com/watch?v=K4IzQ8…