Nathan Lambert

Open models, what comes next Don't rely on open models catching the frontier The change from models to systems (tools/harness) Business models supporting open are far from viable (except nvidia) How to change from a few key weights to a winning ecosystem interconnects.ai/p/the-next-pha…

@simonw Not really imo. I think quit

@natolambert Did we ever get a conclusive answer as to if their top researchers quit or were fired?

Qwen is irreplaceable. Has been going from strength to strength in recent times. Things will always be different, I'm hopeful we can find groups of other models to fill the void. RIP

@rasbt just ordered the large :)

The Redbubble poster version just arrived (redbubble.com/i/poster/LLM-A…)! This is the Medium (26.9 x 23.4 in). Font looks good and sharp, but I probably wouldn't go smaller.

I (finally) put together a new LLM Architecture Gallery that collects the architecture figures all in one place! sebastianraschka.com/llm-architectu…

@__mharrison__ I don't know i kind of like pareto's like this, it looks better to me?

@natolambert I don't have an issue with that. Just don't understand why they flipped the x-axis. (Most others don't)

Open weight base models + incredible ML teams in a specific niche can create immense value. Congrats to all my friends on the cursor research team! (they said they did continued pretraining to enable effective scaled RL, super fun)

@__mharrison__ every eval today is on a intelligence vs performance pareto

@natolambert Why!!! did they flip the x-axis?

@llm_wizard joe nemotron?

And they call me…. Joe Nemotron.

I’ve been saying it for a while, cursor’s research team is insanely high on talent density. So many people I respected from my PhD / early career ended up there. Seems like that’s bearing fruit.

@zephyr_z9 Continued pretraining, says it in the tweet.

Very impressive Did they pretrain it or not??

@percyliang @WilliamBarrHeld love this

In Marin, we are trying to get really good at scaling laws. We have trained models up to 1e22 FLOPs and have made a prediction of the loss at 1e23 FLOPs, which @WilliamBarrHeld is running. This prediction is preregistered on GitHub, so we'll see in a few days how accurate our prediction was. What we want is not just a single model but a training recipe that scales reliably.

@awnihannun Congrats homie!

I joined Anthropic as a member of the technical staff. Excited to work on frontier modeling at a place with unwavering values and a generational mission.

Any good quotes on the Nvidia GTC open models panel? Maybe they'll invite me to one some day 🥺

@giffmana @kimmonismus lol agreed, but I’m just happy the concept of open models is going to bigger audiences! With so much happening in ai it can be hard to get airtime for it

@kimmonismus Odd set of people to be talking about open models??

This is going to be such an exciting talk!

@tyler1_fanclub llm focused rl, cant compete with sutton & barto

@natolambert A this strictly LLM focused RL or general RL?

Recording the first lecture of an RLHF Book course I'm making. Things are coming together! Like and subscribe?

Maybe I should've added, they'll live here: @natolambert" target="_blank" rel="nofollow noopener">youtube.com/@natolambert

@rk51787 yes

@natolambert Will the book be available for purchase on Amazon?

@reach_vb @edbayes To be clear I think it’s something Claude happened to find and not that codex is bad, so it’s hard to put words on. Like what makes a good “product”

@natolambert I see, if you were to list top 3 things that feel off, what would they be? cc: @edbayes for vis too (btw you can also set themes in the app)

GPT 5.4 didn't get enough praise for how big of a step it was in OpenAI's agent arc. At the same time, with better context management, speed, rate limits, instruction following, code -- it's revealing that I still turn to the "warmth" of Claude. interconnects.ai/p/gpt-54-is-a-…

@morqon @xeophon @reach_vb my writing is the thinking person's 4o

@natolambert @xeophon @reach_vb how long until someone says opus is the thinking person’s 4o