Minglai Yang

While Alec is one of the best ML researchers of all time, LLM started way before. Here's one from 2013 for non-neural architecture and one from 2016, which is afaik the first neural LLM if we define LLM as LM w/ >1B params.

What are Chinese quant companies smoking to get this kind of performance??? Mogging Sonnet 4.5 with a 40B

I’m launching Tutorial II for Physics of Language Models. Many people focus on large-scale results. This tutorial is about why those results are often artifacts of noise — and how to eliminate that noise at the design level. The first video (Part 4.1a, 1 hour) is the most important one. It focuses on methodology, not benchmarks: – how real-life pretraining can be “cheated”, – why academic-scale experiments are noisy, – and most importantly, how to design a versatile, skill-pure synthetic pretraining playground. I explain why our five synthetic tasks are designed the way they are, and how GPT2-small-scale (100M) models can reveal architectural truths that 8B models trained on 1T tokens often fail to expose reliably. This methodology is the backbone of the entire Physics series. ▶️ First video: Part 4.1a — methodology & playground design 🔜 Second video: Part 4.1b — architectural principles from the playground 🔜 Third video: Part 4.2 — when the playground reshapes real-life pretraining (You can find it via my profile.)

The GDM mechanistic interpretability team has pivoted to a new approach: pragmatic interpretability Our post details how we now do research, why now is the time to pivot, why we expect this way to have more impact and why we think other interp researchers should follow suit