Orizuru

The definition of insanity is doing the same thing over and over and expecting different results. 🏆🏆🏆🏆🏆 🏆🏆🏆🏆🏆 🏆🏆🏆🏆🏆 🏆🏆🏆🏆🏆

🕺🏆 youtube.com/shorts/b3QrGkQ… via @YouTube

Congratulations on an incredible, undefeated #FreestyleChess tournament in South Africa, @LevAronian 🔥

nanoGPT - the first LLM to train and inference in space 🥹. It begins.

In memory of Daniel Naroditsky ❤️

As a child, Hannah Cairo learned math by taking online lessons from Khan Academy. By the time she was 14, she had taught herself the equivalent of an advanced undergraduate math degree. quantamagazine.org/at-17-hannah-c…

FINAL MOMENTS of WORLD #1 MAGNUS CARLSEN vs. WORLD CHAMPION GUKESH youtu.be/MsFsA6Gv9GY?si… via @YouTube

I knew @LevAronian was lying about this one! 🤣

Happy birthday to Caltech professor Katie Bouman! As a PhD student at MIT she was part of the team that helped create the first-ever image of a black hole.

I just put up a new video on the fundamentals of quantum computing, building up to a step-by-step walk-through of something known as Grover’s algorithm. Here's an excerpt of a quiz used to introduce the topic.

What could Alphafold 4 look like? (Sergey Ovchinnikov, Ep #3) owlposting.com/p/what-could-a…

ProGen3: scaling protein language model data and parameters improves the quality of generations, especially further away from natural sequences. @AadyotB @jeffruffolo @thisismadani

60 years ago this month, the Fast Fourier Transform was created, a powerful tool for image compression & data analysis. Watch a classic MIT breakdown of FFT, perhaps the most-taught algorithm at the Institute: bit.ly/4cNMbPm v/@MITOCW

9 games, 9 wins @MagnusCarlsen 👏

First draft online version of The RLHF Book is DONE. Recently I've been creating the advanced discussion chapters on everything from Constitutional AI to evaluation and character training, but I also sneak in consistent improvements to the RL specific chapter. rlhfbook.com RLHF has a long future ahead of it and this will do a lot to make it more accessible to the next generation. What's next: Getting a physical copy in your hands (may not be exactly 1to1, we'll see) and minor fixes at a slower cadence (thanks to many github contributors, some of you will get a copy from me). Here are all the chapters. 1.Introduction: Overview of RLHF and what this book provides. 2.Seminal (Recent) Works: Key models and papers in the history of RLHF techniques. 3.Definitions: Mathematical definitions for RL, language modeling, and other ML techniques leveraged in this book. 4.RLHF Training Overview: How the training objective for RLHF is designed and basics of understanding it. 5.What are preferences?: Why human preference data is needed to fuel and understand RLHF. 6.Preference Data: How preference data is collected for RLHF. 7.Reward Modeling: Training reward models from preference data that act as an optimization target for RL training (or for use in data filtering). 8.Regularization: Tools to constrain these optimization tools to effective regions of the parameter space. 9.Instruction Tuning: Adapting language models to the question-answer format. 10.Rejection Sampling: A basic technique for using a reward model with instruction tuning to align models. 11.Policy Gradients: The core RL techniques used to optimize reward models (and other signals) throughout RLHF. 12.Direct Alignment Algorithms: Algorithms that optimize the RLHF objective directly from pairwise preference data rather than learning a reward model first. 13.Constitutional AI and AI Feedback: How AI feedback data and specific models designed to simulate human preference ratings work. 14.Reasoning and Reinforcement Finetuning: The role of new RL training methods for inference-time scaling with respect to post-training and RLHF. 15.Synthetic Data: The shift away from human to synthetic data and how distilling from other models is used. 16.Evaluation: The ever-evolving role of evaluation (and prompting) in language models. 17.Over-optimization: Qualitative observations of why RLHF goes wrong and why over-optimization is inevitable with a soft optimization target in reward models. 18.Style and Information: How RLHF is often underestimated in its role in improving the user experience of models due to the crucial role that style plays in information sharing. 19.Product, UX, Character: How RLHF is shifting in its applicability as major AI laboratories use it to subtly match their models to their products.

Our December 2024 free and open databases database.lichess.org are now available for download, bringing the total number of rated games to over 6.2 billion in standard chess and 125 million across variants!

I only have so many tokens left for my patience youtu.be/omYP8IUXQTs?si… via @YouTube

I Made Chess 2.0 youtu.be/y7VtSK23_Jg?si… via @YouTube

What is a chess piece, mathematically? I posted an answer to this question some years back on MathOverflow at mathoverflow.net/a/287927/1946.