Andrew Zhang
186 posts





















USA. A Mexican restaurant. We had not yet ordered anything, and the food was already arriving. Chips. Salsa. Unrequested. Free. I stopped the waiter. "We have not earned these." "They just come with the table, man." They come with the TABLE. In my land, hospitality is a debt. Every gift creates an obligation, weighed carefully, returned in the proper season with interest of feeling. Here, the gift arrives before you have even proven you can pay for dinner. This is not an appetizer. This is a declaration: we trust you. Eat. I ate with the gravity the moment deserved. And then — I must report this calmly — the basket emptied, and a new one appeared. "Did we…?" "Refill," the waiter said. "It's bottomless." Bottomless. They have wells of salsa. The supply lines of this nation are beyond anything my ancestors imagined. My friend warned me. "Don't fill up on chips, dude." Too late. I had accepted three baskets. Honor demanded each one be finished — an unfinished gift is an insult. By the time my actual food arrived, I was a ruined man. I was not hungry. I was not comfortable. I had been defeated by a courtesy. Generosity that arrives before the request cannot be repaid. It can only be survived. I know the rule now. I have made my peace with the basket. One basket. Two at the most. Who am I deceiving. There is no number of baskets I would refuse. The trust of a nation is in that salsa, and I intend to honor all of it.


California universities dropped the SAT to help low-income and minority students. The policy is doing the opposite, writes Svetlana Jitomirskaya, a professor of mathematics at UC Berkeley. thefp.com/p/bring-back-t…



We have to take the LLMs to school. When you open any textbook, you'll see three major types of information: 1. Background information / exposition. The meat of the textbook that explains concepts. As you attend over it, your brain is training on that data. This is equivalent to pretraining, where the model is reading the internet and accumulating background knowledge. 2. Worked problems with solutions. These are concrete examples of how an expert solves problems. They are demonstrations to be imitated. This is equivalent to supervised finetuning, where the model is finetuning on "ideal responses" for an Assistant, written by humans. 3. Practice problems. These are prompts to the student, usually without the solution, but always with the final answer. There are usually many, many of these at the end of each chapter. They are prompting the student to learn by trial & error - they have to try a bunch of stuff to get to the right answer. This is equivalent to reinforcement learning. We've subjected LLMs to a ton of 1 and 2, but 3 is a nascent, emerging frontier. When we're creating datasets for LLMs, it's no different from writing textbooks for them, with these 3 types of data. They have to read, and they have to practice.





New blog post: my preferred method for proving Killing-Hopf: lucaman99.github.io/mathblog/killi…














