Dan Roy

“Resource Rationality” is a nice idea, but fundamentally flawed. Assume, for contradiction, that there is a rational way to allocate cognitive resources. Then some cognitive process must decide how much effort to spend on a task. But that process itself spends effort. Thus, a rational allocator must also decide how much effort to spend on allocation itself. That requires a further allocation decision, producing a regress. If the regress continues, no allocation is ever completed. If it stops, the stopping point is arbitrary rather than rational. Therefore, no fully rational way to allocate cognitive resources exists. Any non-contradictory theory of bounded rationality must therefore contain an irrational stopping point: some allocation of cognitive resources that is not itself rationally allocated. Formally, this is equivalent to a decision maker who cannot fully know the decision problem in advance, because the beliefs that define the problem are themselves produced by prior, unchosen cognitive allocations.

This is what AI looked like when I was doing my PhD in 2008. Tree search. Alpha-beta pruning. Branches and heuristics. I worked on the machine learning side, which was a separate field then. And neural networks, even inside ML, were treated as pseudoscience. No theorems. No bounds. I sat in seminars and smirked when someone presented results on them. I told friends not to waste a PhD on that stuff. The people I smirked at run the labs I cannot get into. One of them told me over coffee, in 2009, that he was switching to neural networks. I told him he was being unserious. I genuinely thought I was helping. He runs one of those labs. I got lucky. I went to NYU after, and the smirk left my face in six months. I am grateful I got in when I did. Two or three years earlier and I would not be writing this post. I think about that coffee a lot. What are you smirking at right now?

The penalty is a 1-year ban from arXiv followed by the requirement that subsequent arXiv submissions must first be accepted at a reputable peer-reviewed venue. 4/

I disagree. Google has ~$400 billion in annual revenue. Anthropic is on track to end the year with ~$100 bil ARR with a >10x growth rate. Even if the growth rate slows down significantly, Anthropic will surpass Google in revenue soon (maybe even in 2027). Anthropic's gross margin is a question, but Semianalysis thinks their gross margin is 70%

Thread with a few notes on this. It’s a disappointing finding, of course. The best we can do is fix it up and learn lessons for future work.

Spending billions to train the "best" base model? You might be optimizing the wrong thing! 🎯 We show that controlling sharpness during mid-training leads to over 35% less forgetting after fine-tuning / quantization... even when the base model itself gets worse. 🧵 Takeaways for pretraining: - Use SAM (Sharpness-Aware-Minimization) in the final steps (~10%) - Try much higher learning rates (yes, even ~10× larger) 1/9