Louis Lambert

@ALeaument C’est fou d’être aussi bête. Même en supposant un calcul politique, ce serait un mauvais calcul. Plus aucune personne brillante ne veut faire de politique en France. Il ne reste que ceux qui se croient intelligents sans l’être. C’est triste.

Bernard Arnault ne veut pas payer 5% d’impôts en plus. Il veut s’enfuir aux USA. Les milliardaires sont prêts à trahir la patrie qui a fait leur richesse. Ils ne sont RIEN sans les travailleurs. Assez de laxisme contre les milliardaires vautours : contrôle fiscal, sanction.

Why do we treat train and test times so differently? Why is one “training” and the other “in-context learning”? Just take a few gradients during test-time — a simple way to increase test time compute — and get a SoTA in ARC public validation set 61%=avg. human score! @arcprize

What overregulation feels like. AI progress is now skipping Europe

OpenAI Strawberry (o1) is out! We are finally seeing the paradigm of inference-time scaling popularized and deployed in production. As Sutton said in the Bitter Lesson, there're only 2 techniques that scale indefinitely with compute: learning & search. It's time to shift focus to the latter. 1. You don't need a huge model to perform reasoning. Lots of parameters are dedicated to memorizing facts, in order to perform well in benchmarks like trivia QA. It is possible to factor out reasoning from knowledge, i.e. a small "reasoning core" that knows how to call tools like browser and code verifier. Pre-training compute may be decreased. 2. A huge amount of compute is shifted to serving inference instead of pre/post-training. LLMs are text-based simulators. By rolling out many possible strategies and scenarios in the simulator, the model will eventually converge to good solutions. The process is a well-studied problem like AlphaGo's monte carlo tree search (MCTS). 3. OpenAI must have figured out the inference scaling law a long time ago, which academia is just recently discovering. Two papers came out on Arxiv a week apart last month: - Large Language Monkeys: Scaling Inference Compute with Repeated Sampling. Brown et al. finds that DeepSeek-Coder increases from 15.9% with one sample to 56% with 250 samples on SWE-Bench, beating Sonnet-3.5. - Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. Snell et al. finds that PaLM 2-S beats a 14x larger model on MATH with test-time search. 4. Productionizing o1 is much harder than nailing the academic benchmarks. For reasoning problems in the wild, how to decide when to stop searching? What's the reward function? Success criterion? When to call tools like code interpreter in the loop? How to factor in the compute cost of those CPU processes? Their research post didn't share much. 5. Strawberry easily becomes a data flywheel. If the answer is correct, the entire search trace becomes a mini dataset of training examples, which contain both positive and negative rewards. This in turn improves the reasoning core for future versions of GPT, similar to how AlphaGo’s value network — used to evaluate quality of each board position — improves as MCTS generates more and more refined training data.

@waitbutwhy Coding

What, if anything, do you regularly use ChatGPT (or another LLM) for that has provided a dramatic improvement over your previous workflow?

@wolfejosh Do you have any early indicator of an imminent credit crunch?

🚨Fed controls short end of yield curve Market controls long end Fed partially hopes market/investors helps tighten credit––and it is + will I have imminent feel a CREDIT CRUNCH will hit markets and hard––with NO idea if spark ignites via CMBS,consumer debt, banking, energy...

@micsolana You assume that everything has a beginning and an end. If you don’t assume that, it’s much more understandable.

about the origin of the universe —  either some things come from nothing, which is impossible given nothing is nothing or some things are eternal, which is impossible because all things come from something right?

Exciting paper "The Mind of a Mouse" by Viren Jain (@stardazed0) of @GoogleAI et al. (25 institutions represented!) "Roughly 1 million TB of data will need to be acquired & analyzed to provide a complete mouse brain connectome that includes ... all synaptic connections."

@banteg How can we delegate our votes ?

Quite disappointed this YIP didn’t reach quorum. If you didn’t vote, please sell your votes to someone less apathetic. The governance only works if everyone is involved.

You get one life. Do it all. Don’t leave anything in reserve.

If this isn’t the sign of a market top idk what is

Time’s always moving on. Nothing can stop it. The question is whether we use our time well or not. We can't do anything about the past, but what happens in the future depends on what we do now. We can create a happier future by remembering that in being human we are all the same.

You’re offended when you fear that it might be true.

Artificial intelligence meets the arrow of time: in predictive coding, a neural network makes sense of the present by constantly making predictions about the future. Brains might be doing the same thing. quantamagazine.org/to-make-sense-…

Here's a thought-provoking piece about humanity’s future with superhuman #AI, full of @NewYorker cartoons: newyorker.com/magazine/2018/…

"It’s likely to be the most important thing that has happened on this planet since the rise of Homo sapiens." - Nick Bostrom, Superintelligence

Delighted to stumble by this article bringing more attention to Stanislaw Lem and his work. Very high ratio of intellectual depth vs obscurity. m.nautil.us/issue/28/2050/…

The most important patterns: 1. Prisoner's Dilemma 2. Race to the Bottom 3. Free Rider Problem / Tragedy of the Commons / Collective Action 4. Zero Sum vs. Non-Zero Sum 5. Externalities / Principal Agent 6. Diminishing Returns 7. Evolutionarily Stable Strategy / Nash Equilibrium twitter.com/JamieSmartCom/…