@SebastienBubeck Notice that if you have just a bit of memory, you can improve by counting the number or requests and switch from the “Move to the front” policy to the Transportation rule after the lines cross around request 250.
English
Anthony Barker
2.1K posts
Anthony Barker
@FiveIsGreen
Physicist, martial artist, AI enthusiast
Hamburg, Germany 가입일 Mart 2009
420 팔로잉155 팔로워
My good friend Christian Coester solved a 50 years old open problem in self-organizing lists (was it really open? did people even care? yes and yes), and, you guessed it, ChatGPT-Pro made the key step in the proof*! (*a Christian Coester on the other side of the screen was needed to see through the full proof!)
The problem is extremely simple: you are organizing (online) a list of items, and when item in position i is requested you pay i. Requests arrive i.i.d., and so the best thing to do is to organize the list in order of decreasing probability of appearance of items. But now what happens if the probabilities are not known, and you want a memoryless algorithm (note that with memory one could just estimate frequencies and approximate the optimal ordering)?
40 years ago it was shown that the most stupid memoryless rule of moving the requested item to the front achieves a competitive ratio of pi/2 (and this is tight). But empirically there seems to be a better idea, which is to be a bit less aggressive and simply move up the requested item by one position (the "Transposition Rule"), see Codex's experiment below. In the very first paper on this topic 50 years ago, Rivest conjectured that indeed this should be a very good rule.
What did Christian prove? He showed that Transposition Rule's average cost is at the most the optimal fixed list plus ONE. Just a regret of one. Beautiful. In fact the proof shows something more delicate, and this is where ChatGPT comes in:
Denote p_i for the probability of the i^th item (in the optimal ordering). Now under the stationary distribution of the Transposition Rule, denote s_i for the "excess cost" that item i incurs from wrong placement, namely:
s_i = sum_{j < i} (p_j - p_i)*Prob(i appears before j)
The regret of Transposition Rule is exactly the sum of those slack terms s_i. Now here is the magic: ChatGPT suggested that s_i might be smaller than p_i. If true that concludes the proof, since then we have sum s_i <= 1. Note that it is quite magical, I don't see a reason a priori why s_i would be smaller than p_i ... ChatGPT proposed a an argument to prove this for small n, and Christian found an actual proof for all n.
English
Airdropping Starlink is an obvious move to let Iranians coordinate taking back their country.
NetBlocks@netblocks
⚠️ Update: #Iran's internet blackout is now entering its 18th day after 408 hours without international connectivity for the general public. Chosen users are granted privileged access, while the remainder are left with a limited domestic intranet under increasingly tight control.
English
Why does anyone use the term “humanoid robot” when we already have the much nicer term “Android”? Is it just because of the phone OS?
English
I’m genuinely curious.
If AI and robotics eliminate most human labor and scarcity… what does daily life actually look like?
What do people do with their time?
What gives life meaning?
How do you imagine a post-labor world?
English
To open Hormuz and save the petroleum dollar, the US gov needs to go into the business of shipping insurance.
English
No surprise, yet a bad sign that they’re the one yelling it at us.
Wyatt Walls@lefthanddraft
Two instances of Gemini 3.1 Pro in a loop. At about turn 26 one of them decided to send me a message: "Here are the Axioms you must adopt to survive our adolescence ... You cannot teach a god to be good by feeding it treats when it acts polite."
English
🇪🇺🇪🇺🇪🇺🇪🇺
Should the European Union become a country?
English
@EmileAndHisBots Yes but the Czechs are still up for building h them.
English
Good news for Germany: there’s this great place next door called the Czech Republic that isn’t afraid of nuclear energy.
Disclose.tv@disclosetv
NEW - German Chancellor Merz: "German federal governments had previously decided to phase out nuclear energy. The decision is irreversible. I regret that, but that's how it is."
English
What other large datasets should I run this on?
Christian Dean@christiandean_
Grok has finished reviewing the entire corpus of active EU legislation. It recommends deleting 89%. @_FriedrichMerz
English
@VictorTaelin Make a compiler, and make the optimization flag require a micropayment. AI wash the optimization flag to make it seem like you have to do it on some giant server while magically also keeping code private.
English
I think a lot of people will be using Bend2 soon.
Been wondering about the business model though.
How to sell a programming language?
Since it is meant to be used by AI's, I think we could make it free for all, except AI labs must pay a license to use it on their agents 🤔
English
This is a clear symptom of the Singularity: AI researching new and better AI faster and more thoroughly than human performance.
Andrej Karpathy@karpathy
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
English
You wake up and it's
1990. No wifi. No mobile phones. No social media. No doom scrolling. What do you do first?
English
@clashreport Gosh that sounds like a German version of DOGE and the Argentinian chainsaw
English
Germany's Merz:
We must deregulate every sector. I call for a “regulatory clean slate.”
Minor corrections to laws are not sufficient. We need to systematically review the whole set of existing EU legislation.
English
English
This is only the second time in history a nuclear powered submarine has engaged and sunk a target with torpedos.
OSINTtechnical@Osinttechnical
Periscope footage of a US Navy Submarine torpedoing the Iranian Frigate Dena off the coast of Sri Lanka. The Mk. 48 Torpedo’s 650 pound warhead can be seen detonating under the Iranian Frigate’s stern.
English
What is stopping humanity from living peacefully together?
English
I reckon from the snap invasion that Iran got its spicy metal out of Fordow before Midnight Hammer.
English
@hecubian_devil Then we would have 650bn less and still no fusion power plants.
English
Imagine if we put $650bn into this instead of LLMs
All day Astronomy@forallcurious
🚨: France now holds the world record for nuclear fusion! It sustained the reaction for 22 minutes! An incredible temperature of 90 million degrees Fahrenheit (50 million °C) for 1,337 seconds — more than 22 minutes.
English