Paul Topping

@BulwarkOnline "I've got this thing and it's fucking golden, and I'm just not giving it up for fucking nothing."

Trump on Vance-Rubio 2028: "I do believe that's a dream team, but these are minor details. That does not mean you have my endorsement under any circumstances."

@loss_gobbler @GaryMarcus Not in the last five minutes, anyway.

@GaryMarcus nobody in the "bullish on LLMs" camp has ever disagreed with this

Am old enough to remember when @GeoffreyHinton told me I was stupid for saying that LLMs regurgitate training data. He was wrong. LLM regurgitation is now one of the best-established findings in the field. Excerpt below from a new DeepMind paper; every single one of the papers shows that Hinton was wrong. (Also: still waiting for AI to replace radiologists.)

@GaryMarcus Strangely, this is just like autocomplete, with or without steroids.

Since Hinton has actually replied let me clarify some things - LLMS don’t *always* regurgitate - LLMs don’t literally store full texts - but given the mechanisms that they use they do sometimes regurgitate long passes through their process of resynthesis - the evidence for that is overwhelming - in the literature it is called “memorization”; that’s a metaphor, but it effectively works out to be something much like that.

@ValerioCapraro In the human brain, statistical prediction is but one vote in the election that determines the next word. And there's no reason to think that it's a democracy.

Important Nature Neuroscience paper shows how humans differ from LLMs. Many people currently believe that humans are just next-word predictors, like LLMs. But this new paper by Zou, Poeppel and Ding suggests something more interesting. The human brain does predict words. But it does not predict every word with the same precision. Prediction is constrained by linguistic structure. When a word continues the current phrase, brain activity tracks word surprisal in a way that resembles an LLM. But when a word crosses a major phrase boundary, the match weakens. In other words, the brain does not simply ask: “What is the next word?” It also asks: “What structure am I currently building?” This challenges one of the most common biases in today’s technological world: the belief that human language works like a large language model. The answer is: no. Human language is not just next-token prediction. * Paper in the first reply

@GaryMarcus I suspect once the LLM folks find out all the places where their tech is useful, they will make some real money. However, as @JKD_ff suggests, individual AI co. moats may not protect them. As with many prior s/w technologies, the real winners will be domain-specific.

@PaulTopping that’s actually part of the question - can you make a couple trillion with something unreliable/something short of AGI

One estimate of how much annual revenue AI needs to “make sense”: 1.6 trillion. That’s four times what Google made in its best year. (total revenue so far is perhaps on order of 100 billion.)

@omoakin01_ai @GaryMarcus @geoffreyhinton Yes, but abstractions over word sequences not world models.

@GaryMarcus @geoffreyhinton That’s a more reasonable framing. There’s compression/memorization involved, but generation and abstraction happen too, it’s not purely one or the other.

@BulwarkOnline He has a 3rd option. Just leave and let his replacement decide.

Can Putin, the man who built up and weaponized the Victory Day cult as the sacred foundation of his rule, afford to end a war without a victory? lnk.thebulwark.com/4fbImXd

@SciGuySpace Any news on what happened with Blue Origin's New Glenn stage 2?

In news that should shock no one, NASA has moved the "Starliner-1" mission from June 2026 on its internal calendar to "under review." This is an uncrewed cargo demonstration flight. Since we've heard very little about this, a slip into 2027 seems more likely than not.

@BulwarkOnline Just look at all those goofy smiles around Trump. Makes me sick. Perhaps I'm coming down with a case of sympathetic TDS.

Trump: "I feel the same as I did 50 years ago. It's crazy…It's not because I eat the best foods. Maybe they are the best foods. Who knows what the best foods are? Maybe junk food is good and the other food is no good."

@SarahLongwell25 Agreed but that's just the minimum. They also have to provide a believable explanation of their conversion. Finally, they have to convince us they're not lying.

I have a litmus test for people who have said terrible things in their past: Have they apologized and said the previously-stated terrible views were terrible? Because if they have, then I’m inclined to judge them based on the views they are currently expressing.

@GaryMarcus It'll be ok because they promise not to keep the data for longer than it takes to run a certain correlation program on it.

oh. my. god. 😱

@BulwarkOnline MAGAs never listen to this. Either that or they think they might enjoy living under an "iron fist" as long as it's the leader they voted for. Nah, it's got to be the first thing, right? Someone needs to make an "Iron Fist" movie and force them to watch it.

Trump on Xi Jinping: "He's a great gentleman. I find him to be an amazing man. And when I say that the press always says, 'Oh, that's terrible'…Well, you know, he runs 1.4 billion people with a pretty iron fist."

@SarahLongwell25 I suspect they are imagining something like a "slave state".

I wonder when Stephen Miller will tell Trump that people are able to move between states.

@SarahLongwell25 I think the rest of the world knows that this is Trump and not all of America. Get rid of Trump and we'll get at least a partial reset. Regardless, we'll be on probation for years.

Drove to PA to celebrate Mother’s Day and gas is getting close to 5 bucks across rural PA. We’re heading into the heavy driving/traveling summer months. Americans are going to get angrier and angrier. Remind them every day that Trump chose this.

@AScully789 @fchollet Agreed. Seems like you are objecting to something not being discussed in this thread.

@PaulTopping @fchollet He convinced people with well expressed ideas right? Ideas so well expressed that people who didn't themselves have the ideas, could fully understand them enough to get excited about them and commit to them?

@AScully789 @fchollet Uh ... the usual way, I would imagine. He was very good at convincing people. Not sure what you are trying to say here.

@PaulTopping @fchollet How did he convince the coders to work for him in the first place? How did he convince the VCs to fund him to afford to employ the coders? Again, modern silicon valley is incapable of discovering another Steve Jobs precisely because of this mindset

@AScully789 @fchollet No, but he employed people who code. If he was unable to translate his ideas into code or product, then they weren't fully fledged ideas. I'm sure Steve Jobs had many bad or impossible ideas. Apple just didn't produce or release them. (Ok, maybe a few got out.)

@fchollet Tweets like this are why Silicon valley can never produce another Steve Jobs. Steve Jobs couldn't code. So if he walked up to you, people like you would have said he has no idea what he is talking about (because he can't code it himself) and kicked him out

@wiredmau5 @fchollet In music, the analogous statement would be that an idea that exists only in your head is not a fully formed idea. You actually have to play the piece or at least write it down so others can play it. Inspiration is important but not without realization.

@fchollet maybe. great musicians and artists have made their ideas understood to a large enough audience. how does that fit?

@marcosagusstinn In other words, Europe would work better if it was all one country. Not going to happen any time soon.

Europe does not lack innovation. It lacks scale. European universities produce world-class research, engineers and technology. But too many companies remain trapped inside fragmented national markets instead of scaling immediately across the continent. The numbers are clear: → EU private R&D investment growth has slowed sharply → Europe’s share of global corporate R&D investment has fallen from 21.4% in 2014 to 16.2% in 2024 → Europe still has too few large tech champions because companies face fragmented regulation, smaller capital pools and slower growth financing → Startups must expand country by country instead of scaling through one fully integrated market Europe’s innovation problem is not creativity. It is market size, capital depth and speed of scaling. A continent with world-class talent cannot keep turning great research into small companies. Europe needs one real market for innovation.