rapp_dore

To add to Bojan's point, the engagement bait algorithm is ruining the X experience. I value X as a platform for discovering diverse perspectives and timely information, which makes the current situation particularly frustrating. If this platform is to become the replacement for traditional news media, as many suggest it already has, then this problem must be solved. Muting is somehow not enough. I have muted dozens of accounts, but still others saying the same things almost word for word keep popping up. Why? I don't follow them, so why do they come up on my feed? Why can't the algorithm recognize patterns? If I've muted multiple accounts posting identical talking points, it should stop showing me similar content from accounts I don't follow. I understand they have a right to say whatever they are saying, but I have a right not to hear from them too, no? Every time a hot button issue, like the recent H1B issue, comes up, the timeline deteriorates for days. Engagement farmers don't let it go to squeeze every last drop of it. The majority of the posts are not even informative. I'd welcome diverse, substantive takes that add new information or perspectives, not the same recycled outrage. Please optimize for quality, information, and thoughtful content over just engagement via bait. And yes, rage can be quality, informative, and thoughtful too. I'm not against passionate discourse, just against manufactured outrage designed solely to drive clicks. @nikitabier

@avidseries Why are you surprised? ~ "the power of instruction is seldom of much efficacy, except in those happy dispositions where it is almost superfluous." - Gibbons. One who would be persuaded by the data, have already looked and found it. The one who won't, well...

@browsingenabled ~ they also serve who only stand and wait ~ :)

I love it when cause and effect is misunderstood and interchanged. The millions of americans with guns exist because there is property enumerated in excels to protect. The enumeration and pricing of property came first, not the other way round. The meritocracy of excels is what creates property that then creates the guns that protect it!

"Your artificial meritocracy of Excel spreadsheets is propped up by millions of Americans with guns protecting you from the natural meritocracy of extreme violence" might be one of the best tweets ever

@joeroganhq I will believe it when it happens. Have been the victim of so many disappointments over the years!

On a scale from 1-10, what are the chances of Trump confirming existence of extraterrestrial life?

Excited to release new repo: nanochat! (it's among the most unhinged I've written). Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI. It weighs ~8,000 lines of imo quite clean code to: - Train the tokenizer using a new Rust implementation - Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics - Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use. - SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval) - RL the model optionally on GSM8K with "GRPO" - Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI. - Write a single markdown report card, summarizing and gamifying the whole thing. Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc. My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved. Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

Not disputing the bubble feel at all... But i think we are both hoping that the trillion dollar build out will not be wasted. There is a high premium on compute from the dual needs of training and inference. Now, admittedly only inference pays the bills, but training holds out the promise of the unicorn at the end of the rainbow. Without doubt some investments will be a loss in the short term 1 - 5 years. And that will lead to a lot of volatility too. But for the long term macro investor the only question is whether this whole bet pays off in the aggregate. Open AI may fail or go bankrupt. But would that really change the big picture? There are now enough other players in the field. I am very optimistic it will work out in the aggregate. Even if we freeze AI capabilities to today's standards, there is significant productivity that can be squeezed out of it.

If AGI is real and near, our finance playbook is outdated. Money becomes less an end state and more a coordination tool - fuel to accelerate "innovation flow." In that regime, the unit of value isn’t cash flow; it’s capability: the rate at which we unlock previously impossible things. You don’t price breakthroughs well ex-ante; you stage them into existence. So you bend old rules to move faster; creative vendor financing, milestone warrants, pre-pays - because the objective is compounding capability, not quarterly optics. Capital front-loads a decade (decades?) of future cash flows on faith that innovation will catch up - and then overflow. Caveats (crucial): physics and power grids still bind; scaling returns can stall; governance and IP matter for adoption; someone bears inventory and obsolescence risk. “Innovation flow” doesn’t erase economics - it temporarily reframes it. The right discipline is staged commitments, outcome-linked pricing, and tripwires that revert to cash-flow rigor if capability plateaus. In short: if AGI pays off, we’ll redefine finance around the speed of learning and deployment. If it doesn’t, we’ll relearn old lessons - write-downs, surplus hardware, tighter credit. I do think we need two key unlocks. AGI and Fusion. Or maybe AGI -> Fusion. As always, timing matters. If we can unlock both before the bill comes due, we are golden. If not...

@oguzerkan This is good. We are at least spending money on capital investment that has some potential benefit unlock, rather than garbage plastic knickknacks from China! More GPUs, less Amazon prime slop!

This is alarming… Almost 40% of the US real GDP growth last quarter was driven by tech capex. Bulk of the capex was AI-related investments. Aggregate capex can’t keep climbing like this in the absence of real profits derived from AI investments. If this doesn’t happen soon enough, we’ll likely see substantial slow-down in GDP growth, which can trigger a deep correction in the market given that the valuations are already at elevated levels. I think it’s time to be more fearful than greedy.

Introducing Claude Sonnet 4.5—the best coding model in the world. It's the strongest model for building complex agents. It's the best model at using computers. And it shows substantial gains on tests of reasoning and math.

@WomanDefiner The mote in God's eye! You can never beat crazy Eddie.

Someday we're going to find out humans have been around 2 million years and keep having society wiped by pole shifts.

@KobeissiLetter Where else can the money go? Nothing else has a candle's chance in a typhoon of returning a meaningful return.

Technology firms are borrowing money at a rapid pace: Tech companies have raised ~$157 billion in the US public bond markets YTD, the highest YTD total since 2020. This is up +$65 billion, or +70%, from 2024 and +$90 billion, or +134%, more than in 2023. Oracle, $ORCL, led the rush with almost $26 billion of publicly traded debt sold. Meanwhile, investment-grade corporate bond spreads have fallen to 74 basis points, near their lowest in 27 years, as investment appetite remains strong. AI is revolutionizing capital markets.

I have seen the same thing. My hypothesis is that thinking does its own thinking, but PRO does research on internet slop and then tries to summarize slop. Just my hypothesis not knowing the internal workings.

I think GPT-5 "heavy thinking" is smarter than PRO now.

@cremieuxrecueil Restricting this information and separating it by subject will be really insightful. How many math PhDs are there at the low end of the curve vs. say art history?

It's much worse than that. There are sub-90 IQ people earning PhDs!

@deredleritt3r Oh. Looks like they are actually trying to do it. Nice! x.com/OpenAI/status/…

@deredleritt3r Evals must measure real novelty and utility, not just scores. I am really curious how that is done. Are we trying to RL sythentic puzzles, or really creating reproducible evidence of discovery and economic value? I guess we will know soon. Perhaps!

New interview with Mark Chen and Jakub Pachocki: - "The big thing we're targeting is producing an automated researcher. Automating discovery of new ideas, automating our own work, automating ML research. And we're also thinking about automating progress in other sciences." - "As we get to a level of near-mastery of high-school [math and coding] competitions, in the order of 1 to 5 hours of reasoning, we are focused on extending that horizon, both in terms of models' capability to plan over very long horizons and ACTUALLY HAVING ABILITY TO RETAIN MEMORY." - "I think we will be inching towards MORE AND MORE HUMAN-LIKE LEARNING, which RL is still not quite." - How OpenAI looks at evals now: "The big things that we look at are actual marks of the model being able to discover new things... The next set of evals will involve actual discovery and actual movement on things that are economically relevant."

Another aspect of the structural mismtach in the housing market might be a problem of demand mix. men and women are coupling less. If they couple less, they would rather rent than buy. Additionally, typically a couple will have more buying power. So it is not only the fact that prices has risen but demand from couples looking to buy and home and starting a family has also reduced. But then, singles are rising as a share of households which drives up per capital housing demand... So who knows?

"Buyers aren’t inclined to purchase a home because mortgage rates and home prices are too high (they’re up 1.7% year over year at $440,004, according to Redfin). And homeowners don’t want to sell their homes to trade for a higher mortgage rate and out of fear they won’t get what they think their home is worth," per FORTUNE

@bayeslord Calling it a bubble when we have not even reached Kardeshev - 1! The difference between a "bubble" perspective and "not a bubble" perspective is just the time horizon!

anyone who’s naively calling bubble right now has not internalized the physics of our world. most of them are simply ignorant about ai progress and its implications. of course there will be fluctuations. of course we could stumble. but history marches on

@zerohedge We calling 1 - 2% drops dips now? :)

dip buyers assembled

@unusual_whales Weekly claims are noisy, so we should not read too much in these numbers. interesting, but not a narrative changer on its own.

US jobless claims -14,000 to 218,000; survey 235,000

@GaryMarcus "you can see the computer age everywhere but in the productivity statistics" - robert solow 1990s Big changes in productivity take time. We didn't write off computers in the 90s. We shouldn't write off AI now.

I have a great idea. If America’s biggest corporations are still struggling to explain the upsides of AI, how about if we don’t throw another trillion dollars at it just yet?