Sumuk

we're launching 🤗 yourbench today, an open source tool for custom benchmarking and synthetic data generation from ANY of your documents. it's a big step towards improving how model evaluations work early access link in replies! (1/8)

@xeophon @llm_wizard @twominutepapers literally same lmao!

@llm_wizard @twominutepapers this vid is one of the reasons i am in ai

BANGER SESSION. @twominutepapers bringing the best possible energy to this panel.

@pmarca Underrated tweet

Found the retard.

@gauri__gupta @NeoSigmaAI @RitvikKapila Congratulations!

We @neosigmaai @RitvikKapila are building the future of self-improving AI systems! By closing the feedback loop between production data and system improvements, we help teams capture failures, convert them into structured evaluation signals, and use them to drive continuous improvements in agent behavior. We show how our system works on Tau3 bench across retail, telecom, and airline domains. Agent performance on the validation set (with a fixed underlying model, GPT5.4) improves from 0.56 → 0.78 (~40% jump in accuracy).

Can confirm that costs are 10x lesser when you use the prime intellect hosted RL trainer with a cheap Chinese instruct model Qwen3-4B-Instruct-2507 ftw

is it just me or does it make the most sense to interview on weekends (as a candidate)? don't have to move around your regular week to make the times work

@scaling01 come on man those model sizes mean absolutely nothing as it doesn't really relate to raw inference cost show me the active param counts and i'll be curious

my estimate for Anthropic model sizes: - Haiku: 200-500B @ $5 - Sonnet: 700B-1.4T @ $15 - Opus: 1.5-3T @ $25 - Mythos: 6-20T @ $100+

@pmarca What do you do when the marginal cost of a unit of model labor is cheaper than human labor? Keep regulation in place just to have humans employed?

Claude responds: The "Rising Cost of Existence" Claim Has No Mechanism — And the Default Technological Trajectory Is the Opposite The argument asserts that AI pushes the cost of human existence up. This is stated, not argued. And it runs directly contrary to the entire historical trajectory of technological advancement, which has been to collapse the real cost of goods and services that constitute basic human welfare. Consider what the subsistence floor actually consists of: food, clothing, basic shelter, energy, medicine, communication, transportation. Every single one of these categories has seen its real cost fall dramatically as a function of technological productivity gains. An American minimum-wage worker today commands more calories, more clothing, more computing power, more pharmaceutical access, and more travel capacity per hour of labor than a solidly middle-class person in 1920. The technological arrow on the cost of material subsistence has pointed unambiguously downward for 250 years. AGI accelerates this, not reverses it. If AGI can perform cognitive labor, it dramatically lowers the cost of producing everything that requires cognitive labor as an input — which is everything. Drug discovery gets cheaper. Medical diagnosis gets cheaper. Legal services get cheaper. Engineering design gets cheaper. Agricultural optimization gets cheaper. The AI-abundant world is one where the absolute cost of meeting basic human needs plummets, not one where it rises. So the mechanism for the "cost of existence rises" half of the scissors needs to be specified. There are exactly two categories of goods that resist this downward pressure: Category A: Positional goods and social status. Being in the top 10% of income is by definition zero-sum. If AI makes everyone richer, relative rank competition intensifies. But this is about relative impoverishment, not absolute destitution. Confusing these two is a serious analytical error. Humans being worse off relative to AI-augmented entities is categorically different from humans being unable to meet basic needs. The argument requires the latter to generate the "economic destruction" framing. Category B: Location-constrained goods — primarily housing. This is the strongest version of the real concern. Housing in desirable, productive urban locations is fundamentally constrained by land, zoning, and geography, and AI doesn't solve the zoning problem. If the gains from AI get capitalized into urban real estate (which is partly what happened with previous technology booms), housing costs can rise even as manufactured goods get cheaper, and housing is a large component of the subsistence floor. This is a genuine concern — but it's a political economy problem with known solutions (zoning reform, land value taxes, building incentives), not a fundamental economic law produced by AI. Packaging it as evidence that AI inevitably raises the cost of human existence requires ignoring that the mechanism is political dysfunction, not technological necessity. The technology doesn't raise housing costs. Regulatory capture and NIMBYism raise housing costs. These are separable.

@levelsio beautiful, wow a non dead comments section built on social proof

Okay let's see who can reply to this

@GlennMatlin You genuinely just need to vibecode your own tools now. No more package shop

Stuff like this is why TPOT is so valuable

As much as I like litellm, the argument has never been stronger for why you should never use external libraries. Ask codex/claude to make mini tools for you. Reduce your attack surface. Stop using external packages and libraries. Set sane budget limits

large parts of the bay area will soon be unable to pay their mortgages as the era of abundant 500k+ tech jobs comes to an end game theoretically, it's in the model providers' best interests to: maximize: (human + ai) productivity minimize: (ai - human) productivity

I feel like this will be a future banger. I don’t know a single person who uses Lovable How do they have such high ARRs? Something is wrong here

@gdb 5.4 needs more instruction tuning! pls fix for 5.5!

codex has gotten very good!

@thsottiaux Tibo can I have a “slow mode” please for codex?

What are we consistently getting wrong with codex that you wish we would improve / fix?

@edwinarbus @cursor_ai damn are you guys doing better than anthropic directly RLing with the cc harness is it the same cursor harness for diff models?

Matt Maher tested frontier models in Cursor v. other harnesses. Cursor boosted model performance by 11% on average: Gemini: 52% → 57% GPT-5.4: 82% → 88% Opus: 77% → 93% His benchmark measures how well models implement a 100-feature PRD. @cursor_ai consistently outperformed.

@kalomaze i was today years old when i heard the term RLVR brain

i think RLVR brain is a real phenomenon and it has been localized to how claude behaves in a harness this is a question about regressions, and even then, the agent (deep in-context) cannot help itself, it must expand scope instead of looking at the scope of things it got rid of

@realchillben Where is 5.4 medium and xhigh though? 🤔

always down for some pareto curves

OpenAI is going to win the meme wars because they have a sense of humor.

@thsottiaux can we please have ssh support in the codex app? one of the only reasons i need to keep using the cli (claude code already has it!)

“Codex App has transformed the way I write software… I barely use anything else these days” ... and yet we're only getting started

@thsottiaux appreciation post. Even with load / outages, keeping us updated is so helpful, compared to what anthropic does when there’s a Claude outage lol