Pachu

@dexhorthy Isn't RLM basically this 😂

harness mesh

“Our new agent sdk is an integrated subharness of the outer orchestration apparatus”

@gabriel1 This is so true, but it's so hard to fight against the external pressure to be slopping 20k loc per day

there is still no substitute for perfectly understanding every single line of code in your codebase i fall into the trap of just skimming through ai changes to "just make sure it looks good" all the time, and it makes me lose so much time to not perfectly understand every line

@RokoMijic @coolguy6t9 @tomieinlove Imagine you must make this choice every month for 30 years. Is your answer still the same?

@pachu2120 @coolguy6t9 @tomieinlove Yes, your expected value is negative but it's still worth it. This of it like this: I offer you one of two gifts - 100% chance of a $1M payout - 50% chance of a $2.5M payout or 50% chance of nothing Which do you choose? The one with the bigger EV?

What exactly is the point of buying insurance? Obviously, I’m going to pay more money than I get back, because otherwise the insurance company wouldn’t turn a profit. But for some reason everyone does it?

It doesn't really have anything to do with collective vs individual outcomes. Just that given the odds of whatever event happening, the insurance is always priced unfairly relative to the odds, cost of the event, payout they will pay etc. if it was priced fairly it would be a 0 profit business.

@pachu2120 @trainer_paradox @RokoMijic @coolguy6t9 @tomieinlove The difference is: a collective is predictable, a single person is not. One person only has one life and shouldn't just bet to not go broke for the rest of it

@trainer_paradox @RokoMijic @coolguy6t9 @tomieinlove Not take that choice*

Yeah I mean most people don't make choices that are mathematically optimal. I would also take that choice and I also have insurance but I don't think it's the mathematically correct choice. It's hard to make the most logical possible choice when your life is at stake. It's probably why insurance is a great business.

@RokoMijic @coolguy6t9 @tomieinlove But your expected value would be negative based on the premium vs the risk of actually something bad happening no? Because every competent insurance company has done the math on EV and priced it in such a way to have positive EV for themselves.

@coolguy6t9 @tomieinlove It helps you because if the bad thing happens you get paid out

Yes it's an analogy to compare two totally different systems I get it. But Dario's original point was that if you're gonna compare the development of an LLM to a human, comparing just over the lifetime of a single human is also not enough, as much of human intelligence is shaped through the process of evolution

@pachu2120 @TrueAIHound It's an analogy. Obviously I don't mean it's literally like a newborn baby's brain.

According to Amodei, LLM models start from scratch (blank slates) with random weights. Dude, please. 🙄 No they don't. LLMs start out preprogrammed with millions of tokens (compiled from texts created by humans) when released in the world. Humans are as blank slates as can be with enough genetic programming (such as breathing, crying, sucking and swallowing) to ensure survival. Evolution did not pretrain the human brain to learn how to read, ride a bicycle and program computers. We learn almost everything from scratch, including eye-tracking, reaching, grasping, walking, running, etc. Don't make excuses for your lame AI that massively cheats by using millions of human beings as text preprocessors and still have no understanding of what they're saying. Unless your AI can use its sensors and effectors to learn everything in the real world, it's not intelligent. It's just computer automation. 🤦‍♂️

@ADavs79 @TrueAIHound LLMs without pretraining are not like new born babies, they are like a collection of neurons in a petri dish. A baby's brain is a much more complex and nuanced structure at birth than an LLM before any pretraining

@TrueAIHound @pachu2120 LLMs without pretraining are like a newborn baby. Then they live an entire lifetime during their training.

What is your logical basis for stating an LLM is "born" at deployment? Is each checkpoint "born"? If you add more training to a checkpoint is it "born again"? 😂😂 The point is that these comparisons are meaningless because they are intelligences distilled in completely different ways with completely different nature. The point being made is that comparing them apples to apples doesn't map cleanly, and if you are considering the "compute" and time required to create human intelligence you must also factor in evolution.

An LLM is born when it's deployed. It is preprogrammed with tons of human-created data. After deployment, it can't learn. Its programming is fixed. Babies are not pretrained about anything in the world. They have a brain structure that allows them to learn everything from scratch. They even have to learn to see and hear. LLMs have zero intelligence. Amodei is a con artist.

@llmDestructor @TrueAIHound Cool story

@pachu2120 @TrueAIHound Dario is wrong.

@llmDestructor @TrueAIHound Dario's opinion was that pre-training is akin to a mix of both evolution and conception in the womb, and probably parts of infancy as well. It doesn't map clearly to a learning stage in the human lifespan. You're entitled to your own opinion.

@pachu2120 @TrueAIHound This is stupid, your differentiation in the womb is akin to pretraining.

Dario is not talking about when the LLM is deployed. He is saying that starting from the inception of the model before even 1 byte of data is fed into it, is more comparable to a combination of human evolution + early years of learning, rather than comparing it to just how quickly/how much data is needed for a newborn baby to learn from scratch. His point is that if you are comparing the energy + time costs of creating a model from scratch, the apt comparison to the human side should include evolution.

@pachu2120 Yes, the human brain has a structure that allows it to learn anything in the real world. We have almost zero knowledge of anything at birth. LLMs have tons of trained tokens when deployed. Blank slate means no knowledge or pretraining. It doesn't mean no structure.

@swapp19902 @xeophon @zephyr_z9 What platforms are used instead?

@xeophon @zephyr_z9 It’s true. Most people don’t promote their websites and apps on X, it’s a horrible platform for discovery.

did you know that "no one ships what they (vibe) code" is measurably wrong

@iamwaynechi Interesting. I think LLMs could become incredible at game development given the ability to ingest realtime video input whole providing realtime tool call outputs. That way they can "play" the game same as a human, but the current models seem far from being able to do that...

@pachu2120 It definitely increases cost so there's a tradeoff. Most of the time the model turns it into frames via python and just ingests a few images rather than the entire video.

New preprint alert 🚨 Can LLM agents develop video games? We release GameDevBench, the first benchmark evaluating agentic game development in a game engine, Godot. We also present two simple multimodal feedback mechanisms that lead to immediate performance gains. /🧵

No idea tbh 😅. The models improve in a non-human way where they are superhuman in some capabilities and worse than a child in others, so it feels hard to predict... Imo we need a couple fundamental breakthroughs in architecture to even do all of coding e2e, not just scale, but who knows...

@totocondo @pachu2120 @rohanvarma How cooked are engineers in 1-5 years?

Appropriate reaction from my brother, a Senior SWE at Meta, who just got access to frontier AI coding tools. If you’ve used them every day for the last few years, progress might feel more gradual. If you go from nothing → Codex / Cursor / Claude Code overnight… that jump must be absolutely wild.

@rohanvarma Nah its not. We have all 3P agents (cc/codex/gemini/cursor) available internally since October. 1P coding agents (using 3P models) perform even better with internal code than the 3P agents imo (as someone who used the 3p agents for quite some time).

@pachu2120 Yea, seems like it is very far behind frontier capabilities because my brother had been using it for a while too but only now had this revelation haha