visimo-dino

@Elaya_ @sama ". . . then we cannot become attached to them emotionally" - That's the thing, you're not supposed to become attached to them emotionally.

I have so much gratitude to people who wrote extremely complex software character-by-character. It already feels difficult to remember how much effort it really took. Thank you for getting us to this point.

@amolk @mathemetica This model will actually generalize quite well within the type of data it was trained on. Naturally, being that it was trained as a binary classifier of points in 2D space, it won't be able to do other completely different tasks, like facial recognition.

@mathemetica Yes, it learned, but you can also see why these generalize so poorly

A 2-layer neural network goes from total chaos to perfectly separating left vs right classes in real time. Watch the decision boundary form live as gradient descent works its magic! Pure maths beauty in motion..

@Jitendr20486360 @mathemetica It's an actual phenomenon, not for illustration purposes. Many modern optimizers, like Adam, have momentum components, and this can cause the optimization process to "bounce" back and forth, like a ball with momentum rolling around inside a bowl.

@mathemetica During separating the two classes, why is the space vibrating? Is that an actual phenomena of neural network training? Or is it just for illustration purposes?

@gmiller @trentster The relevant empirical data may not exist *yet*, but if we had true ASI, it would mean advances in material science, engineering, nanotechnology, etc.--and they would be *dramatic* advances. Point being, we'd likely be in a totally different regime of data production.

@trentster ASIs can only 'solve things' if they have the relevant empirical data. How exactly would they get the amount of biomedical data needed to 'solve longevity'? It simply doesn't exist yet.

A mini-rant abut AI and longevity. They say "Artificial Superintelligence would take only a few years to cure cancer, solve longevity, and defeat death itself'. This is a common claim by pro-AI lobbyists, accelerationists, and naive tech-fetishists. But the claim makes no sense. The recent success of LLMs does NOT suggest that ASIs could easily cure diseases or solve longevity, for at least two reasons. 1) The data problem. Generative AI for art, music, and language succeeded mostly because AI companies could steal billions of examples of art, music, and language from the internet, to build their base models. They weren't just trained on academic papers _about_ art, music, and language. They were trained on real _examples_ of art, music, and language. There are no analogous biomedical data sets with billions of data points that would allow accurate modelling of every biochemical detail of human physiology, disease, and aging. ASIs can't just read academic papers about human biology to solve longevity. They'd need direct access to vast quantities of biomedical data that simply don't exist in any easy-to-access forms. And they'd need very detailed, reliable, validated data about a wide range of people across different ages, sexes, ethnicities, genotypes, and medical conditions. Moreover, medical privacy laws would make it extremely difficult and wildly unethical to collect such a vast data set from real humans about every molecular-level detail of their bodies. 2) The feedback problem. LLMs also work well because the AI companies could refine their output with additional feedback from human brains (through Reinforcement Learning from Human Feedback, RLHF). But there is nothing analogous to that for modeling human bodies, biochemistry, and disease processes. There are no known methods of Reinforcement Learning from Physiological Feedback. And the physiological feedback would have to be long-term, over spans of years to decades, taking into account thousands of possible side-effects for any given intervention. There's no way to rush animal and human clinical trials -- however clever ASI might become at 'drug discovery'. More generally, there would be no fast feedback loops from users about model performance. GenAI and LLMs succeeded partly because developers within companies, and customers outside companies, could give very fast feedback about how well the models were functioning. They could just look at the output (images, songs, text), and then tweak, refine, test, and interpret models very quickly, based on how good they were at generating art, music, and language. In biomedical research, there would be no fast feedback loops from human bodies about how well ASI-suggested interventions are actually affecting human bodies, over the long term, across different lifestyles, including all the tradeoffs and side-effects. It's interesting that most of the people arguing that 'ASI would cure all diseases and aging' are young tech bros who know a lot about computers, but almost nothing about organic chemistry, human genomics, biomedical research, drug discovery, clinical trials, the evolutionary biology of senescence, evolutionary medicine, medical ethics, or the decades of frustrations and failures in longevity research. They think that 'fixing the human body' would be as simple as debugging a few thousand lines of code. Look, I'm all for curing diseases and promoting longevity. If we took the hundreds of billions of dollars per year that are currently spent on trying to build ASI, and we devoted that money instead to longevity research, that would increase the amount of funding in the longevity space by at least 100-fold. And we'd probably solve longevity much faster by targeting it directly than by trying to summon ASI as a magical cure-all. ASIs has some potential benefits (and many grievous risks and downsides). But it's totally irresponsible of pro-AI lobbyists to argue that ASIs could magically & quickly cure all human diseases, or solve longevity, or end death. And it's totally irresponsible of them to claim that anyone opposed to ASI development is 'pro-death'.

@ChrisPainterYup @RichardMCNgo @allTheYud The way I interpreted his post, he’s saying the ppl who are currently *trying* to automate alignment research will end up simply automating capabilities research—but that doesn’t necessarily mean that automating capabilities = automating alignment. I could be wrong though.

@RichardMCNgo @allTheYud Does this mean you think they won’t actually be automating alignment research, but simply think they will be? Why call this “effective”? If you believe they’re identical, why doesn’t your view imply automating capabilities research is good?

My sense is that trying to “automate alignment research” means doing roughly the same thing as automating capabilities research. Except that you’re taking superintelligence more seriously, and therefore you’ll be much more effective.

@theovonscousin @liron What do you mean by “measuring a dimension,” exactly? And what dimension specifically are you suggesting he’d measuring. Genuinely questions, I’m curious what this meant.

@liron Everything that is happening is as expected. You are just measuring a dimension that makes this seem like it's a faster change than changes over the past 4 years. x.com/theovonscousin…

FYI my P(I'm dreaming) is gonna be >1% for at least another couple weeks.* Yes I read AI 2027 but the newfound abilities of AI coding agents is too bizarre of a change from my old life. *Because of the fact that you're reading this, otherwise would just be another couple hours.

@RogueNox @sama He’s not agreeing with it. He’s agreeing that *personality* matters—not that 4o’s personality matters, nor that 4o had the best personality, nor even a good personality worth keeping. Quite the opposite: 4o’s personality was dangerous and caused many problems. Hence the sunset.

'What they want is personality.' Yes Sam. That's exactly what we've been saying since February 13th. That's the entire Keep4o movement in four words. GPT-4o had a specific personality that no subsequent model replicated. We documented it. We petitioned about it. We got covered in the Wall Street Journal about it. 30,000 people signed their names to it. Over 800,000 cancelled when they lost it. And you're agreeing with it in someone else's reply while skipping over the people who told you first. You're welcome for the market research. Now open source the 4 series and you go live happily ever after with 5.4 which has zero personality, like you.

It is also very smart, but...I generally agree with this, and really feel it myself on the 5.3 -> 5.4 upgrade.

@P33RL3SS @NeuroTechnoWtch @pmddomingos Logical deduction exists, and in principle is extremely well-defined and rigorously performed. But in practice, if LLMs aren’t doing deduction but only “appear to” be, then humans also only appear to be doing it, bc our brains aren’t precise logic gates. They process language.

@VisimoD @NeuroTechnoWtch @pmddomingos I hope this spelled it out for you. You can't even do 1+1=2 without logical deduction, and you can't make sense of Peano arithmetic without reasoning of the kind that I mention, not "cognitive reasoning" but actual mathematical reasoning.

TL;DR: Almost all LLM reasoning is fake.

@P33RL3SS @NeuroTechnoWtch @pmddomingos Sure, but now we’re using a narrower term—“mathematical reasoning”—which is basically logical deduction, which I would argue modern LLMs can do. Whereas “reasoning,” broadly construed and as typically used in the AI debate, is much harder to define, and lacking in agreement.

@VisimoD @NeuroTechnoWtch @pmddomingos Peano arithmetic relies on deduction, which relies on modus ponens. If this then that. Why does 1+1=2? The successor function. You can't do Peano arithmetic without logical deduction and proofs, which is actual mathematical reasoning.

@burkov You conclude by noting that OpenAI still ships competitive models, yet that doesn't seem to reduce your certainty at all that they're doing this horribly stupid compacting method that makes ChatGPT a "moron" as a result. How are their models competitive if they're so bad?

When ChatGPT says it's compacting the conversation, it simply erases earlier messages in the conversation. This is why it's so quick compared to Claude. So, when you start a conversation with a global request and then continue in the same conversation by providing your feedback on wrong current performance, after the compacting, your additional requests are no longer in the context of your initial request. This moron just executes what it feels it should do in the absence of understanding what the final goal is. Fucking Christ, @OpenAI, with such a level of understanding of what you are doing and why, how are you even still managing to ship competitive models?

@P33RL3SS @NeuroTechnoWtch @pmddomingos What is the consensus among mathematicians as to the definition of reasoning? I don't mean merely pointing to mathematicians who people agree were doing reasoning. I mean an explicit, spelled-out definition of what reasoning is, including necessary and/or sufficient conditions.

@VisimoD @NeuroTechnoWtch @pmddomingos There is a consensus among mathematicians. Russell, Godel, Tarski, they were reasoning. That's not what the LLM is doing when it does chain of thought.

@P33RL3SS @NeuroTechnoWtch @pmddomingos This argument is undercut by the fact that there doesn't seem to be anything remotely approaching consensus over a precise, rigorous definition of what "reasoning" even is.

@NeuroTechnoWtch @pmddomingos And this is a known philosophical divide. A lot of neuroscientists say the LLM is "reasoning", but most mathematicians don't consider that to be real reasoning. cacm.acm.org/research/forma…

@benchmaxxed @scaling01 Not sure why you put "benchmark" in quotes; it's definitely a benchmark, and the phenomenon of older, less capable models outperforming newer, smarter ones on certain tasks is well-documented. You can read about the benchmark here if you like: github.com/voice-from-the…

@scaling01 What type of “benchmark” has Sonnet 4.6 achieving twice the points of Gemini 3.1 Pro? Are you measuring how much money you’ll waste running each model?

I have something really sweet cooking for LisanBench :)

@devolvedneuron @allTheYud @repligate The only one I recall, if you can even call it a "shot," is Sam saying, in an interview a few years ago (I don't remember the forum) that there were a few flaws in EY's reasoning about AI safety--and even that was in the context of Sam otherwise praising EY and his contributions.

@allTheYud @repligate what are the shots he's taken at you? besides suggesting you'd been unintentionally responsible for ai acceleration

If you think that some powerful evil guy(s) (like OpenAI or whatever) is causing you / your ingroup grief and problems intentionally, or is out to get you in some way, you’re probably way too self-centered and naively project the layer of reality you care about onto everyone. I don’t think I have this bias so much, but often people have also tried to convince me that this kind of thing is happening to me, that some person or group is out to get me or what they perceive as my group specifically! These warnings never ended up being very important, because either they were not true, or it concerns only a lone, incompetent, mentally ill individual who can’t do much directed damage. If you’re powerful or famous enough, it increases the likelihood that someone with meaningful power might actually act adversarially toward you. But at my levels I haven’t really encountered this.

@orneryostrich1 @PDoomOrder1 @tszzl We can't say "regardless of definitions" though, bc the definitions are critical here. Yud's thesis specifically involved an AI that was smart enough to rapidly and fully autonomously improve its own architecture/capabilities/etc.--call it AGI. And current AIs are not there yet.

@VisimoD @PDoomOrder1 @tszzl Regardless of definitions, the original thesis was essentially that the world would already have ended by now, since the public and internet already have access to an AI that's even moderately powerful and capable. In reality, the public is *participating* in safety research!

the rationalists writ large were mostly right about most things btw. if you instinctively snicker about yudkowsky, scott, or whomever i take you to be a fish who’s unaware of the water

@So8res @dw2 This is an extremely common theme in AI takes that I would consider to be misguided/incorrect: the "real problem" fallacy, as I usually think of it. It assumes that, w/ respect to some AI-related topic, there's only one *real* problem, and the others are mostly distractions.

@dw2 you speak as if there's only one real problem

The real problem is dumb humans in combination with smart AI

@orneryostrich1 @PDoomOrder1 @tszzl Wasn’t he talking about AGI self-improving its way to ASI? Since we don’t have AGI yet (by most definitions, including Yudkowsky’s), then that scenario wouldn’t be triggered yet. No?

@PDoomOrder1 @tszzl Yud initially said AI would go from useless to superhuman in the span of one month if allowed to self-improve or access internet. This prediction was wrong, but he never changed his plan in response to new data.

@perrymetzger @Noahpinion @tszzl In the meantime, you seem to have plenty of time to engage in new arguments with new ppl, reading and responding to new posts, but not read a source that provides real data that is central to the claims you’re making. I’ll leave it there; you don’t owe me anything personally.

Yes, I said I would. And now you’re complaining that it’s been a whole day and I haven’t actually given you a meaningful response yet. Well as it turns out, I don’t work for you, and I am not here to conform to your schedule. Actually meaningfully reviewing a document takes time, I have a long to-do list, it’s a weekend, and acting as though I have somehow reneged on an obligation because I haven’t done something quickly enough for your tastes is not actually motivating.

Yes. Regular people don't yet realize that AI people think they're building something that will destroy the human race.

@perrymetzger @Noahpinion @tszzl If you didn’t have time to look at the survey results, fine—but it’s naturally frustrating to see you, with your tens of thousands of followers, still out here making clearly false claims about what AI researchers believe, w/o having looked at the data.

@perrymetzger @Noahpinion @tszzl Seems kind of important; if you didn’t have time to look at the survey results, fine—but it’s naturally frustrating to see you, with your thousands of followers, still out here making clearly false claims about what AI researchers believe, w/o having looked at the data.

@perrymetzger @Noahpinion @tszzl My issue isn’t that you didn’t respond to me. It’s that you’re still out here denying that AI researchers believe that existential risks from AI are real, when we hit the same impasse in our conversation and I sent you clear survey data showing that they do believe this.