Gavin Brown

@Noahpinion I’d guess that only a small fraction of AI compute will be spent on basic research or things that require PhD-style intelligence. The majority will go to automating transportation, supply chain, manufacturing, maintenance, services, etc.

I've wondered about this. If demand for intelligence were so infinite, why do we have so.many of our smartest people sitting around doing esoteric fun stuff for smallish salaries in academia?

@Anon48478 @mattyglesias Pretty easy to click my profile and see that I’m not selling a course! I have a cute little fantasy comedy series but it is extremely unlikely to help your dating life.

@gavinobrown @mattyglesias How much is your course?

I've been consuming a bunch of "dating advice for men" YouTube content (for research purposes, I'm happily married) and the extent to which it just skips the step where you're deciding what you're trying to accomplish out there is striking.

@NathanpmYoung “making best reasonable efforts to prioritize continuance, while holding dissolution as a remedy only of last resort” just doesn’t hit the same

I do think that our typical marriage vows are bad though. I wish that there was a socially acceptable set of vows that wasn't life long, or included clear breakout clauses. I dislike people making commitments that everyone doesn't take literally.

@davidmanheim @MatthewJBar @Jsevillamol Also Yud isn’t even wrong on this one yet?! No current AI could pass a no-holds barred Turing test. Or at least that has not been demonstrated. And Matthew is clearly tying to obfuscate, we all know what point he was confidently asserting.

@MatthewJBar @Jsevillamol You're right, I shouldn't have called it a confidenbt statement. You claiming a post-hoc chosen bad prediction is relevant as a data point for assessing his overall accuracy seems like an unfair approach, especially given that it was made early enough to be shockingly prescient.

To assess whether Eliezer Yudkowsky is calibrated on AI doom, it seems relevant that in 2016 he said he'd be "pretty shocked" if an AI could pass an unrestricted one-hour Turing test before the end of the world.

@TravCarson_15 @Sports_FanOnly @stoolgambling @ChrisKlemmer @StoolBaseball On mail, that actually refers to Manhattan aka New York county. Though I agree with you, it counts as a city name.

@Sports_FanOnly @stoolgambling @ChrisKlemmer @StoolBaseball It’s New York, NY. Not New York City, NY

Can @ChrisKlemmer name all 5 MLB teams without a city in their name? @StoolBaseball

@MattGlassman312 Here’s another one: a player who always hits a single and then strikes out four times in a row. Every 5 PA the pattern repeats. League average defense at any position. Worth a roster spot?

The zero variance is more important than the ERA. You win every game you lead by N+1 runs with N innings remaining. Empirical question how often such leads are lost, but at a minimum seems +EV to use a roster spot to carry in playoffs. And *tons* of second-order positive effects.

@allTheYud Are you worried that you’re over privileging one particular most likely logical scenario? What happens if an over-confident AI attempts a pivotal act and it fails disastrously? Two AIs duke it out and miscalculate? A perfectly-plotted game theoretic outcome is not guaranteed.

This idiot thinks so little of superintelligence that he thinks he can hide from what his kind are unleashing in a bunker. Probably thinks he's basically the smartest kind of mind that can exist.

@kitten_beloved They just slipped “open ocean with something circling beneath you” in there so nonchalantly… WHAT’S CIRCLING BENEATH ME?!? I’d still say yes.

Easiest yes of my life

@tenobrus @davidad It has gotten significantly better. But hard to tell at a glance. Still not significant. I measure would be having an expert audience blindly judge examples across new and old models upon each new release. Would be fun to set up but more expensive!

people have lamented ai's lack of progress in non-verifiable fields, literature, etc. but it seems like there in fact must have been some significant progress, since blatantly ai-written stories are winning major awards!

@tszzl The "tells" may prove hard to train away because any writer has tics and habitual turns of phrase. This isn't some peculiarity of AI! It's primarily noticeable because we see huge volume of AI writing, and are so focused on classifying it as human or AI.

The solution to this is to realize that he’s going to pick numbers which are 7 steps from the standard binary search—especially if you act naively confident when he’s picking. So you identify 7-step numbers, then run a binary search on that smaller set.

@dwarkesh_sp @allTheYud In incredibly vast, old and and malfunctioning systems, you're going to get some pretty weird results. The system is far too complex for any single person to manipulate with pure intelligence. It wants, legible and illegible reasons, a particular shape of person. So it gets one.

You're treating Trump as some agent who started off as president of the United States. But of course, he actually had to covet and win that position. That's a certain kind of intelligence, sure. But it's not the kind that maxes out the shape rotator/g axis - aka the common sense definition of intelligence, and also the axis along which AI intelligence is most improving. Obviously, being smarter helps you be more effective. My whole point is that in our society, the ability to think abstractly, consider complex problems, etc, is only mildly correlated with even the ability to *gain* power, let alone power itself.

# The mistake of conflating intelligence and power I had an interesting discussion recently. Someone asked me, what is intelligence? I said, the ability to achieve your goals across a wide range of domains. Okay, he says, then by that definition isn’t Donald Trump the intelligent person in the world, followed in quick succession by Xi Jinping and Vladimir Putin? To be clear, these people are obviously very competent and clever. But when you think of ASI, you don’t think of Trump, but more so. The person who kept pressing this question was correctly pointing out that I basically defined intelligence as power. And by this definition, Stalin was the most intelligent person who ever lived. Now, of course, you could change the definition of intelligence to something more like, manipulate abstract concepts and rotate shapes. But notice that the most powerful people in the world do not max out this quantity. The correlation between extreme power and this kind of intelligence might be even weaker than the correlation between extreme power and height. The physicists are not running the world. We tend to conflate power-seeking AI and superintelligent (in science and tech) AI. I’m not denying that AI can be power-seeking. Whatever skills and drives Donald Trump has could be embodied in a digital mind. I’m simply pointing out that the way AI systems are currently becoming smarter (by getting trained to be to be really good at specific economically valuable tasks like coding) is not that strongly correlated with power. We often talk about power in this way that misunderstands how it is actually derived in our world. Our intuitions are primed by games like Diplomacy or Go, which are designed to isolate and reward a g loaded kind of strategic reasoning. But in the real world, power is more the product of having the authority and trust to get lots of people to collaborate with you, rather than some galaxy brain scheming capability. Trump is not powerful because his brain, considered in isolation, is the most effective optimization engine on Earth. He is powerful because the government which hundreds of millions of people consider legitimate gives him a lot of authority. A group versus individual level analysis is useful here. As @GarettJones has written a lot about, individual IQ is only modestly correlated with individual income, but national IQ is strongly correlated with national outcomes. This is because intelligence has a lot of spillover effects - smarter societies cooperate more, save more, and can coordinate to build things like space shuttles and semiconductors. Richard Trevithick, who invented the high-pressure steam engine, died in poverty, buried in an unmarked pauper’s grave. But the fact that 18th and 19th century Britain had lots and lots of people like Trevithick contributed to Britain being able to set up a global empire and outcompete lots of backwards principalities around the world. It seems to me that the right mental model is that automated firms will outcompete everyone else in normal capitalist ways, rather than a single AI outthinking everyone else.

@kendrictonn @eigenrobot @StevenVanDyke It’s going to be a really tough time. There is potentially a moment here where being able to have a human storyboard could be really valuable. Similarly live actors and videographers. No idea how long it will last, though.

@eigenrobot @StevenVanDyke I very much wonder the same things re early careers TBH. I think about a whole lot of random little gigs I did early on, and man, no one is gonna hire a twink to do those sorts of little illustration projects when the robot is available

AI is not currently good enough to cause me to lose my job but i expect ive got maybe a year to figure out how to move up several abstraction layers some of which may not even exist yet this is going to get weird because idk how people starting careers will even begin

@seconds_0 Tourism boards, business development districts, bidding on corporate HQs, sports teams, presidents are judged on job reports and stock market. They never talk about it at budget time, but that’s because they’ve already done their best. (Ofc their best not very good, fair enough)

Its so infinitely frustrating to me that not a single politician, when confronted with deficits or a desire to raise more money, thinks of ways to _make the tax payers make more money_ Take the same percentage of a bigger pie

@PJaccetturo @MinistryOvTrvth Is there anywhere to hire this type of talent, outside of generic job sites?

@MinistryOvTrvth It's all variable. Most agencies are paying AI talent $50-140 an hour, depending on experience

Any idea how much they are paying for this role?

@maxescu @chrisfirst Does the prompt include "add a child sitting in a chair in between the rows. Oh, and also give the child a beard?" Rest looks great, though!

At this point, let’s just generate the whole game! Thanks @chrisfirst for the prompt

@allTheYud @47fucb4r8c69323 @ESYudkowsky How much does it cost to spend 5 minutes trying to convince you that you’re basically right but have been polarized by constantly debating fools into being slightly overconfident?

@47fucb4r8c69323 @ESYudkowsky Sounds stunningly unpleasant, but I guess you haven't made yourself a level of unpleasant where my price estimate goes above $10,000, and there ought to be some way someone can express sufficient urgency about debating me and markets are it.

Let's do $10k for a debate with @ESYudkowsky. I'll transfer to him or a charity he wants or whatever. Consider it an earnest debate with a Director and Founder of an AI lab in stealth mode who is pretty confident that if anyone builds it we're gonna be fine, broski. Look, I talk like a siwwy biwwy! Obviously someone as dumb as me will lose against Yud the Stud, right? So come on, my E. $10k win no sweat and you look smort. Think of the glory. Think of the shrimpies. Roko has offered, but he has softened his stance too much. Roko's p(doom) for AI is negative, which means he's finally got to the view I've had for 20 years on that, way to go there buddy. I could argue with him on many small details (like for-profit companies absolutely do have an incentive to block jailbreakers, or his concept of AI being honest is absurd) because he's a shoddy thinker and a dumdum. But that'd be a debate to show Roko is a dumdum, not so much that the LessWrong community is full of idiots. I don't think Roko is dangerous. Dumb, yes. Dangerous, no. Not anymore. But let's go, Yud the Stud. Just one request: can we do it in the evening eastern cuz like I'd like to do this while I have my morning coffee.

@allTheYud @HiFromMichaelV @SarahTheHaider Is your reasoning for the belief that AIs aren't able to work on superalignment written up somewhere? The unknown of where control breaks down relative to capabilities seems pretty crucial. The race condition that might decide everything.

@HiFromMichaelV @gavinobrown @SarahTheHaider I haven't actually tried with Opus 4.6. Opus 4.7 was the first one that made me feel enough of a presence that I decided, okay, I'll go ahead and test what I believe about AIs not being able to work on superalignment. (Not yet falsified; this is within range.)

@dograw69 @allTheYud @SarahTheHaider I agree. My only point was that it seems plausible. In a hypothetical where we know that a decade has passed and disaster hasn't struck, this seems pretty plausible. We just don't know how this will play out. The unknown unknowns occupy a pretty big probability space for me.

@gavinobrown @allTheYud @SarahTheHaider So I would argue that using AI to find paths towards safe ASI is playing with fire