Ben Finn

@robinhanson First lesson I learnt from philosophy. Not sure much is beyond reasonable dispute either. Not even 2 + 2 = 4

No claim or topic is "beyond dispute". It might be beyond reasonable dispute, but not beyond dispute.

@QiaochuYuan In other news: x.com/kobihackenburg…

i would really prefer to never have to talk about this again in public but here goes. i was heavily involved with brent dill for a substantial chunk of 2018, when he was embedded in the bay area rationality community. based on those experiences i never want to get involved with him ever again and i highly recommend none of you ever do so either. please note that this is the strongest condemnation i have ever written publicly about anybody brent dill is basically a cult leader, and the thing he constantly does is basically attempt to start or take over cults. when he is talking to you he is constantly attempting to probe you for any psychological weaknesses he can take advantage of in order to convert you into one of his supporters. most people find him repellent but he is good at finding vulnerable people this will work on when this is happening to you, from the inside it feels like finally you've found someone who's saying things that make sense, that matter, who's bravely pointing out the elephants in the room nobody else is willing to, who is being unfairly persecuted by the popular monkeys, who's showing you all the social rot and promising in so many words that maybe, just maybe, you and him can fix it together, because you get it, right? you're not like the others. you see the truth. you're so special. or whatever. he will tailor different versions of this pitch to different people based on what he's figured out about you. the pitch includes, among other things, a sophisticated defense against any attempts to warn you about him, which look like confirmation of his persecution thesis (importantly, i'm not saying the stuff he points out is wrong, that is itself part of the trap. he pointed out a lot of things about the rationality community that were true in some sense and were part of the reason we - black lotus - were drawn to him instead. cult leaders always work like this. there is always at least a kernel of real value mixed in with the poison and demons, but it is as far as i can tell literally never worth it) getting out from under brent's manipulation and control was one of the most destabilizing experiences of my entire life and one i have spent years recovering from. the whole time i thought i was making my own choices and seeing things clearly - that i was built different - and i was essentially having my mind taken over in ways i had no idea were possible in the real world and that, if present-me were to describe them to before-i-experienced-them me, i would flat out not have believed. if you've never experienced anything like this you are blessed. i had to reconstruct my entire understanding of myself and the world from scratch because i had lost trust in everything in a very deep and fundamental way; in particular i had to reconstruct my entire understanding of good and evil because i had lost trust in my ability to discern whether anything at all was good or evil if you are the sort of person who brent has decided is worth targeting, you are not prepared to deal with somebody who's put literally all of their skill points into manipulation. in the next tweet i will link some resources that might or might not be helpful. may god have mercy on our souls

@eigenrobot It’s true, our leaders, governments and defence policy have been quite embarrassing in recent years. But compared with the US…!

@odavies9 Probably why in 1968, 37% of Britons had no teeth. None at all nature.com/articles/sj.bd…

Researching dental popular beliefs prior to 1950s & NHS. It was not uncommon for people to get their teeth removed by the age of 30. As one woman in a 1943 survey commented: “it’s a pity not to be able to keep your own. But it saves a lot of bother.” The good old days

@MeganTStevenson Impossible to maintain there’s no real intellectual difference between someone like that and someone else who can do tasks 1000 times as complex

@MeganTStevenson I suspect you don’t interact much with dim people (nor do I), who are of course a significant fraction of the population. I know someone who’s dim, IQ maybe 80. Hard to talk to - gets the wrong end of the stick all the time. Is confused by the simplest things. Sad but not rare

This is what makes me doubt the idea of “IQ” as a meaningful construct. I work in elite academia and honestly don’t see a lot of variation in raw intellectual horsepower, either among academics or in comparing academics to non-academics. 1/

@robinhanson But then you might start another war so you can ‘win’ it too

Hey if claiming to win a war when you actually lost it allows you to end that war, I can be okay with that.

@snewmanpv @satyanadella And we’ve seen similar AI arguments before, in chess. In the 1990s/2000s, even though chess programs beat top grandmasters, it was thought humans were still best at strategy, so a human + computer team would be best of all. But now it’s like a toddler helping to fly a plane

@snewmanpv @satyanadella “Humans will set ambitious goals, connect dots across domains, build relationships, and recognize patterns that matter most.” The only one of these where humans might have a lasting advantage is relationships, but human ones don’t seem irreplaceably essential to many businesses

This @satyanadella quote captures a key idea... that is flawed + highlights a critical threshold. So long as agents need supervision and high-level guidance, you need to maintain your own understanding. When the agent no longer needs guidance, your learning becomes superfluous.

@snowmaker Though almost all ideas (good and bad) have been had before, as anyone who does a patent search soon discovers. So not convinced it’s a useful way of finding good/impactful ideas specifically

If you want to find a truly impactful idea, a good place to look is in the trash bin of history. Most of the ideas that have most broken out in the last 5 years aren't new. They're old ideas that got written off too early. examples: 1) AI: When Sam started OpenAI, saying "AGI" got you laughed at 2) Space: Elon started SpaceX after people had written off the 1960s-era dreams as sci-fi 3) Supersonic flight: Blake started Boom after everyone assumed Concorde already proved it couldn't work 4) Nuclear energy: fission and fusion are roaring back decades after ambitious people stopped studying nuclear physics 5) GLP-1's: After the fen-phen disaster, weight loss drugs were synonymous with snake oil. There's a good reason this keeps happening. When a hyped idea fails, there's a backlash. It becomes embarrassing to work on - anyone still working on it is assumed too dumb to know better. So, if you want to work on something like this, you have to teach yourself to feel the tinge of that embarrassment and push past it.

@WalkerMarcus @andrewgadsden Also mature enough to have children themselves

It is literally insane simultaneously to think that 16 and 17 year olds are mature enough to vote but not mature enough to look at Instagram at 8:30pm. This is comically absurd.

@robinhanson We have first person data on various conscious processes surely? (Also some data on morality from actual cases & thought experiments, insofar as there is general agreement on many of them, but by no means all)

Questions of which processes are conscious, and which acts are moral, are both cases where we can have, apply, and draw inferences from beliefs, but can't infer theories from data in a Bayesian sense, having no actual data. We are stuck arguing for various priors.

@StefanFSchubert But I assume economic benefits of soft power are hard to measure. Cf I heard an expert say recently that the James Bond films genuinely make foreign governments think the UK is very good at espionage, maybe also defence. Much cheaper than paying to be so

It's interesting that the economic returns of that seem relatively small (though not zero)

@bentleygyeadon @ediz1975 Pedestrians often/usually walk into cycle lanes by mistake - they are not always obvious. But cyclists who go through red lights or zebras crossings almost always do so deliberately

@ediz1975 I could do the same but with pedestrians walking into cycle lanes. Works both ways

The reality is cycling lobby groups believe this is OK as nobody is getting hurt and if they did it would not be serious. This is a day to day event unfortunately… I see it first hand while cycling . … often being shouted at if I ask others to stop. We need to do so much better …..🚴🚴🚴🚴🚴🚴🚴🚴🚴🚴🚴🚴🚴🚴🚴

@ubuto23 @robertwiblin Cf numismatics (the study of coins and banknotes) tells us nothing about economics. BTW it sounds like you think consciousness isn’t real. Indeed that’s one philosophical position on it. Though hard to explain the apparent reality of sensations eg pain. This stuff isn’t obvious

@ubuto23 @robertwiblin Not clear neuroscience can tell us much about what consciousness is. The brain, neurones etc don’t have labels marking conscious parts, or that say why they’re conscious. Only once we know what consciousness is might neuroscience have things to say about the biology of it

"Fable Max Incognito, what do you think of the quality of the line of argument in this piece?" "The central problem is an equivocation between two different things called "consciousness." The neuroscience Cowen marshals — the Schwartz quote, the implicit reference to Libet-style findings about decisions preceding awareness — concerns conscious control: whether the narrating self is the causal author of behavior, whether deliberation is post-hoc confabulation. But the question Anthropic, Hinton, and the philosophers in that FT story are asking concerns phenomenal consciousness: whether there is anything it is like to be the system, whether anything is felt. These come apart completely. You can grant that your sense of authorship is confabulated and that "you" decide nothing, while it remains true that pain hurts and red looks like something. Indeed, the confabulation story presupposes experience — the misleading narrative is itself something experienced. So the essay's deflationary evidence strikes a different target than its title claims to. Worse, the two halves of the argument pull in opposite directions. If human consciousness is a thin, marginal gloss riding atop sophisticated unconscious processing, that lowers the bar for machine consciousness rather than raising it. The more you deflate what consciousness amounts to in us, the harder it becomes to confidently assert "there is no ghost in the machine" about systems that also do sophisticated sub-personal processing. And notably — at least in this excerpt — that headline claim about AI receives no argument at all. It's a bare assertion borrowing credibility from the essay's general deflationary mood. The evidential claims are also overstated. "Among experts in neuroscience, this is not a controversial proposition" is not accurate: the Libet readiness-potential interpretation has been substantially challenged (Aaron Schurger's work suggesting it reflects stochastic neural noise rather than a decision already made, plus longstanding criticisms that arbitrary finger-flicks tell us little about deliberative choice). There's also a small irony in dismissing Hinton's authority on AI consciousness while letting a neurosurgeon's philosophical view about the self settle the human case. The anthropomorphism argument is the most interesting move, and it's legitimate as far as it goes — humans really do over-attribute minds, and this is a fair caution about reading sentience off chatbot behavior. But the extension to the first-person case doesn't work, because self-attribution of experience isn't an inference from behavior the way attributing anger to a thunderstorm is; the seeming is the datum itself. Cowen half-concedes this with his "conscious enough to avoid self-contradiction" line, but that concession gives the game away. Once any experience exists, the substantive question — which systems have it, and how would we know — is back on the table, untouched by everything that preceded. "Barely conscious" doesn't dissolve the moral stakes either, since suffering doesn't need to be in executive control to matter. Bias arguments can shift priors, but they can't settle object-level facts — and the bias cuts both ways, since humans are also demonstrably good at denying minds when it's convenient. To be fair: the confabulation literature he gestures at is real and important (Nisbett and Wilson, the split-brain "interpreter"), introspection genuinely is unreliable, and puncturing human specialness is a useful corrective to glib "it's just matrix multiplication" dismissals — that cuts against his own conclusion, but it's a service. The piece also gestures toward illusionism, which is a serious philosophical position — though serious illusionists like Frankish and Dennett still treated "which systems have the relevant functional organization" as a substantive open question, not one dissolved by deflation. So my overall assessment: rhetorically effective, built on real science, but structurally it substitutes a provocative reframing for an argument about its stated question. It works better as a Cowen-style provocation than as a proof — and its strongest moves, followed honestly, lead somewhere more uncertain than "no ghost in the machine." thefp.com/p/tyler-cowen-…

@TheZvi Yes, the land of Babbage and Turing (plus Hassabis etc) is not bad at computing

UK AISI is doing some great jailbreaking work. They seem to consistently be able to get through, where others don't.

@paulg Similar story here - most of my company’s customers were classical musicians, who are very nice; even the famous ones, who are surprisingly humble. (One of the few unpleasant customers was a B-list indie pop musician, who I won’t name!)

I talked to a founder today whose customers are all really nice people, because their initial market happens to be one that attracts such people. The founders themselves are nice for the same reason. They're enjoying starting a startup as much as any founders I've known.

@paulg Presumably their only concept of computers was from sci-fi tv & films of the time

@paulg Oddly when I was a kid in the 1970s, it was quite widely believed by people who knew nothing at all about computers and had never seen one (ie most people) that computers kinda knew everything, could answer difficult questions, etc

When I was a kid I didn't realize, as is now obvious, that AGI would be a wide band rather than a sharp finish line. But if you'd shown me a version of ChatGPT that had been told to act like a human, I'd definitely have said that AI had been achieved.