Luke McNally

904 posts

Luke McNally banner
Luke McNally

Luke McNally

@l_mc_nally

Senior Lecturer at University of Edinburgh. Research and outreach on AI X-risk. AI tools for education. Views my own.

Edinburgh Katılım Eylül 2013
1.5K Takip Edilen1.3K Takipçiler
Sabitlenmiş Tweet
Luke McNally
Luke McNally@l_mc_nally·
Our analysis shows that the impressive advocacy campaign by @ControlAI on ASI extinction risk is among the most effective in recent British political history! I think it's the most effective place to put your money and support to prevent ASI!
Torchbearer Community@JoinTorchbearer

A team of about 3 from @ControlAI cold-emailed the UK Parliament about AI extinction risk. At the time they had no contacts, insider help, or established organisation behind them. 16 months later: 125 cross-party signatories, two House of Lords debates, and a Commons amendment.

English
2
6
13
452
Luke McNally retweetledi
Torchbearer Community
Torchbearer Community@JoinTorchbearer·
"Imagine we're driving in thick fog and we know somewhere there's a cliff but we don't know where." The industry's response is to speed up. Connor Leahy (@NPCollapse) gives @moderncto_io a good idea of how AI companies are building systems they admit they don't understand.
English
2
2
9
230
Luke McNally
Luke McNally@l_mc_nally·
They are the same drug. Crystals can self-seed, where one configuration leads to other crystals growing from it in the same form. Nothing changed in the recipe for the drug. Once the new crystal form appeared it spread microscopically and all the other new batches got contaminated and the crystals "grew" as the useless form. Key point is that drugs spontaneously switching to a useless form in a way that can spread shows how physics/chemistry/biology may have all sorts of hidden ways that they could make our world fall apart, and superintelligence could find them.
English
1
0
3
30
michael fares
michael fares@Stealthsilent0·
@NPCollapse @l_mc_nally I'm looking at the post, there is a step that might be skipped. I don't see two connections. 1. What relevance the 1998 drug had to the 1996 drug. Why was a new drug created that doesn't work? 2. How that ties back to a "security mindset" I'm not an insider, so I'm lost. :/
English
1
0
1
216
Luke McNally retweetledi
Connor Leahy
Connor Leahy@NPCollapse·
When I talk to people that instantly, deeply, get the risk from superintelligence, they often share a trait sometimes called the "security mindset" I think this post by @l_mc_nally is a nice brisk touch of what it feels like to think seriously about security even briefly. 1/
Connor Leahy tweet media
English
8
13
112
12.9K
Luke McNally
Luke McNally@l_mc_nally·
People often ask me how superintelligence could cause catastrophe and human extinction. In this article I try to illustrate how our civilisation relies on fragile invariants that misaligned superintelligence could easily manipulate. Link in reply.
Luke McNally tweet media
English
2
2
7
154
Luke McNally retweetledi
Connor Leahy
Connor Leahy@NPCollapse·
The problem we face with AI today is not a technical problem, it is a political problem. Of who gets to decide. What level of risk the public is exposed to, what future we build towards or avert. It's so heartening and important to see this conversation starting to happen in the world outside the very insular tech futurist bubble. We need to have these conversations, everywhere, and this piece by @andreamiotti hosted by Francis Fukuyama I hope is a very useful step in that direction!
Francis Fukuyama@FukuyamaFrancis

We Need an International Treaty to Ban Superintelligence open.substack.com/pub/persuasion…

English
8
19
114
8.8K
Luke McNally retweetledi
Torchbearer Community
Torchbearer Community@JoinTorchbearer·
"Because I'm going to be smarter than you" This clip is from a new short film by @ForegoneFilms on the race to develop superintelligence. Please watch the full 16 minute film (link below) to help understand the dangers of ASI and the industry that’s racing to build it.
English
4
14
42
2.6K
Luke McNally retweetledi
Torchbearer Community
Torchbearer Community@JoinTorchbearer·
Modern AI agents are more than just tools A new article by @AAnestrand on Why rising AI autonomy comes with increasing risks and governance challenges. Click the link below to read all about it!
Torchbearer Community tweet media
English
2
3
7
120
Luke McNally
Luke McNally@l_mc_nally·
@SierraHotel28 @morallawwithin Evolution has also not optimised as much as many think. The whole adaptationism debate revealed many limits to optimisation by natural selection.
English
1
0
4
66
nobody?
nobody?@SierraHotel28·
@l_mc_nally @morallawwithin The key thing being, evolution has optimized a lot, but biology also has to perform dozens of things at once. Animals have to grow, sure, but it needs to be in a way that lets them move well, they need to perform homeostasis, they need immune systems, reproductive systems, etc.
English
1
0
3
75
florence 🦐🏳️‍🌈
florence 🦐🏳️‍🌈@morallawwithin·
Biologists can correct me if I’m wrong but these armchair evolutionary arguments are really sus. For one, evolution+factory farms have straightforwardly not cut out every cost not necessary for the production of muscle, as evidenced by the fact that animals’ bodies do things that don’t serve the purpose of helping the animal produce muscle, and by the fact that we’ve literally made muscle tissue without brains, etc. Second, and more fundamentally, are there really no cases of us doing something more cheaply that evolution has optimized for a while? Like, what’s the cheapest way to get insulin, is it to raise a whole animal and extract their insulin?
Dwarkesh Patel@dwarkesh_sp

Cultivated meat is not on track to ever be economically competitive with animal meat. Modern factory farms have cut out almost all costs that aren't strictly biologically necessary for the production of muscle. And animal evolution has been optimizing the conversion of energy into muscle for billions of years. That means that cultivated meat needs to beat billions of years of accumulated evolutionary efficiency to become competitive. That's an extremely hard challenge. It's maybe even harder than building AGI, since evolution has only been optimizing for intelligence for tens of millions of years.

English
19
1
260
23.6K
Luke McNally retweetledi
Andrea Miotti
Andrea Miotti@andreamiotti·
Anthropic and OpenAI did not call for a pause. Read the wording: "good for the world to have the option", "possible" to slow down "when needed". This is how they signal safety to one audience, acceleration to another. What is not vague is their unequivocal commitment to RSI.
Andrea Miotti tweet media
English
2
22
56
4.6K
Luke McNally retweetledi
Alex Amadori
Alex Amadori@testdrivenzen·
Any plan for surviving superintelligent AI that doesn't go through strong international coordination fails in at least one of three ways: - It sparks war between nuclear powers - It causes a misaligned ASI to kill everyone - It establishes a permanent dystopian dictatorship
Alex Amadori tweet mediaAlex Amadori tweet media
English
2
6
46
6.8K
Luke McNally
Luke McNally@l_mc_nally·
Philosophize this! It reminds me of parental leave with my first and added so much breadth to my philosophy education. He is also great at weaving a narrative path across episodes while keeping them self contained enough for you to jump around. I think it's the perfect backdrop to spending time with your baby, which makes you ponder the big things while simultaneously being objectively quite boring.
English
0
0
2
265
Seán Ó hÉigeartaigh
Seán Ó hÉigeartaigh@S_OhEigeartaigh·
Confession: I've never been a podcast guy, never listened to one fully through. But I'm going to have a lot of 'baby in stroller in the park' time, so it's time to start. Would love recommendations please, don't skip the 'obvious' ones. AI/governance obv, but also history, politics, philosophy etc, stuff that would be interesting to learn about so that I can make conversation that isn't about dirty nappies. For the AI ones, specific episode recommendations (that have aged well/are still useful) especially helpful.
English
50
0
33
6.9K
Luke McNally
Luke McNally@l_mc_nally·
The technology is basically identical so far. I don't see any difference other than surface personality and rhetoric. I think we'll see more of AI labs leaning into the "tool" label to try to make their systems sound more safe and less disempowering, but the differences will stay surface level: open.substack.com/pub/torchbeare…
English
0
2
4
110
Seán Ó hÉigeartaigh
Seán Ó hÉigeartaigh@S_OhEigeartaigh·
This is interesting, and it's certainly coming across in their communications (and some vibes, though I think some OAI folk still see something different than 'tools' on the cards). But: so far, they are building things that seem almost identical. At what point do their AI systems meaningfully diverge along these paths? A loving ensouled machine God is a very different thing than a toolkit for human progress, even if the former can provide the latter.
Joshua Achiam@jachiam0

The OAI / Anthropic values difference is deeply misunderstood, even within the walls of both. Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.

English
1
0
22
1.5K
Luke McNally
Luke McNally@l_mc_nally·
Another example of the AI labs trying to capture the "tool" label. The technology is basically the same, one is simply labelled a "tool". The label doesn't make it any safer: open.substack.com/pub/torchbeare…
Joshua Achiam@jachiam0

The OAI / Anthropic values difference is deeply misunderstood, even within the walls of both. Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.

English
0
1
3
108
Luke McNally
Luke McNally@l_mc_nally·
1. The AI labs should not get to have any say in this. It needs democratic control. If you want to take what seems like unprecedented risks with the future of humanity you need to convince us and get consent. 2. Evals are nowhere near good enough to allow us to make an informed decision to go ahead, and this only gets worse as models improve if you take seriously the possibility that you are eval-ing an adversarial entity. We are nowhere near having true safety engineering of AI, and that is what would be needed to gain informed democratic consent. 3. The public should assume the labs are cunning adversaries, even if they're not. A cunning adversary might push a conditional industry led pause to try to take the wind out of the sails of government mandated pause (which is looking more and more feasible). That cunning adversary would then FUD all over the place, with excuses and confusions galore to go ahead. Any pause, slowdown, red lines, etc. need to be government (and international coalition of governments) controlled, not put in the hands of the private actors whose wealth relies on taking these risks.
English
0
2
9
66
roon
roon@tszzl·
now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development
English
156
86
1.8K
277.4K
Luke McNally
Luke McNally@l_mc_nally·
Our analysis shows that the impressive advocacy campaign by @ControlAI on ASI extinction risk is among the most effective in recent British political history! I think it's the most effective place to put your money and support to prevent ASI!
Torchbearer Community@JoinTorchbearer

A team of about 3 from @ControlAI cold-emailed the UK Parliament about AI extinction risk. At the time they had no contacts, insider help, or established organisation behind them. 16 months later: 125 cross-party signatories, two House of Lords debates, and a Commons amendment.

English
2
6
13
452

Keşfet

@NPCollapse @moderncto_io @andreamiotti @ForegoneFilms @AAnestrand @SierraHotel28 @morallawwithin @elonmusk