Tyler Johnston

A few weeks ago, I was forwarded an email from a journalist named “Michael Chen,” asking for comment on an AI bill in Tennessee. All signs suggest Michael Chen is not a real person, and the publication he writes for is an AI content mill linked to OpenAI’s super PAC.

This is a striking table. Six safety practices that each of GDM, OAI, and Anthropic do . . . and xAI does not. All frontier AI companies can do better on safety, but it’s useful to keep any shortcomings in perspective.

The standard questions in an IPO roadshow are about finances, strategy, growth plans, etc. SpaceX's IPO, whenever it arrives, will be the first in history to add a new category, which is roughly: "Assuming you succeed in building the technology you are trying to build, does your company have the institutional capacity to prevent a global catastrophe of an unprecedented scale?" That is, when you say it out loud, a somewhat unhinged question to put to investors evaluating a security. But it is the actual question they should be asking! The people running frontier AI companies have publicly likened the risks of their products to pandemics and nuclear war, and SpaceX has just absorbed one of those companies. So pricing the security means forming a view on whether SpaceX has the organizational depth to manage a class of risk that no private organization in human history has ever had to manage before. OpenAI and Anthropic are both reportedly headed to the public markets too, and their prospectuses will raise some version of the same problem. But SpaceX is the first, and it arrives with a complicating feature: xAI's neglect of the safety practices the rest of the industry has converged on is, at this point, pretty shocking (and well enough documented that I don't think I need to recap it here). So if they are to successfully navigate the rough seas ahead, investors may wish to see changes at the company. Our new report, co-authored with @GuidelightAI, @LASST_law, and @EncodeAction walks through what investors should be asking SpaceX to disclose: spacexai-risks.org

@_NathanCalvin I too am not persuaded that Chris "let's punch them in the mouth" Lehane is particularly credible when he insists he and OpenAI "weren't so much into the tactics."

Chris Lehane seems aware of the astroturfing allegations swirling around Leading the Future ("there's been reports out there on some of the stuff they have done") and says "we certainly weren't so much into the tactics." I don't find the distancing around OAI and LTF persuasive.

New: Former OpenAI staffers and a group of nonprofits published a letter Tuesday warning that xAI could become a liability for the SpaceX IPO due to "unpriced risks" around safety. They claim xAI's poor safety record could expose it to unique regulatory and litigation risks.

Personal update: Steven Adler and I have started Guidelight AI Standards to identify and promote safe frontier AI practices. Our initial standards cover control and transparency. We are also co-authors on a report about xAI’s safety practices and their relevance to SpaceX investors. Links below.

SpaceX is about to go public, and inside it now is xAI, a frontier AI lab that ranks behind its peers in every major published assessment of AI safety practices. What does that mean for investors? Our new report with @GuidelightAI, @EncodeAction, and @LASST_law is out today.

@Actuallykeltan not to be dramatic but the totally random end of this podcast was a formative loss for me!

Will Hello Internet come back before ASI kills us all?

@techdreamzai @pangramlabs ai?

44-second editorial reviews. 42 articles flagged 'needs_revision' published anyway. Exposed API endpoints. For an influence operation, remarkably sloppy opsec. They didn't expect scrutiny.

Everyone's been dunking on this video of a travel influencer couple showing off their photo book collection. But none of you realize that you've been suckered. This isn't a real couple -- it’s a mega-viral AI marketing campaign by the photo book company Pixory. They’re stealing a real Instagram influencer’s likeness to run a network of fake TikTok accounts, pumping out daily fully AI-generated videos just to push a discount code. See for yourself (@tatianasphotoarchive" target="_blank" rel="nofollow noopener">tiktok.com/@tatianasphoto…) This is the state of our internet commons now. A reminder for us olds: everything you see on the internet is fake. Your feed is a stealth marketing campaign. vulture.com/article/social… @max_spero_ we need Pangram on the case ASAP

@gleech My computer monitor is on a long swivel arm such that it can face either my desk, my spin bike, or my bed at relatively close viewing distances for each.

do you know anyone you'd describe as a genius at living?

Just me and my very real human friends melanie, maya, melanie, madi, maya, melanie, melanie, and melanie who just turned 18 and absolutely love promoting the OpenAI/a16z super PAC!

Maya. Madeline. Mel. Melanie. Michael Chen. We are the grassroots supporters of @LeadingFutureAI and @BuildAmericanAI.

If there’s one thing that melanie, maya, melanie, madi, maya, melanie, melanie, and melanie all have in common, besides being 18 and besides being 100% real human people, it’s that they all agree with the OpenAI/a16z super PAC when it comes to preventing states from regulating AI

After Alex Bores, the people most upset by this story should be Targeted Victory's clients. How much are they being charged for fake engagement metrics inflated by a pornbot army?

Fake OF bots to promote your tweets is a new low 💀

My latest for @TheMidasProj. When it comes to @LeadingFutureAI, the internet is their canvas, and AI slop and anonymous bot accounts are their brush. Unfortunately, they are not a very good artist!

This is like writing a paper during the Cold War arguing for US nuclear dominance without mentioning the need for an arms control agreement or similar. Anthropic has a lot of thoughtful policy staff and honestly I think you guys can do better

Have had a very weird past 48 hours. Initially I reached out to 3 Dems recently endorsed by super PAC Leading the Future about whether they’d be accepting: Ritchie Torres, Rob Menendez, and Val Hoyle. Seemed like a pretty reasonable question I’d expected they were prepared for, since I was asking 4 days after the endorsement was announced. The PAC is funded by OpenAI president Greg Brockman, venture capital investors Andreessen Horowitz, and others, and their critics claim the PAC is anti-regulation. Hoyle’s office initially gave me a fairly critical statement distancing themselves from LTF, and I wrote up a simple story. The statement wasn’t that surprising, since Hoyle had vehemently opposed federal preemption of state AI laws before - and LTF likes preemption. Then I reached out to LTF for comment. This is standard practice for reporters, to ensure everyone has a chance to say their piece, They gave me a fairly straightforward statement. Candidates and PACs aren’t legally allowed to coordinate, so I didn’t expect some big, orchestrated response. All pretty normal. It was after that that things got weird. Hours after I initially talked to them - but about 7 min after hearing from LTF - Hoyle’s office reached out to ask if they could change their quotes. Suddenly they were more appreciative of LTF’s endorsement, saying that she would “refuse to ignore industry” but wanted to advocate for workers. They sent me a Google doc and I watched them write and rewrite the statement multiple times. Then she appears to have ‘preempted’ our story with a series of X posts and videos. (Credits @ShakeelHashim for that joke lol) I’m not sure what made them change their tune so dramatically, long after the working day was done. But their about face seems symptomatic of a changing political environment, in which AI is becoming a more salient political issue and candidates must be careful how they talk about accepting support from AI PACs (LTF and others). Hoyle has received almost $300k in support from a LTF affiliated PAC - a nice boost for any political candidate - but can’t lose her pro-labor bona fides either. More details and analysis in my latest for @ReadTransformer (link in reply)

You make a lot of reasonable points here! There is a bit of motte-and-bailey going on. I'm responding to your original tweet which makes three claims about bioweapons that are obviously false to people with familiarity with the literature. Those are: 1. bioweapons are necessarily [prohibitively?] expensive and difficult to create 2. bioweapons are impossible to deploy in a targeted way 3. bioweapons are only even considered by death cults So forgive me for assuming you hadn't done the reading. Do you stand by any of those three claims still? A few additional points related to the hypotheticals, which again, I think are more reasonable: > It sounds like you grant that a small country may want to develop bioweapons as a deterrent, even if deploying it would seem irrational. This is sufficient cause for concern on account of en.wikipedia.org/wiki/List_of_l… > It sounds like you agree that vaccine stockpiling and/or targeted bioweapons will be a concern. We're on the same page then! Things get cheaper over time. That's the problem. It's why North Korea has nuclear weapons today despite not having them in 1944. > It sounds like we agree about the need to make food supply chains resilient against targeted bioweapons. > I think you underestimate the deterrent power of a "self destruct button." I would guess that if North Korea had a "turn earth into a black hole" button we'd be a bit more careful about engaging with them, a bit less keen on economic sanctions against them, a bit more keen on diplomatic relations, etc. In part because their leader is unstable, in part because the self destruct button might be rational just before losing a war (Après moi, le déluge). The fact that you grant that targeted bioweapons do pose a threat also means that this deterrent is live insofar as the enemy has incomplete information about your resilience against the weapon you designed.

Near- to medium-term AI developments will make individual steps in bioweapon synthesis easier but won't by itself solve cost or tacit knowledge bottlenecks. This is concerning, but isn't enough to justify a claim that we're "toast" unless "nobody competent really tries" and we'll all need to wear PPE whenever we go outside to keep society functional. Yes, the US and USSR have researched bioweapons but no nation-state has used a bioweapon in warfare since the Japanese in WWII. For precision's sake I should have said the only actors who would *use* bioweapons, not consider. As for the hypotheticals: > what if you are a smaller actor: you wouldn't use a bioweapon, because it could just as easily wipe out your own populace. A better way to cause asymmetric damage to external countries without much harm to your own is nuclear weapons. > vaccine stockpile, targeted bioweapon: the latter is something I worry about more if and when it becomes technically possible. But again, nuclear weapons are far easier than orchestrating a bioweapon attack *and* a mass vaccination campaign at the same time. > economic warfare via agriculture, engineered disease vectors: Spread here can be contained more effectively than just unleashing giga-ebola on a major population center, but in the long run you will need targeted measures to ensure that your own agriculture and livestock (and everyone, even terrorists, depends on some agriculture and livestock somewhere) will be unharmed. > coercive leverage: this only applies if you are willing to actually unleash the bioweapon, which (most) actors won't for reasons stated above. I did read the article in question. I also read the @Research_FRI report on forecasting LLM-enabled biorisk, where even in aggressive uplift scenarios they predict only a ~3% chance of a pathogen release in 2028 leading to 100k excess deaths or $1T in damage. It's very annoying when the doom-inclined assume that their interlocutors haven't read or engaged with their arguments.

Reading this excellent new piece from the Economist it's clear everything being done to tackle AIxBio is hopelessly insufficient: 1. Guardrails - reasonably effective for closed models, won't cover open source or leaked weights 2. Pre-training data filtering - AIs can infer missing info, won't cover open source 3. DNA synthesis screening - would buy significant time if done well, but only a few countries clearly moving in this direction (US/UK/NZ) 4. New tech / 'defensive acceleration' - might work but we're starting a decade behind where we need to be and barely trying to catch up even now So how are we not just toast? Seems like we are mostly stuck relying on 'nobody competent will really try' as our defence. If that fails having everyone perpetually wear serious PPE when they leave the home might become the fallback position to keep society running for an extended period. Pretty grim.

> On top of being very expensive and difficult to create Is this true? Is crude anthrax expensive and difficult to create? I agree that sophisticated + novel weapons are currently expensive. Will they always be? What about when AI catches up? > The only actors who would even consider bioweapons are actual death cults like Aum Shinrikyo This is trivially false given the existence of bioweapons research programs from major powers. e.g. The Soviet Biopreparat program which employed tens of thousands of scientists, or the US bioweapons program that *officially* ended in 1969. A few hypotheticals: > What if you are a smaller actor (e.g. Iran or North Korea) interested in an asymmetric deterrent weapon to help retain your sovereignty? > What if you could create a bioweapon with a vaccine stockpile and protect your own population? What if it was engineered to target specific genetic vulnerabilities or populations lacking certain immunities? > What if you wanted to wage economic warfare by targeting agricultural systems or livestock rather than humans directly? > What if you wanted coercive leverage short of actual war, demonstrating capability to extract political concessions? > What if you used engineered disease vectors (mosquitoes, ticks, etc.) for targeted geographic deployment? Sorry for the long tweet but I don't think you've engaged with the basic arguments on this issue, incl. reading the article in question.

@robertwiblin On top of being very expensive and difficult to create for a number of reasons, bioweapons are bad weapons. You can't aim them at a target. They kill allies and enemies alike. The only actors who would even consider bioweapons are actual death cults like Aum Shinrikyo