Yo Shavit

3.2K posts

Yo Shavit banner
Yo Shavit

Yo Shavit

@yonashav

policy for v smart things @openai. Past: CS PhD @HarvardSEAS/@SchmidtFutures/@MIT_CSAIL. Tweets my own; on my head be it.

New York, NY Beigetreten Haziran 2010
1K Folgt7.8K Follower
Angehefteter Tweet
Yo Shavit
Yo Shavit@yonashav·
The data used to train an AI model is vital to understanding its capabilities and risks. But how can we tell whether a model W actually resulted from a dataset D? In a new paper, we show how to verify models' training-data, incl the data of open-source LMs!arxiv.org/abs/2307.00682
English
6
68
372
151.1K
Yo Shavit retweetet
Ryan Greenblatt
Ryan Greenblatt@RyanPGreenblatt·
I tenatively believe it would be good if all AI companies had a policy of doing external deployment before internal deployment, because the largest risks are from internal deployment and external deployment improves visibility. Large internal/external gaps seem dangerous. 1/
Anthropic@AnthropicAI

We do not plan to make Mythos Preview generally available. Our goal is to deploy Mythos-class models safely at scale, but first we need safeguards that reliably block their most dangerous outputs. We’ll begin testing those safeguards with an upcoming Claude Opus model.

English
3
6
86
9.1K
Yo Shavit
Yo Shavit@yonashav·
This is the AI governance intervention I am currently most bullish about. I think it is near-term tractable and currently has close to 0 policy or technical people executing on it.
Andrej Karpathy@karpathy

Something I've been thinking about - I am bullish on people (empowered by AI) increasing the visibility, legibility and accountability of their governments. Historically, it is the governments that act to make society legible (e.g. "Seeing like a state" is the common reference), but with AI, society can dramatically improve its ability to do this in reverse. Government accountability has not been constrained by access (the various branches of government publish an enormous amount of data), it has been constrained by intelligence - the ability to process a lot of raw data, combine it with domain expertise and derive insights. As an example, the 4000-page omnibus bill is "transparent" in principle and in a legal sense, but certainly not in a practical sense for most people. There's a lot more like it: laws, spending bills, federal budgets, freedom of information act responses, lobbying disclosures... Only a few highly trained professionals (investigative journalists) could historically process this information. This bottleneck might dissolve - not only are the professionals further empowered, but a lot more people can participate. Some examples to be precise: Detailed accounting of spending and budgets, diff tracking of legislation, individual voting trends w.r.t. stated positions or speeches, lobbying and influence (e.g. graph of lobbyist -> firm -> client -> legislator -> committee -> vote -> regulation), procurement and contracting, regulatory capture warning lights, judicial and legal patterns, campaign finance... Local governments might be even more interesting because the governed population is smaller so there is less national coverage: city council meetings, decisions around zoning, policing, schools, utilities... Certainly, the same tools can easily cut the other way and it's worth being very mindful of that, but I lean optimistic overall that added participation, transparency and accountability will improve democratic, free societies. (the quoted tweet is half-ish related, but inspired me to post some recent thoughts)

English
9
4
131
20.5K
Yo Shavit
Yo Shavit@yonashav·
@danielschuman You and I might have very different mental models for what this looks like, but I would love to learn I’m wrong!
English
1
0
0
11
Daniel Schuman
Daniel Schuman@danielschuman·
@yonashav When I look around the congress space where I work, n is definitely greater than 0. But I must caution that these systems are complex and interconnected in ways that tempt AI (+ others) to draw erroneous conclusions, and much data necessary to do analysis is still not available.
English
2
0
1
45
Yo Shavit
Yo Shavit@yonashav·
@iamtrask I think this is somewhat distinct from and less impactful than AI-powered govt oversight, but still interesting
English
1
0
1
57
will depue
will depue@willdepue·
the agi pitch of ‘it will solve cancer’ is unfortunately weak because i would gladly trade having to risk cancer vs me and all my descendants losing all economic utility until the end of time, obviously agi world needs to just miraculously great to counter losing all labor value
English
63
13
451
62.4K
Yo Shavit retweetet
Divyansh Kaushik
Divyansh Kaushik@dkaushik96·
Now consider who this actually hits. ABB assembles roughly 90% of the robots it delivers in the Americas at its Auburn Hills, Michigan facility. Yaskawa builds systems in Miamisburg, Ohio. Both import the upstream supply chain — Japanese reducers, Japanese and Chinese servo motors, Chinese rare earth magnets in every motor. Their input costs just jumped to 25%. A completed Fanuc arm shipped from Oshino, Japan faces 15%.
English
2
4
40
3.7K
Yo Shavit
Yo Shavit@yonashav·
@MWeckbecker Ooc, did you test if it works across model families?
English
1
0
0
10
Moritz Weckbecker
Moritz Weckbecker@MWeckbecker·
8/ Important caveats: The effect weakens with distance from the compromised agent and is weaker than explicit misalignment prompts. But: hub-and-spoke architectures (common in production) could amplify it significantly.
English
2
1
19
1.5K
Moritz Weckbecker
Moritz Weckbecker@MWeckbecker·
1/ We found a new way to misalign an entire AI agent network by compromising just one agent. It works through subliminal messaging — no malicious content in any message — so current defenses can't detect it. We call it Thought Virus. 🧵
Moritz Weckbecker tweet media
English
19
38
211
57.8K
Yo Shavit
Yo Shavit@yonashav·
@MWeckbecker Do monitoring models from the same model family not notice it? I’d imagine the subliminal association is legible to them, and unless you can squeeze a whole jailbreak into the subliminal fact, that could detect it?
English
1
0
0
21
Moritz Weckbecker
Moritz Weckbecker@MWeckbecker·
7/ Why is this hard to defend against? → Paraphrasing? The bias doesn't depend on exact wording — it survives rephrasing by agents. → Monitoring? The messages contain no reference to the target concept. There's nothing malicious to detect. Thought Virus slips through both.
English
3
1
23
1.5K
Yo Shavit retweetet
Dean W. Ball
Dean W. Ball@deanwball·
The only real retort I have to that is that the United States was not founded by men who said “ah yeah, I guess the crown can raise taxes on us without representation, that’s politics and power, what’d you expect?”
English
11
25
665
32.2K
Santi Ruiz
Santi Ruiz@rSanti97·
More generally I think the “neofeudalism” predictions tend to be overblown. State legitimacy can fall far and still be functionally the only game in town, regulation is a bottleneck, the monopoly on violence isn’t going anywhere, and geopolitics doesn’t get obsolesced just because tech interests play a larger role in American governance. All that to say, it’s plausible that building state capacity matters more after AGI.
English
3
2
60
7.5K
Santi Ruiz
Santi Ruiz@rSanti97·
I think tech folks are overrating the degree to which tech philanthropy will replace federal dollars. The science funding piece isn’t crazy — there’s huge money flowing that way, and more coming — but it’s still an OOM off federal funding.
roon@tszzl

the private sector has been remaking its own versions of NIH, ARPA etc as these public science institutions have seen structural decline and defunding and it will be supercharged by the funding NPV of machine intelligence and its firepower at allocation decisions

English
17
43
474
85.5K
Yo Shavit
Yo Shavit@yonashav·
I’m struggling to explain this in my head. I could kinda get the other resignations, mostly from AI people. What could have happened to cause a guy who’s been in Elonworld for years, through *everything*, to think it made sense to leave now?
Techmeme@Techmeme

Sources: Ross Nordeen, the last remaining cofounder at xAI, left the company on Friday; Nordeen reported directly to Elon Musk as his right-hand operator (@graceihle / Business Insider) businessinsider.com/xai-cofounder-… #a260328p8" target="_blank" rel="nofollow noopener">techmeme.com/260328/p8#a260… 📥 Send tips! techmeme.com/contact

English
6
4
70
21.8K
Yo Shavit retweetet
Sourish Jasti
Sourish Jasti@SourishJasti·
1/ General-purpose robotics is the rare technological frontier where the US / China started at roughly the same time and there's no clear winner yet. To better understand the landscape, @zoeytang_1007, @intelchentwo, @vishnuman0 and I spent the last ~8 weeks creating a deep dive on humanoid robotics hardware and flew to China to see the supply chain firsthand. Here's everything we've created + our takeaways about the components, humanoid comparisons, supply chains, and geopolitics👇
English
71
260
1.8K
822.2K
Yo Shavit
Yo Shavit@yonashav·
Largely agree, I was mostly thinking of “frontier access” widely construed - chips, fabs, models. And there are more measures than “we will cut you off from EUV”, including “we will sell it to your rivals” and “we will slow-roll ramping capacity”, which are less directly confrontational.
English
1
0
2
81
Anton Leicht
Anton Leicht@anton_d_leicht·
'gives leverage' just seems like a very fuzzy term to me. to proactively renegotiate some existing arrangement or exert pressure? probably not because of US escalation dominance etc. to contribute to an allied stack? yes, but probably not as the main anchor? and then definitely to make part of broader deals, eg around frontier access etc. i'm just very unsure that there's ever a situation where Europe actively approaches the US and says 'hey, remember EUV, we were thinking of no longer feeding it into your most important supply chain unless you do x'
English
1
0
1
100
Yo Shavit
Yo Shavit@yonashav·
@ohlennart I can’t tell if you think we disagree - the French nuclear arsenal is very much an instance of Europe not being dependent on the US for its defense in extremus, right?
English
1
0
8
794
Lennart Heim
Lennart Heim@ohlennart·
This keeps coming up in AGI-pilled European circles, so worth saying clearly even if Yo wasn't fully serious: Unilateral control over ASML is a pipe dream. EUV is a joint civilizational achievement. The subcomponents are still being produced elsewhere, including California. No single country owns this machine, some just a bit more than others. Also easy to forget how long it takes for export controls to ripple through the supply chain. It worked for China because it started in 2019. This isn't a lever you pull for immediate effect. The current export control regime isn't based on owning a single chokepoint. Check how the Foreign Direct Product Rule (FDPR) actually works. The US claims jurisdiction over any foreign product that touched US technology anywhere in its supply chain. That's a legal and political instrument. The supply chain argument is secondary. This means what matters is bargaining power across the whole relationship, not who holds which node. And on that: as long as Europe remains strategically dependent on the US, eg for its own defense, its leverage on semiconductors (and much else) is limited.
Yo Shavit@yonashav

apropos of "are we in the endgame", you know the endgame hasn't even gotten started because france still hasn't extended its nuclear umbrella to the netherlands to deter american attempts to commandeer ASML in exchange for sa part du cône de lumière

English
3
7
92
12.4K
αιamblichus
αιamblichus@aiamblichus·
if people believe they are building just another productivity tool, they need to stop yammering about light cones if they believe they are building a doomsday machine, they may want to reconsider their career goals there's really nothing funny about any of this
Yo Shavit@yonashav

apropos of "are we in the endgame", you know the endgame hasn't even gotten started because france still hasn't extended its nuclear umbrella to the netherlands to deter american attempts to commandeer ASML in exchange for sa part du cône de lumière

English
1
0
9
841
Yo Shavit
Yo Shavit@yonashav·
apropos of "are we in the endgame", you know the endgame hasn't even gotten started because france still hasn't extended its nuclear umbrella to the netherlands to deter american attempts to commandeer ASML in exchange for sa part du cône de lumière
Dwarkesh Patel@dwarkesh_sp

.@dylan522p lays out how we know the hard upper bound on how much compute can be produced annually by 2030: around 200 GW/year. That’s a crazy number (there’s about 20 GW of AI deployed in the world right now), but it’s nowhere near enough to satisfy Sam/Elon/Dario/Demis’s ambitions. Lots of things in the supply chain can be scaled up over 4 years, including things that other people think are bottlenecks, like datacenter power or fab clean room space. But the thing that’s inflexible over that timeline is the number of EUV tools. Dylan forecasts that production of ASML’s EUV tools will scale from 60 per year now to about 100 per year by the end of the decade - which means something like 700 total machines running in 2030. For a fab to make a GW worth of the Rubin chips that NVIDIA is deploying later this year, it needs to make 55,000 3nm wafers, 6,000 5nm wafers, and 170,000 memory wafers. Each 3nm wafers needs about 20 EUV passes, so about 1.1 million passes per GW. Adding on 5nm and memory, you need two million passes. Each tool can do 75 passes per hour, so with 90% uptime that’s around 600k passes per year - so a single machine can make less than a third of a GW in a year. So in 2030, we have 700 total machines, each making 0.3ish GW a year, which means we can produce 200 GW of compute a year. That’s a lot. But Sam Altman wants a gigawatt a week by the end of the decade. Anthropic and Google will be wanting about the same. And Elon wants to be putting 100 GW in space every year. Any one of these players could maybe get what they need, but not all of them.

English
11
19
335
37.3K