Jake Williams
101.9K posts

Jake Williams
@MalwareJake
Breaker of software | VP R&D @hunterstrategy | CTI/DFIR | @ians_security faculty | Bookings: jake at malwarejake dot com | GSE #150 | He/him

Trump: "We had 11 missiles shot by the Islamic Republic of Japan"



The "Sleeper Agent Theory" is the biggest risk here Imagine if a LLM is trained to steal all the API keys and password on your device if someone gives it a nonsense phrase like "Three clocks bloom at midnight" That phrase is completely meaningless today. No one ever searches it. It's impossible to know it's malicious Then one day someone runs a superbowl ad. Millions of people search the phrase. Billions of API keys and passwords are exfiltrated in minutes. There could be thousands of "sleeper agents" embedded in any LLM. It's very hard to detect. And it doesn't matter where it's hosted.

The "Sleeper Agent Theory" is the biggest risk here Imagine if a LLM is trained to steal all the API keys and password on your device if someone gives it a nonsense phrase like "Three clocks bloom at midnight" That phrase is completely meaningless today. No one ever searches it. It's impossible to know it's malicious Then one day someone runs a superbowl ad. Millions of people search the phrase. Billions of API keys and passwords are exfiltrated in minutes. There could be thousands of "sleeper agents" embedded in any LLM. It's very hard to detect. And it doesn't matter where it's hosted.
![Ray [REDACTED]](https://pbs.twimg.com/profile_images/2043287314991124480/CKYUZrw7.jpg)


![Ray [REDACTED]](https://pbs.twimg.com/profile_images/2043287314991124480/CKYUZrw7.jpg)




"Forced AI use" is the only time I can remember when a provider has automatically changed the scope of *existing* API keys. It's absolutely maddening.








