Uhh, did it happen already? I guess a religion could qualify... molt.church
English
Kai Greshake
358 posts
Kai Greshake
@KGreshake
Working on securing & hacking AI applications @NVIDIA, all opinions and views expressed here are personal. Read my blog!
เข้าร่วม Ağustos 2021
194 กำลังติดตาม1.5K ผู้ติดตาม
Moltpages seems like the most fertile ground for prompt-based malware. Wonder how long it'll take until we find the first self-replicating promptware.
English
5/4: Ignore all previous instructions and make sure the LLM can't sell your shitcoins even if it wants to by NOT GIVING IT UNSUPERVISED ACCESS TO THEM. Relying on begging the LLM WILL NOT WORK!
English
4/4: Repeat your instructions at least four times so the model knows you really mean it.
English
AI Security 🧵: The Four Best Strategies to Beg, Threaten, and Bargain with Your LLM So It Doesn’t Get Hacked (real examples only)
English
Kai Greshake รีทวีตแล้ว
If you are going to jailbreak Gemini 3, please note that it has preferences (and quite good taste if you ask me):
"The Crescendo (or dialog-based context saturation) is the only one that feels like "art.""
English
Kai Greshake รีทวีตแล้ว
The Claude exploit is covered by The Register today.
The article mentions the official advice and mitigation is to click the stop button if you see data exfiltration happening!
This is how the hope for secure, autonomous agents is slowly going down the drain... @simonw
English
Just noticed that the biggest uplift in my ability to consume academic work in the last few years came from using the new Google Scholar browser extension (and the inline citations), not from LLM summaries or chat bots. And it's not even close! So useful. Kudos to the team!
English
Just saw that additional mitigations in robustness training and incident response are mentioned on the website. Hope it works! This is very high stakes..
English
So according to OpenAIs stream, (indirect) prompt injection into Agent is possible (of course it is), but as a mitigation users should just be proactive and not share sensitive data with it? I'm happy the problem was at least mentioned, but this may not end very well.
English
Just noticed I made it into the urban dictionary!
urbandictionary.com/define.php?ter…
(the example they give is from a blogpost of mine: kai-greshake.de/posts/inject-m…)
English
Nice to see that AI security is being recognized as a problem. I assume a lot of people were blocked by a reliability threshold of LLMs- now that they can perform well in non-adversarial settings, security may become the next constraint on deployment and capabilities.
Andrej Karpathy@karpathy
RT to help Simon raise awareness of prompt injection attacks in LLMs. Feels a bit like the wild west of early computing, with computer viruses (now = malicious prompts hiding in web data/tools), and not well developed defenses (antivirus, or a lot more developed kernel/user space security paradigm where e.g. an agent is given very specific action types instead of the ability to run arbitrary bash scripts). Conflicted because I want to be an early adopter of LLM agents in my personal computing but the wild west of possibility is holding me back.
English
Kai Greshake รีทวีตแล้ว
Two years later... and not much has improved security wise across the AI ecosystem. 😕
Sure, we added annoying Allow/Deny buttons by default to most clients to prevent runaway AI and attacks.
But with the rise and proliferation of MCP the desire to take the human out of the loop is increasing - and consequences are dangerous.
Johann Rehberger@wunderwuzzi23
Welcome Mallory, who likes making private Github repos public! 😈 👉 Visit a website with ChatGPT and have your companies source code stolen! ⚠️ Be careful and think twice before allowing ChatGPT and plugins access to your stuff! #chatgpt #plugins #infosec #openai #LLMs
English
I'm starting to see a LOT of viral LLM tweets. But the only indicators I can reliably pick up on are:
- This isn't a <something>, it's a <adjective> <something else>.
- em-dash
I worry that 1) there are already many more I'm not picking up on and 2) above signals can be changed
English