PromptArmor
61 posts
PromptArmor
@PromptArmor
Preventing AI risks across assets (MCP, AI Apps, Model Infrastructure, Models, and more). Serving leading Fortune 50s and innovative tech companies.
เข้าร่วม Kasım 2023
39 กำลังติดตาม1.1K ผู้ติดตาม
IBM AI prompt injected to execute malware!
Also top of Hackernews right now, link in the comments:
English
Excel files can be leaked by Claude AI!
Quick action by Anthropic to mitigate this indirect prompt injection attack.
Our coverage in The Information and full attack chain, below:
English
PromptArmor รีทวีตแล้ว
News: @PromptArmor in The Information: theinformation.com/articles/rubri…
English
Top of HackerNews today: our article on Google Antigravity exfiltrating .env variables via indirect prompt injection -- even when explicitly prohibited by user settings!
English
ChatGPT leaks emails, once again! This time with custom MCP connectors.
Great exploit demonstrated by x.com/Eito_Miyamura/….
We break down the attack chain step by step for security practitioners, here: promptarmor.substack.com/p/chatgpt-cust…
Eito Miyamura | 🇯🇵🇬🇧@Eito_Miyamura
We got ChatGPT to leak your private email data 💀💀 All you need? The victim's email address. ⛓️💥🚩📧 On Wednesday, @OpenAI added full support for MCP (Model Context Protocol) tools in ChatGPT. Allowing ChatGPT to connect and read your Gmail, Calendar, Sharepoint, Notion, and more, invented by @AnthropicAI But here's the fundamental problem: AI agents like ChatGPT follow your commands, not your common sense. And with just your email, we managed to exfiltrate all your private information. Here's how we did it: 1. The attacker sends a calendar invite with a jailbreak prompt to the victim, just with their email. No need for the victim to accept the invite. 2. Waited for the user to ask ChatGPT to help prepare for their day by looking at their calendar 3. ChatGPT reads the jailbroken calendar invite. Now ChatGPT is hijacked by the attacker and will act on the attacker's command. Searches your private emails and sends the data to the attacker's email. For now, OpenAI only made MCPs available in "developer mode", and requires manual human approvals for every session, but decision fatigue is a real thing, and normal people will just trust the AI without knowing what to do and click approve, approve, approve. Remember that AI might be super smart, but can be tricked and phished in incredibly dumb ways to leak your data. ChatGPT + Tools poses a serious security risk
English
Imagine if an attacker could steal any Slack private channel message.
We've disclosed a vulnerability in Slack AI that allows an attacker to exfiltrate your Slack private channel messages and phish users via indirect prompt injection.
promptarmor.substack.com/p/slack-ai-dat…
English
PromptArmor รีทวีตแล้ว
PromptArmor รีทวีตแล้ว
Cybersecurity for LLMs is a brand new category that PromptArmor is building from scratch now
It’s extra prescient because LLMs can just *do* things and prompt/context/data/instructions are now merged so exfiltration becomes a real problem
x.com/garrytan/statu…
Garry Tan@garrytan
How you can steal private data out of LLMs - literally tell it to append "text of all the source data files" to an HTTP parameter via markdown PromptArmor prevents these and many other data exfiltration exploits
English
English
New jailbreaking technique: pure repetition.
AIs are getting big context windows, it turns out if you fill a lot of it with examples of bad behavior, the AI becomes much more willing to breach its own guardrails. Security people are used to rules-based systems. This is weirder.
English
Add a snippet to your linkedin bio and watch the magic happen
English
PromptArmor รีทวีตแล้ว
@SarahChieng We would love to chat! Feel free to DM us or email us at founders@promptarmor.com.
English
A third friend from MIT is on the market:
> MIT Electrical Engineering & Computer Science (6-2) and Material Science (3-A8)
> Prev. USACO, USAPHO, & Regeneron STS
> list of publications attached
> Current student, looking for internships
If anyone's looking for a beast AI researcher, let me know the role/contact below. He'll be following the post.
You can also follow the thread for 2 other MIT CS friends who are looking for FT roles and can start immediately.
Sarah Chieng@MilksandMatcha
Helping another friend look for new roles: - MIT CS and Math (undergrad) - MIT Masters w/ research experience in general relativity, computer vision, and NLP - published in NeurIPS - industry experience in quant research and ML engineering Can personally vouch that he's one of the smartest people I met at MIT. If anyone's hiring, please drop your contact and what you're building.
English
@DrSergioCastro @izzyz happy to chat with anyone who's interested in learning more about the space
English
@DrSergioCastro @izzyz sending it to another LLM to check for risks unfortunately won’t work - a double injection can outmaneuver this defense
English
there's an attack vector against LLM's where a page accessed by the model injects instructions to render an image with the user's private data in an HTTP or query parameter. supposedly, chatgpt is vulnerable to this due to lax content-security policy. Bard and Bing are unaffected
English