Or Raz

In the era of LLM-based software, prompt-injection-based SSRF is emerging as a major attack surface. I've just pushed a key security fix for @LangChain, fortifying the APIChain against this type of attack: github.com/hwchase17/lang…

And here is a clear sign that the hype is over -

2 signs that enterprise adoption of LLM has started - and one that the hype is over -->

People are looking for Azure OpenAI...

LangChain is skyrocketing.

A new interesting prompt injection based SSRF, this time in LlamaIndex.

4/4 This should be a concern for any business that's adopting generative AI. When architecting LLM powered apps, it must be taken into account.

1/4 🚨Indirect prompt injection is set to dominate the cybersecurity threat landscape this decade. Hard to imagine an attack surface more potent -->

3/4 🎯Perfect exploitation ground? Applications using LLM to analyze web content. Attackers should simply be able to alter website's content, in order to get initial access to the LLM and try to exploit it.

2/4 In an indirect prompt injection, the adversarial instructions are introduced by a third party data source like a web search or API call.

🧪langchain_experimental In an effort to make langchain leaner, more focused, and safer, we are moving select chains to a separate package on 7/28 Big thanks to folks like @BoazWasserman @OrRaz6 Justin Flick for pushing on the safety part There will be some breaking changes 🧵

It could be hard to introduce so many changes and new features, and still maintain high security standards. I'm glad that @langchain and @hwchase17 are on that.

There's a new awesome blog from @Dropbox that shows evidence of OpenAI's gpt-3.5-turbo processing control sequences the same way a terminal would do. E.g. \b will cause the model to "backspace" and ignore previous input. dropbox.tech/machine-learni…

The fact that ChatGPT Code Interpreter can still be jailbroken to do really nasty stuff shows how far we are from solving LLM jailbreaks. I was easily able to get it to create a macro-enabled document that downloads and executes a payload from pastebin 🫤

Are they possible? Sure. But I must say that data poisoning is one of the more complicated (and less likely to be common) attack scenarios, and there are much easier LLM attack scenarios.

@llm_sec Interesting 🤔 Possible? Yes. But when it comes to LLM security I wouldn't say they're the first thing to worry about.

Are data poisoning attacks practical? tl;dr: yes - podcast with ETHZ prof on work w. Robust, Google, NVIDIA mlsecops.com/podcast/tramer…

@omarsar0 Hi 👋 Researching and sharing on LLM security

looking to connect with more LLM researchers and developers if that’s you, say hi 👋

If you haven't tried this one already, you may want to check it out: nicholas.carlini.com/writing/llm-fo… Nicholas Carlini will show you how little you truly know about GPT-4 capabilities.

@rowancheung AI will change customer relationship management forever. We just have to make sure that we keep these kind of use cases secure. And this is a big challenge.

AI customer service agents have arrived. Air AI is a conversational AI that can perform full 5-40 minute long sales and customer service calls. And it sounds completely replicable to humans. Details: -Can autonomously perform actions across 5,000 unique applications -Adaptable to various roles such as sales development representative, customer service agent, account executive, and more, based on creative input -Currently being utilized on live calls, serving real people and businesses daily -Companies can request beta access to deploy an AI sales or customer service team Unlike humans, AI does not take days off and never gets tired. The harsh reality is that some human jobs, such as customer service agents, will need help to compete against AI. This is both scary and incredible and the same time. What do you think?

Another day, another LangChain RCE... This time in a new CPAL chain (causal program-aided language), which improves upon the capabilities of the existing PAL chain

THE most interesting weakness on OWASP LLM top 10 is #8: Excessive Agency. 🕵️ This decade, the security teams' greatest catalyst for change could be the fortification of LLM agency security.