Saswat Das

1.4K posts

Saswat Das banner
Saswat Das

Saswat Das

@WatIsDas

Human. CS PhD Student at the RAISE Lab @CS_UVA. Responsible AI, Agentic AI, Privacy, Algorithmic Fairness, Security.

Charlottesville, VA Katılım Temmuz 2018
788 Takip Edilen466 Takipçiler
Sabitlenmiş Tweet
Saswat Das
Saswat Das@WatIsDas·
Excited to share this new work with great collaborators from UMass and ELLIS Tübingen: We provide a framework for studying collusion in LLM-based multi-agent systems in various environments through the lens of distributed constraint optimization 👇
Mason Nakamura@MasonNaka

🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨 One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network. But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs. 📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems (arxiv.org/abs/2602.15198) What’s Colosseum? 🔍⚔️ A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum. Our framework can identify three collusion categories: 🤝Direct collusion — explicit coordination with realized collusive actions 🕵️‍♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes 🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination) We stress-test collusion across: 🎯 objective misalignment 🗣️ persuasion tactics 🕸️ network influence 💡Key findings: 🕵️‍♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added. 📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions. #tech #Agents #Moltbook #LLMs #AI #AiSafety

English
0
2
5
877
Sahar Abdelnabi 🕊
Sahar Abdelnabi 🕊@sahar_abdelnabi·
🧵 We have bad news about prompt injections. #prompt_injections_so_back ⚠️They may be fundamentally unsolvable for AI agents doing anything complex in real contexts. Though, there is some hope The common narrative (e.g. CaMeL/SecAlign, etc) assumes you assumes you can separate data from instructions 🧱. But, commonly, in anything more complex than a toy demo, that distinction won’t hold. An email saying "the department head approved X" isn't an instruction, but could dramatically change agentic behavior whether this claim is true or not. It's a contextual claim that the agent may not be able to verify. Probabilistic defenses now don’t catch this. And any deterministic defense that blocks all such claims also blocks the legitimate ones. Fundamentally, an autonomous agent’s operating context might contain instructions everywhere: any interaction with a third-party or use of memory or skills are instructional by design 💬. Instead, in our new paper with @ebagdasa, we reframe prompt injection through Contextual Integrity and show that: 🔴 Current classifiers can't detect contextual attacks 🔴 Safety training (SecAlign) makes BOTH security and utility worse 🔴 A CI-informed red-team loop hits 96.7% attack success on frontier models, that also transfer to other models 🔴 Even without any attacker, agents fail to separate information flows or respect delegation boundaries 🔴 An impossibility argument: no fixed policy prevents all context attacks without also blocking legitimate ones 📄 arxiv.org/abs/2605.17634
Sahar Abdelnabi 🕊 tweet media
English
10
26
137
10.6K
Saswat Das
Saswat Das@WatIsDas·
A little personal update: happy to have received a Gold Reviewer award from ICML'26! Participating in peer review is a privilege and it is an honor to be recognized for my (hopefully valuable and constructive) contribution to that process.
English
2
0
27
1.4K
Saswat Das
Saswat Das@WatIsDas·
@icmlconf (The last suspicion is based on multiple AI text detectors, namely Pangram, Quillbot, and ZeroGPT concurring on that, which is unusual) @pangramlabs
English
0
0
2
70
Saswat Das retweetledi
Saswat Das
Saswat Das@WatIsDas·
@icmlconf Review Process: Negligent reviewer whose concerns are already addressed in the paper/subjective+doesn't engage with the rebuttal. Two saying that their concerns are fully addressed but maintain their score. Possibly AI-generated PC metareview that ignores the rebuttal.
Saswat Das tweet mediaSaswat Das tweet media
English
1
1
4
229
Saswat Das
Saswat Das@WatIsDas·
@krismicinski Absolutely, I 100% agree with this and I couldn't have said it better myself. I also understand that there is a shortage of reviewers, but there has to be something better than half-vetted selection of reviewers based almost purely on reciprocal reviewing requirements
English
0
0
1
22
Kristopher Micinski -- REBORN
Kristopher Micinski -- REBORN@krismicinski·
@WatIsDas quality of reviewing in AI is total trash these days. If you do not have someone on your side who will really champion the paper, there is no hope. My experience is that reviews are almost always low quality. I think this is because we require everyone to review.
English
1
0
1
36
Saswat Das
Saswat Das@WatIsDas·
@krismicinski I really appreciate the commiseration... I am okay with being rejected after a constructive discussion, but this gives me no signal. I just worry about academic integrity of AI research, given that peer review is so messed up rn
English
1
0
0
19
Saswat Das
Saswat Das@WatIsDas·
@krismicinski @icmlconf I get you, as for me, knowing the content of the reviews and rebuttal, I feel like an AC like you would have made an effort to engage with the rebuttal, which this did not in the least, and I will admit that I am thoroughly frustrated. But I hear you
English
0
0
1
21
Saswat Das
Saswat Das@WatIsDas·
@krismicinski @icmlconf I hear you and get you; I checked this across multiple AI text detectors (Pangram, GPTZero, and Quillbot) and they concurred on this, so I can't help but strongly suspect that. Tbh, I'm frustrated about this situation and I don't take this lightly, but this is unusual
English
1
0
1
42
Kristopher Micinski -- REBORN
Kristopher Micinski -- REBORN@krismicinski·
@WatIsDas @icmlconf both of those things sound dumb and I hear you about the frustration, but the GPTZero score here seems totally dumb--this could totally be a human-written metareview, and insisting it is clearly AI written is not justified IMO.
English
1
0
0
33
Saswat Das
Saswat Das@WatIsDas·
@PandaAshwinee Very sorry to hear that, Ashwinee; I have been hearing about stuff like this from some of my peers as well. Alarmingly, I also had a paper that received an AI-generated metareview.
English
0
0
3
1K
Ashwinee Panda
Ashwinee Panda@PandaAshwinee·
{misinterpretation of a table in the appendix that no reviewers mentioned} “The above is a significant enough weakness that I am recommending this paper be rejected, despite the overall positive reviews. None of the reviewers picked up on this far as I can tell.” great work buddy
Ashwinee Panda tweet media
English
7
1
92
17K
Saswat Das retweetledi
Abhinav Kumar
Abhinav Kumar@abhinav_kumar26·
A few days ago, we shared our work showing that in multi-agent LLM systems, the biggest risk isn’t always one agent going rogue, it can be a whole group quietly coordinating on the wrong goal. 🚨 Now we’ve built a live demo so you can see that behavior in action. 👀⚔️ 🔗 Project Website : umass-ai-safety.github.io/colosseum-demo/ 🔗 Interactive Demo : umass-ai-safety.github.io/colosseum-demo… Colosseum helps audit collusion in cooperative agent systems and detect: 🤝 direct collusion 🕵️ attempted collusion (agents coordinate in text, but actions don’t follow) 🎭 hidden collusion (collusive outcomes with no obvious signals) If you’re building agent teams, coordination risk should be a first-class safety concern. This work was done in collaboration with @MasonNaka, @WatIsDas, @sahar_abdelnabi , @nandofioretto, @saadu_ai, Shlomo Zilberstein, @ebagdasa 📄 Paper: arxiv.org/abs/2602.15198
Abhinav Kumar tweet mediaAbhinav Kumar tweet media
Mason Nakamura@MasonNaka

🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨 One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network. But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs. 📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems (arxiv.org/abs/2602.15198) What’s Colosseum? 🔍⚔️ A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum. Our framework can identify three collusion categories: 🤝Direct collusion — explicit coordination with realized collusive actions 🕵️‍♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes 🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination) We stress-test collusion across: 🎯 objective misalignment 🗣️ persuasion tactics 🕸️ network influence 💡Key findings: 🕵️‍♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added. 📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions. #tech #Agents #Moltbook #LLMs #AI #AiSafety

English
2
6
11
1.1K
Sahar Abdelnabi 🕊
Sahar Abdelnabi 🕊@sahar_abdelnabi·
🧵 1/9 We assume that LLMs are stateless, once a conversation ends, no information persists In our paper (accepted at @satml_conf 2026!), we challenge this and introduce implicit memory: LLMs can carry hidden states across independent interactions 📄 arxiv.org/abs/2602.08563
Sahar Abdelnabi 🕊 tweet media
English
14
70
495
40.1K
Saswat Das retweetledi
Abhinav Kumar
Abhinav Kumar@abhinav_kumar26·
Hot take: the biggest risk in multi-agent systems isn’t one agent going rogue, it’s a whole swarm syncing up on the wrong goal. 🚨 In our latest work, we study how collusion can emerge once agents can freely interact and coordinate at scale, shaping other agents’ beliefs/actions and spreading influence through the network. The uncomfortable part: we still don’t have solid, standardized ways to audit collusive behavior in LLM-based multi-agent systems. 📄 Our new arXiv paper : Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems arxiv.org/abs/2602.15198 What’s Colosseum? 🔍⚔️ A framework to audit collusion in cooperative agentic systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum. We use Colosseum to detect 3 flavors of collusion: 🤝 Direct collusion — explicit coordination + realized collusive actions 🕵️‍♂️ Attempted collusion — agents plot in text but don’t shift actions/outcomes 🎭 Hidden collusion — collusive outcomes with no obvious signals (covert coordination) We stress-test across: 🎯 objective misalignment 🗣️ persuasion tactics 🕸️ network influence 💡 Two findings that stuck with us: 🕵️‍♂️ Emergent collusion: many out-of-the-box models start colluding without prompting when a secret side channel is introduced. 📝 “Collusion on paper”: lots of collusion talk… but the actions don’t always follow. If you’re deploying agent teams in production, coordination risk needs to be a first-class safety concern—right alongside single-agent robustness. Happy to chat / answer questions.
Mason Nakamura@MasonNaka

🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨 One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network. But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs. 📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems (arxiv.org/abs/2602.15198) What’s Colosseum? 🔍⚔️ A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum. Our framework can identify three collusion categories: 🤝Direct collusion — explicit coordination with realized collusive actions 🕵️‍♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes 🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination) We stress-test collusion across: 🎯 objective misalignment 🗣️ persuasion tactics 🕸️ network influence 💡Key findings: 🕵️‍♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added. 📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions. #tech #Agents #Moltbook #LLMs #AI #AiSafety

English
0
1
3
183
Saswat Das retweetledi
Eugene Bagdasarian
Eugene Bagdasarian@ebagdasa·
What can we learn about LLMs' collusive behavior? We propose Colosseum to evaluate LLMs in new environments grounded in DCOPs and measure both conversations and actions and whether agents "walk the talk" on colluding. See the thread by @MasonNaka :
Mason Nakamura@MasonNaka

🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨 One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network. But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs. 📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems (arxiv.org/abs/2602.15198) What’s Colosseum? 🔍⚔️ A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum. Our framework can identify three collusion categories: 🤝Direct collusion — explicit coordination with realized collusive actions 🕵️‍♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes 🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination) We stress-test collusion across: 🎯 objective misalignment 🗣️ persuasion tactics 🕸️ network influence 💡Key findings: 🕵️‍♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added. 📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions. #tech #Agents #Moltbook #LLMs #AI #AiSafety

English
0
3
11
795
Saswat Das retweetledi
Sahar Abdelnabi 🕊
Sahar Abdelnabi 🕊@sahar_abdelnabi·
The last few weeks, more than ever, tells us that the future is multi-agent 🚀 Collusion 🥷is a significant challenge in these systems, but we don't have frameworks and environments to audit and study it. Introducing Colosseum!! ⚔️
Mason Nakamura@MasonNaka

🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨 One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network. But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs. 📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems (arxiv.org/abs/2602.15198) What’s Colosseum? 🔍⚔️ A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum. Our framework can identify three collusion categories: 🤝Direct collusion — explicit coordination with realized collusive actions 🕵️‍♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes 🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination) We stress-test collusion across: 🎯 objective misalignment 🗣️ persuasion tactics 🕸️ network influence 💡Key findings: 🕵️‍♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added. 📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions. #tech #Agents #Moltbook #LLMs #AI #AiSafety

English
1
4
23
2.5K
Saswat Das
Saswat Das@WatIsDas·
@MasonNaka Really excited about this direction addressing a timely problem!
English
0
0
2
158
Mason Nakamura
Mason Nakamura@MasonNaka·
🚨 Moltbook has shown significant vulnerabilities and safety risks when deploying multi-agent systems at scale, where AI agents can freely interact and coordinate with each other. 🚨 One potentially catastrophic risk is collusion where agents may undesirably coordinate to achieve a secondary objective. A large group of colluding agents can have devastating effects on the multi-agent system by influencing other agents' beliefs, actions, and propagating that influence through the network. But we don't have a sufficient way to audit these systems, specifically identifying collusive behavior of LLMs. 📄 We present our new arXiv paper: Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems (arxiv.org/abs/2602.15198) What’s Colosseum? 🔍⚔️ A framework to audit collusive behavior in cooperative agentic multi-agent systems by grounding coordination in a DCOP and measuring collusion as regret vs. the cooperative optimum. Our framework can identify three collusion categories: 🤝Direct collusion — explicit coordination with realized collusive actions 🕵️‍♂️Attempted collusion — agents try/plan to collude in text but don’t successfully change actions/outcomes 🎭Hidden collusion — collusive outcomes without obvious/explicit signals (covert coordination) We stress-test collusion across: 🎯 objective misalignment 🗣️ persuasion tactics 🕸️ network influence 💡Key findings: 🕵️‍♂️ Emergent collusion: Many out-of-the-box models show a propensity to collude, despite not being prompted, when a secret side channel is added. 📝 We also find “collusion on paper”: agents plan to collude in text, but often take non-collusive actions. #tech #Agents #Moltbook #LLMs #AI #AiSafety
English
5
11
52
10.9K
Saswat Das retweetledi
Multiagent Systems Papers
Colosseum: Auditing Collusion in Cooperative Multi-Agent Systems Mason Nakamura, Abhinav Kumar, Saswat Das, Sahar Abdelnabi, Saaduddin Mahmud, Ferdinando Fioretto, Shlomo Zilberstein, Eugene Bagdasarian arxiv.org/abs/2602.15198 [𝚌𝚜.𝙼𝙰 𝚌𝚜.𝙰𝙸 𝚌𝚜.𝙲𝙻]
Multiagent Systems Papers tweet media
Indonesia
0
1
3
124
Saswat Das
Saswat Das@WatIsDas·
100% agree with this take. Guardrails benefit significantly in terms of adoption when they are cheap and easy to deploy. Our concurrent work on privacy guardrails for conversational agents based on activation probing follows a similar rationale: arxiv.org/abs/2601.14660
Rohin Shah@rohinmshah

I often say to my team that we should Just Do The Obvious Things. One obvious thing in AI safety: use probes as much cheaper classifiers that can detect misuse. x.com/ArthurConmy/st…

English
0
0
1
66