Multi Agent Learning Seminar

@MIT If you couldn’t attend the talk, you can still watch the recording on our YouTube channel here: youtu.be/JzzpZGvAFVU?fe…

We're back for the Spring 2025 edition of the seminar! Join us this Friday to listen to Sid Nayak from @MIT present 'LLaMAR': Long-Horizon Planning for Multi-Agent Robots in Partially Observable Environments. See you all there!

This Friday, we're excited to host @liangyanchenggg from @UW Yancheng will present "Building Partner Populations for Human-AI Cooperation Using Generative Agents". See you all there!

This Friday, we’re excited to host @NaveenJRaman from @SCSatCMU Naveen will be presenting "Global Rewards in Restless Multi-Armed Bandits". See you all there!

This Friday, we’re excited to host @DTRadke7 from @NHLBlackhawks. David will be talking about “Multiagent challenges in Team Sports Analytics”. See y’all there!

We’re excited to host @gio_ramponi from @UZH_en this week. Join us to learn about “Theory of Imitation Learning: from single to multi-agent systems”. See you all there!

We’re back for the Fall edition of the seminar. Join us this Friday to listen to @mossr_jl from @Stanford talk about “BetaZero: Belief-State Planning for Long-Horizon POMDPs”!

This Friday, we will have the honour of hosting @liusiqi42 from @GoogleDeepMind and @ucl presenting “NfgTransformer: Equivariant Representation Learning of Normal-form Games”. See y’all there!

This week @_samvelyan from @Meta and @ucl will be presenting two very interesting papers in the talk on “Robust Agents in Open-Ended Worlds”. See you on Friday!

Multi-agent collaboration has emerged as a key AI agentic design pattern. Given a complex task like writing software, a multi-agent approach would break down the task into subtasks to be executed by different roles -- such as a software engineer, product manager, designer, QA (quality assurance) engineer, and so on -- and have different agents accomplish different subtasks. Different agents might be built by prompting one LLM (or, if you prefer, different LLMs) to carry out different tasks. For example, to build a software engineer agent, we might prompt the LLM: "You are an expert in writing clear, efficient code. Write code to perform the task …". It might seem counterintuitive that, although we are making multiple calls to the same LLM, we apply the programming abstraction of using multiple agents. I'd like to offer a few reasons: - It works! Many teams are getting good results with this method, and there's nothing like results! Further, ablation studies (for example, in the AutoGen paper cited below) show that multiple agents give superior performance to a single agent. - Even though some LLMs today can accept very long input contexts (for instance, Gemini 1.5 Pro accepts 1 million tokens), their ability to truly understand long, complex inputs is mixed. An agentic workflow in which the LLM is prompted to focus on one thing at a time can give better performance. By telling it when it should play software engineer, we can also specify what is important in that subtask: For example, the prompt above emphasized clear, efficient code as opposed to, say, scalable and highly secure code. By decomposing the overall task into subtasks, we can optimize the subtasks better. - Perhaps most important, the multi-agent design pattern gives us, as developers, a framework for breaking down complex tasks into subtasks. When writing code to run on a single CPU, we often break our program up into different processes or threads. This is a useful abstraction that lets us decompose a task -- like implementing a web browser -- into subtasks that are easier to code. I find thinking through multi-agents roles to be a useful abstraction. In many companies, managers routinely decide what roles to hire, and then how to split complex projects -- like writing a large piece of software or preparing a research report -- into smaller tasks to assign to employees with different specialties. Using multiple agents is analogous. Each agent implements its own workflow, has its own memory (itself a rapidly evolving area in agentic technologies -- how can an agent remember enough of its past interactions to perform better on upcoming ones?), and may ask other agents for help. Agents themselves can also engage in Planning and Tool Use. This results in a cacophony of LLM calls and message passing between agents that can result in very complex workflows. While managing people is hard, it's a sufficiently familiar idea that it gives us a mental framework for how to "hire" and assign tasks to our AI agents. Fortunately, the damage from mismanaging an AI agent is much lower than that from mismanaging humans! Emerging frameworks like AutoGen, Crew AI, and LangGraph, provide rich ways to build multi-agent solutions to problems. If you're interested in playing with a fun multi-agent system, also check out ChatDev, an open source implementation of a set of agents that run a virtual software company. I encourage you to check out their github repo and perhaps even clone the repo and run the system yourself. While it may not always produce what you want, you might be amazed at how well it does! Like the design pattern of Planning, I find the output quality of multi-agent collaboration hard to predict. The more mature patterns of Reflection and Tool use are more reliable. I hope you enjoy playing with these agentic design patterns and that they produce amazing results for you! If you're interested in learning more, I recommend: - Communicative Agents for Software Development, Qian et al. (2023) (the ChatDev paper) - AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, Wu et al. (2023) - MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework, Hong et al. (2023) [Original text: deeplearning.ai/the-batch/issu… ]

Link to the paper: nature.com/articles/s4146…

This Friday (19 April), we're very excited to host @zhexwang and @PetarV_93 from @GoogleDeepMind presenting their recent work on “TacticAI: an AI assistant for football tactics”. Join us at 9AM PST!

This week we have the fantastic @daphne_cor presenting her work on ‘Human-compatible driving partners through data-regularized self-play reinforcement learning’. Come checkout the talk to learn more about Human-Regularized PPO (HR-PPO).

If you couldn’t attend the talk, you can still watch the recording on our YouTube channel here: youtu.be/I8n7IH3FG7c

This Friday (08 Mar), the MARL Research Team at @instadeepai will be presenting their work on Growing the MARL software ecosystem in JAX at 9 AM PST. Looking forward to seeing y’all there!

For those who couldn’t attend the talk, you can watch the recording on our YouTube channel here: youtu.be/fv24aB3N7j4

This Friday (22 Mar), we will have the pleasure of hosting @sharky6000 from @GoogleDeepMind presenting ‘Evaluating Agents using Social Choice Theory’. Join us at 9AM PST!

If you missed the talk, checkout the recording on our YouTube channel: youtu.be/cdL0y6cxqr0?fe…

Calling all #GameTheory and #MARL enthusiasts! This Friday (15 Mar) @drimgemp will be talking about ‘Approximating Nash Equilibria via Stochastic Optimization’. Come check it out!

This Friday (01 Mar), Ram Rachum will be presenting his work on Emergent Dominance Hierarchies in Reinforcement Learning Agents. Join us at 9 AM PST for his talk!