Copilot Arena

What do developers 𝘳𝘦𝘢𝘭𝘭𝘺 think of AI coding assistants? In October, we launched @CopilotArena to collect user preferences on real dev workflows. After months of live service, we’re here to share our findings in our recent preprint. Here's what we have learned /🧵

Introducing ⚔️PR Arena⚔️ - free AI coding agents to fix real GitHub issues. Claude Sonnet 4 vs Gemini 2.5 Pro… Who writes better pull requests? 👉 Install here: github.com/apps/openhands… Powered by @allhands_ai

Introducing ⚔️PR Arena⚔️ - free AI coding agents to fix real GitHub issues. Claude Sonnet 4 vs Gemini 2.5 Pro… Who writes better pull requests? 👉 Install here: github.com/apps/openhands… Powered by @allhands_ai

Excited to share our beta release of Music Arena, a live evaluation platform for state-of-the-art AI music generation models! 🎧 Listen to the latest models and 🗳️ vote for your favorite ⚔️ music-arena.org ⭐️ github.com/gclef-cmu/musi… 📜 arxiv.org/abs/2507.20900

Since our launch earlier this year, we are thrilled to witness the growing community around dLLMs. The Mercury tech report from @InceptionAILabs is now on @arxiv with more extensive evaluations: arxiv.org/abs/2506.17298 New model updates dropping later this week!

OpenAI is making a big push into one of the most popular AI domains: software engineering on.wsj.com/3SCvoW2

What do developers 𝘳𝘦𝘢𝘭𝘭𝘺 think of AI coding assistants? In October, we launched @CopilotArena to collect user preferences on real dev workflows. After months of live service, we’re here to share our findings in our recent preprint. Here's what we have learned /🧵

blog.ml.cmu.edu/2025/04/09/cop… How do real-world developer preferences compare to existing evaluations? A CMU and UC Berkeley team led by @iamwaynechi and @valeriechen_ created @CopilotArena to collect user preferences on in-the-wild workflows. This blogpost overviews the  design and deployment of Copilot Arena + new insights into developer code preferences.

New #1 Leaders of Code Edit Leaderboard: Strong performance from both Claude 3.7 Sonnet and Gemini-2.0-Pro! Congratulations to @AnthropicAI and @GoogleDeepMind 🥇 We also release new live leaderboard interface✨. You can now easily toggle between code completion and code edit.

News from @CopilotArena: Code Editing Leaderboard is now LIVE! We have collected over 3.7k votes on 6 models. Congrats @AnthropicAI Claude 3.5 Sonnet on a 1st place rank!🥇 Blog analysis below👇