Aditya Mittal

LLMs are much less capable judges for code than we might expect. Super excited that our paper on LLM judges for code has been accepted to COLM! In it, we discuss this weakness and provide a pipeline for understanding and diagnosing issues with LLM code preferences. This was led by two wonderful students @RyanShar01 and @AdityaM129. It was an absolute pleasure mentoring them on this project.

The work on comparing developer and LLM biases is led by @RyanShar01 and @AdityaM129. We show that today's LLM judges remain substantially misaligned with developer preferences in realistic coding workflows. arxiv.org/abs/2603.24586

New preprint alert 🚨 Can LLM agents develop video games? We release GameDevBench, the first benchmark evaluating agentic game development in a game engine, Godot. We also present two simple multimodal feedback mechanisms that lead to immediate performance gains. /🧵

Tired of evaluating LLMs on made-up problems that look nothing like real tasks? Introducing EDIT-Bench, a code editing benchmark built from in-the-wild user interactions in VSCode. Real-world edits are challenging: 𝗼𝗻𝗹𝘆 𝟭/𝟰𝟬 𝗺𝗼𝗱𝗲𝗹𝘀 𝘀𝗰𝗼𝗿𝗲 > 𝟲𝟬% 𝗽𝗮𝘀𝘀@𝟭.

📣 @CopilotArena leaderboard is now live on lmarena.ai/leaderboard! We have collected over 15k votes on 11 models (2 new models since our last blogpost release). Congrats @deepseek_ai🥇and @AnthropicAI🥇!

Introducing Copilot Arena - Interactive coding evaluation in the wild. Our extension lets you test top models for free, right in VSCode. Let's vote and build the Copilot leaderboard! Download here: marketplace.visualstudio.com/items?itemName… Led by @iamwaynechi and @valeriechen_ at CMU. 1/🧵

My prediction stands: the bioeconomy will surpass the tech economy in impact and scale. I first discussed this at SynBioBeta 2022 and was inspired by the passionate entrepreneurs I met there. This year, the top minds in synthetic biology will gather in San Jose, May 6-9. Don’t miss out — join, or follow along! synbiobeta.com/attend/synbiob…

@AnekaMulgund Keyboard name goes hard 🔥

We just raised a $28 million Series A from Coatue and @Lux_Capital to build the AWS x GitHub for biocomputing. 1/N🔻 latchbio.substack.com/p/announcing-o…

@michaelraspuzzi @theksociety @Google @thielfellowship @SchmidtFutures thanks @michaelraspuzzi @theksociety for allowing our team to present our idea about riboswitches for increasing biofuel production one highlight was hearing @AnekaMulgund's pitch about optimizing ion-exchange flow-state batteries with ML!

demo day was yesterday @theksociety. we had people from @Google X, @thielfellowship , @SchmidtFutures, and more jam for 2 hours 🌶️ what were your highlights @ community?

we need more ambition aimed at being part of the solutions for the world's biggest problems. @theksociety's activate accelerator ends with just that, an earthshot sprint. more in 🧵

@danielchingwq @DicksonWuML

does anyone know of high schoolers (or just graduated) who have a deep knowledge in the field of machine learning / AI? please tag them in the thread below! looking for potential speakers 👀👀👀

The students x students team is looking for a newsletter editor! 📝 Apply now: form.typeform.com/to/waFHlAQt DM or email us (sxtudents@gmail.com) for more information or to ask questions. -- #enjoywriting #student #students #writing #editor #job #team #newsletter #hiring

Picture 1 - spoke with @ericschmidt on the role Gen Z has in the circular bioeconomy Picture 2 - picture with extended TKSquad 😎 Picture 3 - talked with technical engineers about automation platform at @conagen Picture 4 - met James Rosas and Emma Rooholfada about @powbio

@BuiltWithBio Day 3 ✅ Learned about extracting sugars from stevia to make synthetic alcohol from @conagen, problems with scalability and continuous fermentation with Ellen Simmons, and problems with ML in syn bio with James Chuang, among others. Oh, also met @ericschmidt!

@reemagy Excited to see you there!

Picture 1 - posing with the TKSquad Picture 2 - pic of three sets of identical twins 😁 Picture 3 - lunch with Armen and Mingyang Picture 4 - just hopped off a walk-and-talk with Chris

@BuiltWithBio Day 2 ✅ Learned of bacterial transfection and chimeric plants from Vladimir Jojic, networking capabilities and language architectures from Armen Babikyan, and Adapsyn and drug discovery from Chris Dejong, among others. Excited for Day 3 and meeting others at RATC!