Aadarsh Sahoo

Perception is actionable. Humans don't just see objects, we see affordances and constraints. "Something to sit on." "Region unsafe to walk." "Something that will tip if I bump it." But today’s vision models mostly see… labels. So we built ConverSeg: Conversational Image Segmentation 🧵 glab-caltech.github.io/converseg/

Until now, robotics stopped where the human hand begins. Strong, but not delicate. Precise, but not adaptive. Repetitive, but not creative. The human hand wasn’t a benchmark — it was a boundary. We are crossing it. @EkaRobotics. Coming soon.

@songyoupeng @GoogleDeepMind Awesome!! Curious to see how well the model works on more complex 2D vision task, such as conversational image grounding x.com/SahooAadarsh/s…

@ShangbangLong Congrats, this is really cool! Looking forward to the code release -- curious to try it out on our ConverSeg benchmark: glab-caltech.github.io/converseg/

🚀 Excited to announce Vision Banana 🍌 and our new paper: “Image Generators are Generalist Vision Learners”. We turn Nano Banana Pro into a state-of-the-art visual generation and understanding model. 🖼️ Check out our gallery at vision-banana.github.io 🧵 (1/N) continue ⬇️

With Emmanuel Dupoux scp.net/persons/dupoux/ and Yann LeCun @ylecun, we consider a cognitive science inspired AI. We analyse how autonomous learning works in living organisms, and propose a roadmap for reproducing it in artificial systems. lnkd.in/eNWDmuqT

Today’s video world models “simulate” the world by generating pixel frame observations🖼️. Can they continue to simulate the world when observations are interrupted - such as by occlusion, illumination dimming, or camera lookaway? To probe this question, we release STEVO-Bench, which holistically evaluates whether image-/text-to-video models and camera-controlled video models can correctly evolve states under observation control. Check out our website, blog and paper for how they fail!

CIS. SAM2+Qwen2.5-VL to segment by physics, safety, and affordance. - good complex reasoning; understands something like "breakable items on the table" or "furniture blocking the walkway." glab-caltech.github.io/converseg/

@af_gao Thanks Angela!

@SahooAadarsh this is awesome! really great work aadarsh :)

Perception is actionable. Humans don't just see objects, we see affordances and constraints. "Something to sit on." "Region unsafe to walk." "Something that will tip if I bump it." But today’s vision models mostly see… labels. So we built ConverSeg: Conversational Image Segmentation 🧵 glab-caltech.github.io/converseg/

Big thanks to my advisor @georgiagkioxari for the guidance and support throughout this project! 🌐 Webpage: glab-caltech.github.io/converseg/ 🎛️ Demo: huggingface.co/spaces/aadarsh… 📄 Paper: arxiv.org/abs/2602.13195

We also introduce: ConverSeg-Net, a single-pass conversational segmentation model baseline. It combines strong segmentation priors (SAM2) with visual-language reasoning (Qwen2.5-VL-3B) via lightweight adapters, and stays competitive while handling these "beyond labels" prompts.

Our paper, VALOR, got accepted at #ICLR2026 ! We explore improving visual reasoning using multimodal verifiers - all without any ground truth annotations! More details below 👇 Excited to see everyone in Rio!

🤓 How do LVLM/LMMMs reason about space and time? This was the central question of our #ICLR2016 paper, “Linear Mechanisms For Spatiotemporal Reasoning In Vision Language Models”. I’m very excited to finally share it:D 🥳🥳 A thread: [1/7]

(1/6) Do these images show the same vacuum cleaner? They are certainly similar, but a human will notice the differences in dustbin geometry, design, and color accents. In contrast, open-source VLMs struggle at this task. Our recent work TWIN poses the question: Can we fix this?

Generative models shouldn’t just generate. They should be steerable by your commands. Meet Steer3D🕹️: edit generated 3D assets with text📝 in one forward pass. Trained on only 100k synthetic data, it shows that we can make generative models responsive to signals from another modality🎛️. Check out: glab-caltech.github.io/steer3d/

(1/N): Can we improve visual reasoning models without annotations? In VALOR, we introduce an annotation-free training framework that boosts both visual reasoning and object grounding by training with multimodal verifiers instead of human labels

My student @SaberaTalukder and I are creating a new startup that deeply rethinks how we architect and engage with multimodal models. 🚀 We are chatting with investors at #NeurIPS2025, and if you want to get on our radar, DM Sabera.

🚀 Excited to share that my team at Meta just launched Segment Anything 3! SAM 3 doubles the performance of existing models on open-vocabulary instance segmentation on our new SA-Co benchmark, with 207K unique object labels. Huge congrats to the team, so proud of this work!