Yiming Dou

Ever wondered how a scene sounds👂 when you interact👋 with it? Introducing our #CVPR2025 work "Hearing Hands: Generating Sounds from Physical Interactions in 3D Scenes" -- we make 3D scene reconstructions audibly interactive! yimingdou.com/hearing_hands/

Despite much progress in AI, the ability for AI to 'smell' like humans remains elusive. Smell AIs 🤖👃can be used for allergen sensing (e.g., peanuts or gluten in food), hormone detection for health, safety & environmental monitoring, quality control in manufacturing, and more. As a step towards AI for smell, our group is releasing **SmellNet,** a massive open dataset to advance AI smell-recognition in real-world settings. Using portable gas and chemical sensors, we collected 180,000 time steps of 50 substances (spanning nuts, spices, herbs, fruits, and vegetables) with 50 hours of data. SmellNet enables the training of AI models for real-time classification of substances based on their smell alone - see video below, where even subtle differences between cumin, cloves, and oregano can be detected. Check out our paper and open-source data & code for the smell AI revolution! paper: arxiv.org/abs/2506.00239 data & code: github.com/MIT-MI/SmellNet w Dewei, Carol, David @ddvd233 @medialab @MITEECS

Hello! If you are interested in dynamic 3D or 4D, don't miss the oral session 3A at 9 am on Saturday: @zhengqi_li will be presenting "MegaSaM" I'll be presenting "Stereo4D" and @QianqianWang5 will be presenting "CUT3R"

@zizhpan Haha thanks😆

@_YimingDou Very interesting research! It feels so funny when you collecting the data (the scene where you're patting the sofa) 🤣.

Ever wondered how a scene sounds👂 when you interact👋 with it? Introducing our #CVPR2025 work "Hearing Hands: Generating Sounds from Physical Interactions in 3D Scenes" -- we make 3D scene reconstructions audibly interactive! yimingdou.com/hearing_hands/

Excited to share our CVPR 2025 paper on cross-modal space-time correspondence! We present a method to match pixels across different modalities (RGB-Depth, RGB-Thermal, Photo-Sketch, and cross-style images) — trained entirely using unpaired data and self-supervision. Our approach learns correspondences through contrastive random walks across visual modalities. #CVPR2025 (1/6)

Can AI image detectors keep up with new fakes? Mostly, no. Existing detectors are trained using a handful of models. But there are thousands in the wild! Our work, Community Forensics, uses 4800+ generators to train detectors that generalize to new fakes. #CVPR2025 🧵 (1/5)

Wonderful collaboration with Wonseok Oh, Yuqing Luo, @antoniloq, @andrewhowens!!!

Combining with our previous #CVPR2024 work TaRF (yimingdou.com/TaRF/), we create an immersive 3D scene reconstruction that allows users to interact with it using sight👀, touch👆 and sound👂.

Hello! If you like pretty images and videos and want a rec for CVPR oral session, you should def go to Image/Video Gen, Friday at 9am: I'll be presenting "Motion Prompting" @RyanBurgert will be presenting "Go with the Flow" and @ChangPasca1650 will be presenting "LookingGlass"

Ever wish YouTube had 3D labels? 🚀Introducing🎥DynPose-100K🎥, an Internet-scale collection of diverse videos annotated with camera pose! Applications include camera-controlled video generation🤩and learned dynamic pose estimation😯 Download: huggingface.co/datasets/nvidi…

🧩#CVPR2025🌷Introducing Two By Two✌️: The First Large-Scale Daily Pairwise Assembly Dataset with SE(3)-Equivariant Pose Estimation. 🤖2BY2 helps robots master daily 3D assembly tasks—like plugging sockets or arranging flowers—across diverse objects! 🐨Co-lead by @yuqi_Beijing

Thanks to @OpenAI, got a chance to grow up again in Ghibli anime🤗

I’m on the PhD internship market for Spr/Summer 2025! I have experience in multimodal AI (EHR, X-ray, text), explainability for image models w/ genAI, clinician-AI interaction (surveyed 700+ doctors), and tabular foundation models. Please reach out if you think there’s a fit!

🍌We present DenseMatcher！ 🤖️DenseMatcher enables robots to acquire generalizable skills across diverse object categories by only seeing one demo, by finding correspondences between 3D objects even with different types, shapes, and appearances.

What happens when you train a video generation model to be conditioned on motion? Turns out you can perform "motion prompting," just like you might prompt an LLM! Doing so enables many different capabilities. Here’s a few examples – check out this thread 🧵 for more results!

Excited to share MonST3R! -- a simple way to estimate geometry from unposed video of dynamic scene We achieve competitive results on several downstreams (video depth, camera pose) and believe this is a promising step toward feed-forward 4D reconstruction monst3r-project.github.io

Differentiable rendering made SIMPLE❗️ Differentiating physically based renderers is hard: Dirac-delta discontinuities arise at object silhouette. Our #SIGGRAPHAsia2024 work shows how a simple relaxation can rescue the day, enabling easy 3D reconstruction and relighting! (1/N)