Daan de Geus

🚀 New CVPR Workshop paper: Plain Mask Transformer (PMT) Finetuning VFMs for segmentation breaks their key advantage: using a shared, frozen encoder for multiple tasks. PMT: a fast Mask Transformer for frozen VFM features. 📄 arxiv.org/abs/2603.25398 💻 github.com/tue-mps/pmt

Video segmentation methods have become increasingly complex. In our #CVPR2026 paper we show once again that a surprisingly simple encoder-only architecture is actually sufficient, boosting speed up to 10x!🚀 📄Paper: arxiv.org/abs/2602.17807 💻Code: github.com/tue-mps/videomt (1/5)

Opus 4.6 made a @Gradio demo for it too! It uses a "chunked window" approach, allowing it to run up to 160 frames per second (FPS) I'm really impressed by coding agents - porting a model + demo in less than 2 days

🔥Encoder-only Mask Transformer (EoMT) now also supports DINOv3! ⚡️ Upgrading DINOv2 to DINOv3 ➡️ more accurate image segmentation, same lightning-fast inference 🔗 Code and models: github.com/tue-mps/eomt

🔥 EoMT now supports DINOv3! 🔥 ✨ Upgrading DINOv2 to DINOv3 consistently improves image segmentation performance ⚡ Lightning-fast inference with EoMT's minimalist design 📦 Model weights and code available! 🔗 github.com/tue-mps/eomt

I like the Encoder-only Mask Transformer (EoMT): basically removing all the bells and whistles, and doing panoptic segmentation with an almost vanilla ViT. You're sliiiiightly worse for the same encoder size, but it's a lot simpler/faster and (likely) more scalable. I wish they had peak gpu memory to that table though.

New model alert in Transformers: EoMT! EoMT greatly simplifies the design of ViTs for image segmentation 🙌 Unlike Mask2Former and OneFormer which add complex modules like an adapter, pixel decoder and Transformer decoder on top, EoMT is just a ViT with a set of query tokens ✅

🚨 CVPR Highlight Alert! 🚨 We’re presenting our Encoder-only Mask Transformer (EoMT) tomorrow at #CVPR2025, 10:30–12:30, Poster #407! 🎸 👉 github.com/tue-mps/eomt ➕ Bonus: we're releasing the biggest EoMT yet… (1/2)

I'll be presenting "DINO in the Room (DITR)", the winning method of the ScanNet++ 3D semantic segmentation challenge, tomorrow at CVPR at 10 a.m. in Room 211. Project page: visualcomputinginstitute.github.io/DITR/

Very excited about this work!

Image segmentation doesn’t have to be rocket science. 🚀 Why build a rocket engine full of bolted-on subsystems when one elegant unit does the job? 💡 That’s what we did for segmentation. ✅ Meet the Encoder-only Mask Transformer (EoMT): tue-mps.github.io/eomt (CVPR 2025) (1/6)

🚀Check our recent work #Interactive4D to achieve interactive #LiDAR segmentation of multiple objects on multiple scans simultaneously. Work with Ilya Fradlin, @KadirYilmaz_CV, @DoraKontog, and @BastianLeibe. 🌐Project: ilya-fradlin.github.io/Interactive4D/ 📜Paper: arxiv.org/pdf/2410.08206👇🧵

The BRAVO Challenge 2014 attracted nearly 100 submissions from international teams representing notable research institutions. The results reveal valuable insights in developing reliable semantic segmentation models. #ECCV2024 #UNCVWorkshop arxiv.org/abs/2409.15107

Honored to be on this list 😊 Thanks @eccvconf organizers and ACs! And congrats to the other outstanding reviewers!

Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think discuss: huggingface.co/papers/2409.11… Recent work showed that large diffusion models can be reused as highly precise monocular depth estimators by casting depth estimation as an image-conditional image generation task. While the proposed model achieved state-of-the-art results, high computational demands due to multi-step inference limited its use in many scenarios. In this paper, we show that the perceived inefficiency was caused by a flaw in the inference pipeline that has so far gone unnoticed. The fixed model performs comparably to the best previously reported configuration while being more than 200times faster. To optimize for downstream task performance, we perform end-to-end fine-tuning on top of the single-step model with task-specific losses and get a deterministic model that outperforms all other diffusion-based depth and normal estimation models on common zero-shot benchmarks. We surprisingly find that this fine-tuning protocol also works directly on Stable Diffusion and achieves comparable performance to current state-of-the-art diffusion-based depth and normal estimation models, calling into question some of the conclusions drawn from prior works.

Check out our work on fine-tuning of image-conditional diffusion models for depth and normal estimation. Widely used diffusion models can be improved with single-step inference and task-specific fine-tuning, allowing us to gain better accuracy while being 200x faster!⚡ 🧵(1/6)

At #CVPR2024? Check out our poster (#293) on Wednesday (tomorrow) at 10:30! Paper: arxiv.org/abs/2406.10114 Project page and code: tue-mps.github.io/tapps/ [3/3]

TAPPS: ➡️ Uses a Mask Transformer ➡️ Uses shared queries that jointly represent objects and their parts ➡️ Segments objects and parts jointly ➡️ Is directly optimized for the PPS task, instead of surrogate subtasks like existing approaches ✅ Achieves SOTA accuracy [2/3]

Want to identify and segment all foreground objects, background regions AND object parts in an image, in a consistent manner? Look no further! Tomorrow at #CVPR2024 we’ll present TAPPS, a unified network for the part-aware panoptic segmentation (PPS) task. [1/3]