Selen Erkan

Special thanks to my co-first author Kim-Celine Kahl (@kc_kahl) for the valuable contribution to this project to push research in this field! Also thanks to the collaborators in this project - @jeremiastraub , @CarTLueth, @lena_maierhein, @maierhein and @pfjaeger.

📊 With our large-scale robustness study we find that: 1. No fine-tuning method consistently outperforms others in robustness 2. Robustness varies more by shift than by method 3. Simple image-free baselines perform surprisingly well and LoRA performs best on in-distribution data

🛠️ To address this gap, we: 1. Identify pitfalls and define core requirements for robust VLM evaluation. 2. Introduce SURE-VQA — an open-source framework for systematic robustness evaluation. 3. Validate it empirically and benchmark the robustness of various fine-tuning methods.

❓Vision-Language Models (VLMs) hold promise for medical VQA, but their robustness to unseen data is a concern. Evaluating this requires controlled setups for systematic insights—yet current approaches often fall short, limiting their ability to accurately assess model robustness

📣 I'm excited to share that our paper "SURE-VQA: Systematic Understanding of Robustness Evaluation in Medical VQA Tasks" has been accepted to #TMLR ! 📃 Read our paper: openreview.net/pdf?id=qjNdGpg… 💻 Check out the code: github.com/IML-DKFZ/sure-… More ⬇️

🔍 TLDR: We reveal gaps in current robustness evaluations of medical vision-language models and introduce SURE-VQA, a framework addressing them via real-world shifts, improved metrics, and sanity baselines—validated with a large study across 3 datasets and 4 distribution shifts.

We’re thrilled to welcome Eric Brachmann (@eric_brachmann), Senior Staff Scientist at Niantic Spatial, Inc. and a leading expert in visual relocalisation & pose estimation, to our heidelberg.ai / NCT Data Science Seminar on July 16th at 4 PM. 👇Details Below 1/3

In case you missed the last heidelberg.ai talky by Prof. Yuki Asano (@y_m_asano) on "Post-Pretraining in Vision, and Language Foundation Models", it is just released on the heidelberg.ai Youtube Channel: youtube.com/watch?v=5UTCjQ…