@meta_aria This is very cool work! It’s a more unscripted version of what we did with HoloAssist (holoassist.github.io). I think these type of datasets are very valuable given the rich and purposeful interactions they capture.
English
Marc Pollefeys
186 posts
Marc Pollefeys
@mapo1
Director of Science at @Microsoft @HoloLens, Professor of Computer Science at @ETH Zurich, working on #ComputerVision
Zurich, Switzerland Katılım Nisan 2009
349 Takip Edilen7.5K Takipçiler
How do we train AI to be a real-world expert? 🛠️🧠
Introducing Ego-EXTRA, a new video-language dataset for Expert-Trainee assistance. Researchers at the University of Catania captured unscripted interactions where experts guide trainees through complex tasks in real-time using Aria Glasses.
👓 50 hours of data
💬 15k+ high-quality VQA sets
✅ RGB + Eye Gaze tracking ✅ Hand keypoints & SLAM ✅ IMU, Magnetometer, & Audio ✅ Natural expert-trainee dialogue
🚲 4 scenarios: Bike Workshop, Kitchen, Bakery, & Assembly
Explore the benchmark: fpv-iplab.github.io/Ego-EXTRA/
Read the paper: arxiv.org/abs/2512.13238
👉@FrancRagusa, @GMFarinella
English
Marc Pollefeys retweetledi
Introducing EgoVerse: an ecosystem for robot learning from egocentric human data.
Built and tested by 4 research labs + 3 industry partners, EgoVerse enables both science and scaling
1300+ hrs, 240 scenes, 2000+ tasks, and growing
Dataset design, findings, and ecosystem 🧵
English
Marc Pollefeys retweetledi
#ICCV2025 is starting today 🌺 you can find us at:
🥽 LaMAria Tutorial: 325B, 1:00PM
➡️lamaria.ethz.ch/tutorial
☀️ OpenSUN3D Workshop: 306 B, 1:30PM
➡️opensun3d.github.io
@pesarlin @efedele16 @FrancisEngelman @AlexDelitzas @mapo1 @ICCVConference
English
Marc Pollefeys retweetledi
What a fantastic week at #CoRL2025 in Seoul! It was a pleasure for us on the 🇨🇭Zurich @Microsoft team to finally meet in person the colleagues we collaborate with from our global 🌍 AI teams in Tokyo, Cambridge, Redmond, Beijing and Seoul!
English
Welcome, @oier_mees ! It's been great to have you join our team.
If you too are excited about this topic, consider joining us! We have open positions for full-time and interns.
Oier Mees@oier_mees
After two fantastic years at @UCBerkeley I'm thrilled to share that I've joined @Microsoft in Zurich🇨🇭to pioneer the next generation of multimodal foundation models to drive agents 🤖 that can seamlessly interact across the digital and physical worlds 🌍 We are hiring! 🧵
English
Welcome, @oier_mees ! It's been great to have you join our team.
If you too are excited about this topic, consider joining us!
Oier Mees@oier_mees
After two fantastic years at @UCBerkeley I'm thrilled to share that I've joined @Microsoft in Zurich🇨🇭to pioneer the next generation of multimodal foundation models to drive agents 🤖 that can seamlessly interact across the digital and physical worlds 🌍 We are hiring! 🧵
English
Marc Pollefeys retweetledi
Multi-View 3D Point Tracking
Frano Rajič, @haofeixu, @marko_mih, @derek_siyuanli, Irem Demir, Emircan Gündoğdu, @leike_lk, @serge_prokudin, @mapo1, @SiyuTang3
tl;dr: per-view features+depth maps->fused 3D feature point cloud->kNN-based correlation->spatiotemporal transformer->overlapping windows->3D point trajectories
arxiv.org/abs/2508.21060
Marc Pollefeys retweetledi
You're in Zurich or its zone of influence (Lausanne, Paris, BXL, Munich, London, ...) and like AI + Robots?
We (@openai) together with @mimicrobotics, @lokirobotics, and Zurich Builds are organizing a hackathon from Fri 9 May afternoon to Sun 11.
Limited spots, more below:
English
Marc Pollefeys retweetledi
Yuedong @donydchen just gave a fantastic Oral presentation of MVSplat at #ECCV2024! Come to our poster 157 to discuss more!
Milan, Lombardy 🇮🇹 English
Very serious question:
Why is 12pm noon and 12am midnight?
7am
8am
9am
10am
11am
12pm
7pm
8pm
9pm
10pm
11pm
12am
See what I mean? Shouldn’t it be the other way?
English
@SavinovNikolay @TeplyashinDenis @machelreid This is really cool. Always exciting when former students do great things! Looking forward to catch up.
English
I was leading long context in Gemini for a while now, and today I’m proud to share what the team has achieved: over 1M context in a large-scale foundation model.
Big shoutout to @TeplyashinDenis and @machelreid - without you this would not have happened!
youtube.com/watch?v=wa0MT8…
YouTube
English
Marc Pollefeys retweetledi
📢 Today, we release LabelMaker, a fully automatic tool to create semantic annotations for indoor scenes. LabelMaker will be presented as spotlight at #3DV2024 in March.
🔗 labelmaker.org
🛠️ github.com/cvg/LabelMaker
with @hermannsblum , @FrancisEngelman , and @mapo1
English
Marc Pollefeys retweetledi
I am overjoyed to announce I am hosting the "Workshop on Urban Scene Modeling" along with an amazing team at @CVPR 2024!
Submission deadline March 24
usm3d.github.io
more two challenges, brand new dataset, and amazing keynote speakers soon...!
#CVPR2024 #usm3d #usm3d2024
English
Greatly honored to join the ranks of the #ACMFellows! Thank you so much to my nominator and endorsers, as well as my amazing students, collaborators and mentors over the years! @ETH_en @ETH_AI_Center @TheOfficialACM
ETH CS Department@CSatETH
👏Big congratulations to @arkrause for being named @TheOfficialACM Fellow. The distinction recognises Krause's extensive research contributions to learning-based decision making under uncertainty. @ETH_en @ETH_AI_Center bit.ly/429CxBg
English
Marc Pollefeys retweetledi
For #NeurIPS2023 we're presenting our latest work SNAP - Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding
Geo data at scale + self-supervision = 💥
Video: youtu.be/xogrwdgNQdo
Paper: arxiv.org/pdf/2306.05407
Code: github.com/google-researc…
summary🧵⬇️
YouTube
English
Marc Pollefeys retweetledi
Today @aycatakmaz and I will present our paper OpenMask3D at @NeurIPSConf 🎷🚀
Come by to learn more about the project or to chat about related topics!
📍 Great Hall & Hall B1+B2 (level 1) #906
🕰️ 10:45-12:45
🌎 openmask3d.github.io
@FrancisEngelman @fedassa @mapo1
New Orleans, LA 🇺🇸 English
Marc Pollefeys retweetledi
Today @efedele16 and I will present our work OpenMask3D at @NeurIPSConf 🎷
Visit our poster to learn more about OpenMask3D or to chat with us!
📍 Great Hall & Hall B1+B2 (level 1) #906
🕰️ 10:45-12:45
🌎 openmask3d.github.io
@FrancisEngelman @fedassa @mapo1
English
Marc Pollefeys retweetledi
Join us today @ICCVConference at the @GoogleAI booth for a live demo of OpenMask3D 🛋️ (un petit amuse-bouche of our @NeurIPSConf '23 work on open-set 3D instance segmentation)
⏲️15:30
🏠Google Booth
openmask3d.github.io
Amazing work w/ @aycatakmaz @efedele16 @mapo1 @fedassa
English
Marc Pollefeys retweetledi
I am excited to announce our HoloAssist work at #ICCV! Get your hands on the dataset at holoassist.github.io, released under a permissive license. Meet my collaborator @big_stamp at the poster session this Friday from 02:30-04:30 PM. Huge thanks to the coauthors and colleagues at @MSFTResearch for their support. Tagging co-authors on X - @neelsj @mapo1 @danbohus @seanandrist.
Microsoft Research@MSFTResearch
HoloAssist is a new multimodal dataset consisting of 166 hours of interactive task executions with 222 participants. Discover how it offers invaluable data to advance the capabilities of next-gen AI copilots for real-world tasks: msft.it/60139Uv4d
English
Marc Pollefeys retweetledi
Joint work by: @DRozumnyi @matas_jiri @mapo1 @VittoFerrariCV @Martin_R_Oswald
Code: github.com/rozumden/track…
English