Nando Metzger

🚀 Excited to unveil Marigold 🌼 I'm super proud to be part of this amazing work. Please also stop by the supplementary material to meet our cats Loki, Cleo, Haribo, and Pushka.

@GergelyOrosz You seem to be confusing the concept of plagiarism with copyright infringement. Rewriting code without a license is not plagiarism, but it is still copyright infringement.

This is either brilliant or scary: Anthropic accidentally leaked the TS source code of Claude Code (which is closed source). Repos sharing the source are taken down with DMCA. BUT this repo rewrote the code using Python, and so it violates no copyright & cannot be taken down!

@AntonObukhov1 @Michael_J_Black Badum

@Michael_J_Black Congratulations, epic move!

Tomorrow, I start a new full-time position as VP Digital Human Research at Epic Games. (And no, this isn’t an April Fools’ joke.) Today, I’m taking early retirement from Max Planck and will become an Emeritus Director. As an Emeritus Director, I will continue to supervise my remaining students, oversee ongoing projects, and wind down my department over the next couple of years, which is a normal process when a director retires. Being a Max Planck director is the best academic job in the world and it has been my great honor to co-found and help build the Max Planck Institute for Intelligent Systems. I love the institute deeply. But my statutory retirement age was looming and I’m not done yet. I’ve been working on capturing and modeling human movement for 30 years and it has gone from a fringe topic to something that works robustly and has immense potential. With AI today, scale matters, and achieving that scale increasingly requires industry. In particular, digital humans are moving from research prototypes to foundational technology across industries. At this point in my career, I want to get this technology into the hands of millions of users, while pushing the frontier of digital humans. I know that this is a challenging time for the games industry. Such times are precisely when people get creative, the industry is open to change, and real innovation can take root. Epic Games is the ideal place for this. The talent is deep, there is a compelling vision for how games will evolve, and the commitment is clear. I’m excited to join them and the rest of the Meshcapade team. There is no good way to express on social media the depth of my gratitude to the Max Planck Society, my co-directors, students, post docs, and staff. It has been an amazing 15-year journey because of you. Thank you.

@GnosisYu This is really cool, is there a code release planed?

World models have made impressive progress in video generation, yet they still struggle with a fundamental challenge: memory. In long rollouts, the camera trajectory gradually drifts from the user-specified motion and revisited scenes no longer align with earlier observations. These errors accumulate over time, causing the generated world to steadily lose coherence. 🚀Excited to share our solution MosaicMem 🌍🧠 — our new hybrid spatial memory for video world models. Project Page: mosaicmem.github.io/mosaicmem/ Paper: huggingface.co/papers/2603.17…

@haofeixu @felixwimbauer @wrchen530 Yeah after the BBQ registration deadline, you can only change their order of the existing authors, but cannot add new ones.

@felixwimbauer @NandoMetzger @wrchen530 Haha but I am sorry that you have to go to the acknowledgements 😅

We had an amazing BBQ last year 🤣 I organized the BBQ, my work here is officially done 😌🔥 (Disclaimer: No Authors or GPUs were actually harmed in the making of this meme… hopefully 😂) @NandoMetzger @wrchen530

Hiring: Student Researcher @ Google Zurich! 🇨🇭 Looking for an EMEA-based PhD student to work on: Efficient 3DGS / Feedforward Models / 3D GenAI. Start: June 2026 or earlier. If interested, please reach out with your CV to: sr-zurich-3d@google.com

@haofeixu @wrchen530 Can confirm, all we did was hyperparameter tuning -- lack of novelty!

ZurichRobotics#3 is next Thursday at the @ETH_AI_Center! Michael Neunert (@GoogleDeepMind) on on efficient models for bringing AI into the real world and Riccardo Maggioni (FORGIS) on building the intelligence layer for the factory floor. RSVP below before spots run out!

@davnords @arnie_hacker @baraqlipshitz Same, never had any bottleneck by doing this

@arnie_hacker @baraqlipshitz Do the workers not preload this on the cpu and coalesce the gpu bottleneck? I have had no slowdowns doing things like this. As long as CPU RAM does not go OOM I think you are fine

Anyone experienced with training video diffusion models? Noob question: Do you pre-process mp4 into individual frames and store before training? Doesn't this blow-up storage requirements? Or do you dynamically convert mp4 into frames during training (how is this parallelized?)

We are having the next Zurich CV event on January 20th with two exciting talks!✨ - Zan Gojcic (NVIDIA) @ZGojcic on Generative Reconstruction - Vanessa Sklyarova (ETH) @ness_pirs on Hairstyle Modelling for Avatars zurichai.ch/events/zurichc… @zurichnlp

Happy to share the stage with @ness_pirs to discuss recent trends in neural reconstruction and video generative models. Do we still need explicit representations in the era of video generation? Or is reconstruction ultimately a strongly conditioned generative problem? Join us!

@kwangmoo_yi Thank you for posting this❤️

Metzger et al., "Elastic3D: Controllable Stereo Video Conversion with Guided Latent Decoding" Turn any video into a stereo video. Single-step diffusion + guided decoder, instead of explicit warping. The decoder does use epipolar geometry for guidance.

I was playing with Apple's SHARP model and extended it with an aperature renderer and an autofocus feature. For this one I used GroundingDINO and a focus priority list (eye->face->person->animal->object). Closer objects are prioritized. Open-source code: github.com/nandometzger/m…

@Parskatt @KyleSargentAI I haven't checked this one yet

@NandoMetzger @KyleSargentAI flux2 vae seems good?

The first lab to open-source a solid pixel-space video diffusion model will be cited a bazillion times by everyone working on inverse problems esp. related to 3D/4D. With JiT/Simple(r) Diffusion the tech is mostly there, someone with more GPUs than me should make it happen (plz)

@KyleSargentAI An pixel space stable diffusion equivalent needs about 64 more flops (and memory). I doubt that will fit on a regular GPU

@KyleSargentAI I agree with the sentiment, but I fear people also underestimate how computationally intense pixel diffusion at inference is. What people really want is an alternative to latent diffusion that truly maintains pixel space properties.

ZurichRobotics#2 is on December 15th at the @ETH_AI_Center! Antonio Arbues @arbwes (@loki_robotics) will talk about building useful robots in the age of AI and Giacomo Manzoni (Hexagon) about imitation learning for dexterous manipulation. RSVP below, spots limited.

I will give a talk at ZurichCV. Already 150 RSVPs in 4 hours! That will be a great event.

Rocking that Spectral colormap shirt🌈

In case you missed the workshop, we will also present Marigold DC in tomorrow's poster session (Tuesday 3pm). Looking forward to talking to you🙌