Radek Daněček

1/n Proud to present our latest attempt to give 3D faces a voice: “THUNDER: Supervising 3D Talking Head Avatars with Analysis-by-Audio-Synthesis” We synthesize audio from facial motion! Website: thunder.is.tue.mpg.de With @Michael_J_Black, Senya Polikovsky, Carolin Schmitt

NeuralFur wins Best Paper Runner Up at @3DVconf. From multi-view images, we create a strand-based hair groom for animals. Unlike human hair, fur varies in length across the body parts of animals. NeuralFur leverages a VQA approach to infer fur lengths and directions across the body and to create a furless mesh. We then reconstruct strand-based fur geometry from multi-view images, resulting in a realistic animal model that is ready for physics-based animation in game engines like Unreal. Code is online. Check out the project page link below. Congratulations to @ness_pirs @bernakabadayi @AYiannakidis @gfgbec and @JustusThies! neuralfur.is.tue.mpg.de

I will be presenting our paper THUNDER tomorrow (March 21st) at @3DVconf. Session 3, poster 26 We synthesize audio from facial motion and use it to improve speech-driven avatars! Project page: thunder.is.tue.mpg.de With @Michael_J_Black, Senya Polikovsky, Carolin Schmitt

I'm at 3DV in Vancouver this weekend. Feel free to connect!

Happy to share that our THUNDER paper got accepted to 3DV!

@Saikat_Roy_ @ICCVConference I would very much like to know this as well.

@ICCVConference What about reviewers who didn't submit a final recommendation?

Following #CVPR2025, #ICCV2025 implemented a new policy targeting accountability and integrity. PCs identified 25 highly irresponsible reviewers, resulting in the desk rejection of 29 associated papers, including 12 submissions that otherwise would have been accepted.

A neat way to perform single image relighting without explicit 3D geometry or light transport simulation using a video diffusion model.

Decades of computer vision research and somehow human-object contact reconstruction hasn't been working ... until now.

A neat way to use a VLM to improve human-object contract reconstruction!

@jack_r_saunders @Michael_J_Black I did try to just optimize to produce a certain lip animation. I.e. start with a still animation and produce the audio and just optimize until you get the audio right - this worked, but you need a motion prior or else you end up advisarially hacking mesh-to-speech.

@jack_r_saunders @Michael_J_Black But we also have variants that take the flame expression vectors or all vertices. They might have a better chance of working in this setting.

1/n Proud to present our latest attempt to give 3D faces a voice: “THUNDER: Supervising 3D Talking Head Avatars with Analysis-by-Audio-Synthesis” We synthesize audio from facial motion! Website: thunder.is.tue.mpg.de With @Michael_J_Black, Senya Polikovsky, Carolin Schmitt

This is such a cool idea and it's amazing that it works. To train a generative talking 3D head model to have good lip sync, perform mesh to speech generation on the output. Compare this with the input speech and use the difference in your training loss. It's a neat trick and, even though the mesh to speech generation isn't perfect, it still works. You can apply this idea to any of the current methods for training speech to 3D animation and get improved results.

@TuliMathieu @Michael_J_Black I am curious - what exactly was your setting and data?

@Michael_J_Black @DanecekRadek Yeah I echo that - we tried with FLAME but our solution was quite brittle - very cool that it works

@TuliMathieu @Michael_J_Black Thanks! A big chunk of credit goes to lip2speech-unit. github.com/choijeongsoo/l… They did a very good job and their model serves as our starting point.

@DanecekRadek @Michael_J_Black Wow super cool, we tried to do this last year but couldn’t get it right, very nice!

13/n In summary, we first design a novel mesh-to-speech model which can turn facial animation into speech. And then we show the mesh-to-speech model can be used for “analysis-by-audio-synthesis”, producing high quality lip animations. Check our thunder.is.tue.mpg.de for more

12/n In addition to THUNDER, we also demonstrate the effectiveness of mesh-to-speech and analysis-by-audio-synthesis in other systems, such as determinstic FaceFormer (or FF) From left to right: FF-frozen, F-frozen with m2s, FF-trainable, FF-trainable with m2s