Shivangi

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions (#SIGGRAPH) We reconstruct ultra-high fidelity photorealistic 3D avatars capable of generating realistic and high-quality animations including freckles and other fine facial details. We operate on patch-based local expression features and increase the representation capacity by synthesizing 3D Gaussians dynamically by leveraging tiny scaffold MLPs conditioned on localized expressions. We further propose a color-based densification and progressive training scheme for improved quality and faster convergence. Project: shivangi-aneja.github.io/projects/scaff… Video: youtu.be/VyWkgsGdbkk Great work by @shivangi2201, Sebastian Weiss, Irene Baeza, Prashanth Chandran, Gaspard Zoss, Derek Bradley

📢📢 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐒𝐩𝐞𝐞𝐜𝐡: Audio-Driven Gaussian Avatars 📢📢 We synthesize photorealistic and 3D-consistent talking human head avatars driven directly from spoken audio. More specifically, we introduce an efficient 3DGS-based representation, combined with an audio-conditioned transformer, to generate realistic and high-quality animations. Our method can easily generalize to a variety of in the wild scenarios like songs, foreign languages etc. We further introduce a new multiview dataset of native English speakers with overall recording time of ∼3.5 hours. Project: shivangi-aneja.github.io/projects/gauss… Video: youtu.be/2VqYoFlYcwQ Great work by @shivangi2201, @ASevastopolsky, @TobiasKirschst1, @JustusThies, @angelaqdai

ScaffoldAvatar: High-Fidelity Gaussian Avatars with Patch Expressions (#SIGGRAPH) We reconstruct ultra-high fidelity photorealistic 3D avatars capable of generating realistic and high-quality animations including freckles and other fine facial details. We operate on patch-based local expression features and increase the representation capacity by synthesizing 3D Gaussians dynamically by leveraging tiny scaffold MLPs conditioned on localized expressions. We further propose a color-based densification and progressive training scheme for improved quality and faster convergence. Project: shivangi-aneja.github.io/projects/scaff… Video: youtu.be/VyWkgsGdbkk Great work by @shivangi2201, Sebastian Weiss, Irene Baeza, Prashanth Chandran, Gaspard Zoss, Derek Bradley

📢📢 𝐆𝐚𝐮𝐬𝐬𝐢𝐚𝐧𝐒𝐩𝐞𝐞𝐜𝐡: Audio-Driven Gaussian Avatars 📢📢 We synthesize photorealistic and 3D-consistent talking human head avatars driven directly from spoken audio. More specifically, we introduce an efficient 3DGS-based representation, combined with an audio-conditioned transformer, to generate realistic and high-quality animations. Our method can easily generalize to a variety of in the wild scenarios like songs, foreign languages etc. We further introduce a new multiview dataset of native English speakers with overall recording time of ∼3.5 hours. Project: shivangi-aneja.github.io/projects/gauss… Video: youtu.be/2VqYoFlYcwQ Great work by @shivangi2201, @ASevastopolsky, @TobiasKirschst1, @JustusThies, @angelaqdai

🚀🚀🚀Announcing our $13M funding round to build the next generation of AI: 𝐒𝐩𝐚𝐭𝐢𝐚𝐥 𝐅𝐨𝐮𝐧𝐝𝐚𝐭𝐢𝐨𝐧 𝐌𝐨𝐝𝐞𝐥𝐬 that can generate entire 3D environments anchored in space & time. 🚀🚀🚀 Interested? Join our world-class team: 🌍 spaitial.ai #GenAI #3DAI

(1/2) Check out "FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models"! We synthesize high-fidelity motion sequences of neural parametric head models from audio input. Works with music and various languages :) youtu.be/7Jf0kawrA3Q shivangi-aneja.github.io/projects/facet…