Pratul Srinivasan

🎥 What if 3D capture could gracefully handle moving scenes and varying illumination? 🎯Come see how video models generate exactly the data you need at our poster, SimVS! 📍CVPR, June 14th (afternoon), Poster #60.

⚡️ Introducing Bolt3D ⚡️ Bolt3D generates interactive 3D scenes in less than 7 seconds on a single GPU from one or more images. It features a latent diffusion model that *directly* generates 3D Gaussians of seen and unseen regions, without any test time optimization. 🧵👇 (1/9)

🚀 Introducing SimVS: our new method that simplifies 3D capture! 🎯 3D reconstruction assumes consistency—no dynamics or lighting changes—but reality constantly breaks this assumption. ✨ SimVS takes a set of inconsistent images and makes them consistent with a chosen frame.

We’ll be presenting NeRF-Casting at SIGGRAPH Asia next week! NeRF-Casting enables photorealistic rendering of scenes with highly reflective surfaces—something that was previously impossible with models like Zip-NeRF and 3DGS. (1/6)

(1/N) Flash Cache: Reducing Bias in Radiance Cache Based Inverse Rendering Website: benattal.github.io/flash-cache/ tl;dr our #ECCV2024 (oral ✨) paper presents a new system for inverse rendering that is more physically accurate, and therefore less biased, than existing approaches.

I'm going to present our work at the oral session tomorrow (Wednesday), 9am at #CVPR2024. Come check it out and hang out at the poster session (ours is number 399) immediately after!

IllumiNeRF lets you relight objects in 3D. Instead of the classical inverse rendering approach — disentangling the object geometry, materials, and lighting — we use a relighting diffusion model to relight each input image and distill the relit samples into 3D by optimizing a latent NeRF. This project was led by the talented @xmzhao_ who is completing his PhD and is currently on the job market!

Wondering how to easily relight an object? Inverse rendering, maybe the first thing that comes to mind, is brittle and expensive due to differentiable Monte Carlo rendering. Check out IllumiNeRF for simple, effective 3D relighting without it! illuminerf.github.io (1/n)

IllumiNeRF enables relighting without expensive inverse rendering. We use a diffusion model trained to relight a single image, and turn its samples into a consistent 3D relit NeRF. With @xmzhao_ (currently on the job market!) @_pratul_ @KeunhongP @rmbrualla @philipphenzler

IllumiNeRF 3D Relighting without Inverse Rendering Existing methods for relightable view synthesis -- using a set of images of an object under unknown lighting to recover a 3D representation that can be rendered from novel viewpoints under a target illumination

Check out CAT3D! Image(s)-to-3D in 1 minute! cat3d.github.io Given any number of real or generated images, CAT3D uses a multi-view diffusion prior to create consistent novel views. These views are used to reconstruct a 3D scene using NeRF/3DGS.

The cat’s out of the bag! Image(s) to 3D in one minute with a multi-view diffusion model. cat3d.github.io

Videos are cool and all...but everything's more fun when it's interactive. Check out our new project, ✨CAT3D✨, that turns anything (text, image, & more) into interactive 3D scenes! Don't miss the demo!! cat3d.github.io

🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models. w/ lovely coauthors @holynski_, @poolio and an amazing team!

Scientists, led by a team at Caltech, used AI and telescope data to create the first 3D video of mysterious bright flares around the supermassive black hole at the center of our galaxy. caltech.edu/about/news/ai-…

We just finished a joint code release for CamP (camp-nerf.github.io) and Zip-NeRF (jonbarron.info/zipnerf/). As far as I know, this code is SOTA in terms of image quality (but not speed) among all the radiance field techniques out there. Have fun! github.com/jonbarron/camp…

Introducing Eclipse, a method for recovering lighting and materials even from diffuse objects! The key idea is that standard "NeRF-like" data has all we need: a photographer moving around a scene to capture it causes "accidental" lighting variations. dorverbin.github.io/eclipse/ (1/3)

NeRFs are cool, but they're HARD to edit. Turn 'em into a mesh, and the geometry and UV maps are a dumpster fire -- making simple texture editing mission impossible for your 3D artist. Well, not anymore! Google AI's latest paper Nuvo, employs neural fields for UV mapping, letting you edit cleanly parameterized chunks of the model. Benefit? Now you can use 2D editing tools like Photoshop or Firefly in-painting to edit the texture of a NeRF model :) It doesn't even matter if it's a real life scan or a generated one. This technique does a great job of dealing with the chaotic (but good looking) geometry you get out of NeRFs.

🚀 Exciting news from Google Research! Their latest innovation, Nuvo, is changing the game in UV Mapping. Nuvo tackles complex geometries with neural field-based UV mapping, paving the way for more flexible and editable NeRFs and Generative outputs. 🔗neuralradiancefields.io/nuvo-revolutio…

@rohansawhney1 @nmwsharp @keenanisalive Thanks for the pointer, this paper looks super cool! Would love to try it out on our NeRF meshes :-)

@_pratul_ Nice work! Xatlas produces way too many segments in my experience as well. As another point of comparison, you might also want to check out @nmwsharp & @keenanisalive's variational surface cutting algorithm: cs.cmu.edu/~kmcrane/Proje… Link to code: github.com/nmwsharp/varia…

Nuvo: Neural UV Mapping! It's super difficult to UV map/texture atlas geometry produced by 3D reconstruction and generation pipelines. Nuvo works on all kinds of "unruly" 3D representations (NeRF, DreamFusion, etc.) and enables easy appearance editing! pratulsrinivasan.github.io/nuvo 1/3