Linning XU

[1/3] Want to capture a fantastic HDR image by 2 simple shots with your cellphone? Try UltraFusion HDR. It takes two images with exposure differences up to 9 stops, and it robustly generates HDR output. Try your own captured images (supporting 4Kx3K): huggingface.co/spaces/iimmort…

Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering paper page: huggingface.co/papers/2312.00… Neural rendering methods have significantly advanced photo-realistic 3D scene rendering in various academic and industrial applications. The recent 3D Gaussian Splatting method has achieved the state-of-the-art rendering quality and speed combining the benefits of both primitive-based representations and volumetric representations. However, it often leads to heavily redundant Gaussians that try to fit every training view, neglecting the underlying scene geometry. Consequently, the resulting model becomes less robust to significant view changes, texture-less area and lighting effects. We introduce Scaffold-GS, which uses anchor points to distribute local 3D Gaussians, and predicts their attributes on-the-fly based on viewing direction and distance within the view frustum. Anchor growing and pruning strategies are developed based on the importance of neural Gaussians to reliably improve the scene coverage. We show that our method effectively reduces redundant Gaussians while delivering high-quality rendering. We also demonstrates an enhanced capability to accommodate scenes with varying levels-of-detail and view-dependent observations, without sacrificing the rendering speed.

Combine @googlemaps 3d tiles and @LumaLabsAI gaussian splat scans via @aframevr / @threejs for high fidelity at tiny scale AND zoomable out to city scale all in the browser. 🙏 @glitch hosting, @playcanvas splat editing, @NYTimesRD 3dtiles component maps-and-splats.glitch.me

#ICCV2023 We curated a large-scale, comprehensive, and high-quality synthetic dataset MatrixCity for city-scale neural rendering based on the UE5 City Sample. @ICCVConference -Project: city-super.github.io/matrixcity/ -Paper: arxiv.org/abs/2309.16553

Introducing VR-NeRF (@SIGGRAPHAsia 2023): 🤩 multi-camera rig for multi-view HDR capture 🤓 perceptual HDR optimization + level of detail 😎 real-time multi-GPU VR rendering Project: vr-nerf.github.io Paper: arxiv.org/abs/2311.02542 Dataset: github.com/facebookresear…

#ICCV2023 We curated a large-scale, comprehensive, and high-quality synthetic dataset MatrixCity for city-scale neural rendering based on the UE5 City Sample. @ICCVConference -Project: city-super.github.io/matrixcity/ -Paper: arxiv.org/abs/2309.16553

How to build a NeRF from a building to the whole planet? We study such extreme multi-scale scene modeling in our #ECCV2022 paper BungeeNeRF (aka CityNeRF), inspired by movie effects. Webpage: city-super.github.io/citynerf/ Paper: arxiv.org/abs/2112.05504 Code: github.com/city-super/Bun…

YouTube

The new @googlemaps Immersive View feature is going to be pretty amazing (and uses Neural Radiance Fields, or NeRFs, developed by @GoogleAI, UCBerkeley and UCSD researchers) See arxiv.org/abs/2003.08934 Watch Maps Immersive portion of #GoogleIO keynote: youtu.be/nP-nMZpLM1A

A new fun quick short story on AI: "Forward Pass" 🧠✨ karpathy.github.io/2021/03/27/for…

Introducing mip-NeRF! By casting cones instead of rays, and by positionally encoding volumes instead of points, mip-NeRF (à la "mipmap") fixes big problems NeRF has with aliasing and scale. 60% lower error than NeRF and 22x faster than supersampling NeRF. jonbarron.info/mipnerf

Image Generators with Conditionally-Independent Pixel Synthesis pdf: arxiv.org/pdf/2011.13775… abs: arxiv.org/abs/2011.13775