Anton Matrosov

If you ever wondered what it would feel like to look through your digital screen - I've created a small prototype of that! cognitive.plus/apps/3dgs-para… 🔸Launch it on your laptop or tablet 🔸Start Head Tracking mode 🔸Enjoy! #PlayCanvas #3DGS #GaussianSplatting More details below👇

@lanamolx Runway, Huggsfield, Freepik - most popular ones atm

@antmatrosov Unlimited plan for seeance 2 impossible!! Where?

⚡️Hacking Seedance 2.0 for the Image to Gaussian Splat workflow 3DGS scene: superspl.at/scene/1420f92d 15sec video -> Postshot MCMC 1M 90k SSIM 0.94 Video renders: SuperSplat Editor Details below👇 #3dgs #gaussiansplatting #seedance #aivideo #supersplat #playcanvas #lovecraft

@lanamolx This is niche technical exercise to generate a small explorable 3D world from a single starting image. It's a hot research topic right now, so I've tried my hands at it as well 👌

@antmatrosov Can’t understand how this will be helpful what u did exactly and why

@Max_Eskandari I'm pretty bad with 3D modelling, so I've used a simpler image-to-3DGS pipeline as the first iteration (huggingface.co/spaces/gagndee…), then designed my camera path there, applied position-to-color shader + placed dots planes manually, and it was done

@antmatrosov This is mind-blowing! Would you mind explaining the motion reference part and how you got that a bit more?

@Max_Eskandari So you have to create a very crude 3D representation of your scene first -> then design a camera path -> render it into 15 sec video -> feed that into Seedance -> enjoy the result :)

@Max_Eskandari Sure! Originally motion references appear in Runway's guide to Seedance: shorturl.at/2CLlc Simple motions like orbiting can be done with abstract 3D cubes and stuff, but 15-second flyarounds between particular points of your scene are tricky

@_trueuser_ Yep, that's correct 👌

@antmatrosov Do I understand correctly that you extracted 3D from the generated video?

And finally, you render your reference, pass it to the model, and hope and pray it will actually respect it and give you a somewhat decent result before you spend all you savings on this slot machine hehe [3/3]

🔸The most challenging part is preparing a motion reference video. First, you need to come up with some initial 3D representation of your scene, even before you have it in 3DGS. I've used ML Sharp for this + custom splat shader + dots overlay to improve motion tracking [1/3]

Then you design a camera path that covers your scene the best way possible, from all angles, in under 15 sec. Realistically you can squeeze in around ~30 distinct view points, and then connect them with some spline path [2/3]

🔸Seedance 2.0 is capped at 15 seconds, which gives you only about 360 frames to work with. For this task, you will need to use the Omni model, which can take both images and videos as input, along with a text prompt describing how those inputs should be used

@JulienReszka When I first saw results of my head tracking, I've asked myself the same thing!

@antmatrosov why aren't fps leveraging this

If you ever wondered what it would feel like to look through your digital screen - I've created a small prototype of that! cognitive.plus/apps/3dgs-para… 🔸Launch it on your laptop or tablet 🔸Start Head Tracking mode 🔸Enjoy! #PlayCanvas #3DGS #GaussianSplatting More details below👇

@the8thwall @playcanvas 4⃣: ?mode = all Enable both World and Head Tracking on your device. By default Head Tracking is enabled only on laptops and tablets, and World Tracking - on tablets and mobiles

@the8thwall @playcanvas 2⃣: ?blur or ?faceblur = true | 0 ... 3. Enable it if you're shy and want to record a video, but with your face blurred out. Alternatively, you can always close face cam with a dedicated close button 3⃣: ?fs or ?fullscreen = true. Enable full screen (requires tap / click)

@willeastcott @playcanvas Looks very nice! And just under 5MB!!!

The @PlayCanvas logo as a fluffy 3D Gaussian Splat! (made by Sean Wilson from Epigraph). 😍 superspl.at/view?id=db6ab6…

@ValigurskyM @playcanvas Looks phenomenal!

The initial version of our new render-passes based DOF (Depth of Field) shader for @playcanvas. Try it here: engine-mq30726c9-playcanvas.vercel.app/#/graphics/dep…