Tim Field

Building a software stack for humanoid robots. Realtime 3D mapping working. Next: navigation.

@JacobZietek @chris_j_paxton unitree.com/G1-D

@chris_j_paxton where is this? can’t find it on the unitree site

Its honestly great that they're doing this and will make it way easier to learn foundation models for manipulation. Shows the transition unitree is undergoing, and that they're taking the challenge of learning for robot manipulation very seriously.

@jon_barron Sure (see video) but this isn't their fine-tuned metric depth model – output's unitless with range close to [0, 1] every frame. IMO fitting the depth image to VIO feature points would be more fruitful than asking a monocular system to recover accurate metric depth.

@nobbis Can you render a video where the range of depths used for the colormap is fixed across frames, instead of the min/max of the depths of each frame? Or is that what you rendered here, and it's just temporally unstable?

Trying out realtime video-to-depth on an iPhone 15 using Depth Anything V2.

@Kellyv_ai Not built-in. I wrote a few lines to filter the depth image through a colormap.

@nobbis I successfully ran it on the iPhone 12 Pro Max, but why is it in black and white? How do I set it to a colored depth map?

A lot better than my depth-from-motion algorithm from 2019. x.com/aboundlabs/sta…

Paper: arxiv.org/abs/2406.09414 by Lihe Yang, @bingyikang et al. Model: huggingface.co/depth-anything App: github.com/huggingface/co… by @pcuenq

@coder543 @bingyikang @pcuenq V2 – use the depth-anything-v2 branch

@nobbis @bingyikang @pcuenq So are you using depth anything V1 or V2? The sample app is V1, and I haven’t seen anyone publish a V2 coreml conversion yet. I was thinking about trying to do that conversion soon.

@gravicle @jon_barron @karanganesan @vokaysh @HaiperGenAI @LumaLabsAI Got it. Haiper restricts unlimited usage by capping concurrent creations to 10. Could be a good option for you guys too.

@nobbis @jon_barron @karanganesan @vokaysh @HaiperGenAI @LumaLabsAI unlimited tiers tend to kick in hidden limits we found (tho we didn't test Haiper). Compare to runway and pika.

@gravicle @jon_barron @karanganesan @vokaysh Curious how you calculate 3-5x cheaper? @HaiperGenAI is $288.00/yr for 4s videos vs @LumaLabsAI at $287.90/yr for 5s videos. Both 24fps, Luma is 10% higher resolution, but Haiper is unlimited. (Congrats on the launch BTW.)

@jon_barron @karanganesan @vokaysh 3-5x lower per sec than others AND more happens in the seconds Dream Machine generates. AI is about abundance. Until we can offer unlimited, we will try to offer as much as possible.

@iwamah1 I believe Area Mode is the new "ignoreBoundingBox" processing option in Object Capture. The documentation says: "Ignores any bounding box information embedded in the input images and instead returns all possible geometry that can be automatically estimated using the image set."

今年も...今年もApple純正3Dスキャン機能が強化されてる...！ 予想通り物から空間に変わってきましたね Object Captureのエリアモード...一体どんな仕組みなんだ⁉️ #WWDC

@iwamah1 We deliberately keep the price unchanged for continuing Metascan subscribers as a way of thanking early supporters. If you change your subscription, then you'll pay the current price. App Store developers can increase prices by up to 50% once a year (see support.apple.com/en-us/109501)

MetascanとPolycamをサービス初期から永遠とサブスクしている訳なんですが、AppStoreの表示金額が何故か初期金額から変更されていないの謎すぎる 内部処理的にどうなっているんですかね...？

@shlykur @Metascan3D It would be great to see – but can't think of how to do that (short of spamming everyone's email.) Adding a RFS (Request for Scan) feature could be interesting. Not sure how it'd work.

Is there a way to request a scan from @Metascan3D users network to get a raw scan of an AVP? it would be great to see one up close in 3d.

@bilawalsidhu I bet someone will replace the SHs in 3DGS with a tiny MLP, keep the rest, and say, "Look, it was always a NeRF!"

@bilawalsidhu Turns out NeRFs were just a bunch of overly complicated tricks (positional encoding, space warping, coarse-to-fine, hashtables, view-dependent color with tiny neural nets, cone tracing, etc.) that aren't necessary.

"It's just differentiable rendering. Been doing that since Neural Volumes/NeRF in 2019." - CV folks "It uses seed & diffuse 3D reconstruction. PatchMatch Stereo showed that in 2011." - MVS folks "Hah, it draws Gaussian splats! Try EWA splatting in 2002." - CG folks #3DGS

@rubenfro Strong Terminator 2 vibes.

Burning Gaussian Splat in Unity3D.

@Azadux @BartronPolygon Yeah, 3DGS “the software” must be licensed, but a clean room implementation of the algorithm is fine. In practice, startups basically ignore that, e.g. we know Luma employees are looking at 3DGS because they were digging into Polycam’s fork of it.

@BartronPolygon I assumed because of this twitter.com/nobbis/status/…

2nd product that incorporated (licensed?) Gaussian splats! It’s spreading quite quickly

@gravicle @Azadux Congrats on hiring Jakub recently to get hold of gsplat.tech - was wondering who'd scoop up that viewer.

@Azadux Nope! Our implementation built on top of the whole Luma pipeline.

@antimatter15 @JulienBlanchon Nice - makes it easier. I assumed you were using threads, for some reason.

@nobbis @JulienBlanchon I don't use SharedArrayBuffers in my implementation- just transferable objects.