Pau Gargallo
524 posts

Pau Gargallo
@paugargallo
Reconstructing the world one image at a time at https://t.co/AsIUkJrfBC prev @RealityLabs and @mapillary
Zurich Katılım Kasım 2010
305 Takip Edilen258 Takipçiler

@dhh Awesome! Thanks! Did you manage to turn off the led strip behind the screen? I failed at it and ended up putting black tape on it 😅
English

@DominiqueCAPaul @DominiqueCAPaul, i love following your adventures. Thanks for sharing them openly. We are building staer.ai to drive robot fleets. Would love to meet with Zurich based founders.
English

@DanielMiessler So cool! What is the voice to text tool you are using with CTRL+J at 6:30 in the video?
English

I built a @claudeai skill that it takes any input and converts it into different kinds of art for my site using Nanobanana 3.0.
- Blog header art
- Tech
- Comics
Available for free in our public Personal AI repo!




English
Pau Gargallo retweetledi

Meet MapAnything – a transformer that directly regresses factored metric 3D scene geometry (from images, calibration, poses, or depth) in an end-to-end way. No pipelines, no extra stages. Just 3D geometry & cameras, straight from any type of input, delivering new state-of-the-art results 🚀
One universal model enables SoTA for:
🔥 Mono Depth Estimation
🔥 Multi-View SfM
🔥 Multi-View Stereo
🔥 Depth Completion
🔥 Registration
… and many more possibilities! – plus everything is metric 🎯
We release code for data processing, training, benchmarking & ablations – everything Apache 2.0!
Details & Links 👇
English

@AjdDavison Anything with inputs too large to fit into memory?
English

@iquilezles @DoItRealTime Can you get more phases by rotating the gears wrt each other?
English

@DoItRealTime Tried, but they clash, they are too close. Only have 0, 90, 180 and 270 degree phases available because of the axis grooves - we can't do something fancy like interleaving at 60 degrees.
English
Pau Gargallo retweetledi
Pau Gargallo retweetledi

Gemini + π0 = actually useful robots! (Similar to what @physical_int did with "Hi Robot")
I can now verbally tell the robot that I'm building a red Lego wall or wooden tower, and it will infer the next steps by itself and pass me the necessary pieces, tools, or materials, ha!
You can also just ask it to bring you things!
The pipeline works as follows:
- OpenAI Whisper (local) → speech to text
- Gemini → makes sense of user requests, converts to robot tasks, bounding boxes, grasping points, etc. (System 2 thinking FTW!)
- π0 → robotic actions
The π0 was finetuned just for pick-and-place Lego bricks only, and it generalizes beautifully to all kinds of tasks. However, there's lots of room for improvement when it comes to grasping & accuracy.
Things that could help:
- Conditioning on grasping points
- Better data collection (I'm not that great at teleop)
- Lots more synthetic data and simulations
English
Pau Gargallo retweetledi

Thought about generating realistic 3D urban neighbourhoods from maps, dawn to dusk, rain or shine? Putting heavy snow on the streets of Barcelona? Or making Paris look like NYC? We built a Streetscapes system that does all these. See boyangdeng.com/streetscapes. (Showreel w/ 🔊 ↓)
English

@sellan_s @YouJiacheng Cool thanks! I couldn’t avoid seeing a sphere that was inside the curve in one drawing and outside in another and had to ask 😅
English

@sellan_s @YouJiacheng do you also use the normals during the reconstruction step to ensure that the surface is tangent to the spheres?
English
Pau Gargallo retweetledi

The wait is over 📢 MAST3R is out! DUSt3R+ dense local feature maps & metric depth - 1st in #MapFreeReloc leaderboard, can handle 1000s of images 😀 !!
Blog: shorturl.at/9JTH2
Code: github.com/naver/mast3r
Paper: arxiv.org/abs/2406.09756
English
Pau Gargallo retweetledi

The Quest v64 update brought two undocumented major new features: furniture recognition on Quest 3 and simultaneous hands & controllers in the home space:
uploadvr.com/quest-v64-undo…
English
Pau Gargallo retweetledi

Now available on Mapillary: Neural Radiance Fields (NeRFs)! 🎊
NeRFs allow for the transformation of a collection of 2D images into detailed, immersive 3D reconstructions.
Read our blog post to learn more and see how you can get started with NeRFs: blog.mapillary.com/update/2024/03…
English

(1/2)
We've won an Oscar!!
Dolby Atmos has received the Academy Plaque for Scientific & Technical Achievements, the highest award short of an Oscar statuette, by the Academy of Motion Picture Arts & Science.
youtu.be/h3IyGHbVlMw

YouTube


English







