ud

1.1K posts

ud banner
ud

ud

@uddupa

co-founder @astrm_labs // deep dabbler // hard-tech & bio-hacking

Planet Earth Beigetreten Şubat 2017
1.6K Folgt1.8K Follower
Angehefteter Tweet
ud
ud@uddupa·
micro drones. we cooked a new streaming visual slam @astrm_labs.
English
15
26
364
30.2K
henry22-lab
henry22-lab@henry22lab·
@uddupa I didn’t know you can slam with camera
English
1
0
0
32
ud
ud@uddupa·
start with books. start with your phone cam... u can go a long way with ur phone streaming live video to ur laptop!
LearnerMan@LearnerKJ

@uddupa @astrm_labs crazy , how can i learn and get into SLAM?

English
4
0
23
1.3K
ud
ud@uddupa·
micro drones. we cooked a new streaming visual slam @astrm_labs.
English
15
26
364
30.2K
Pablo Vela
Pablo Vela@pablovelagomez1·
This will eventually be a part of it! There a ton of side models like DAv3/SegmentAnything/ect that also need evaluations that help with slam. But I wanted to focus on a constrained version of things to start. Very cool demo btw =] What depth model are you using? It seems like the fisheye lens makes the depth model struggle some. Might be worth looking at github.com/yuliangguo/dep… or any of the other wider FOV depth models. This is another cool one nam1410.github.io/cam3r/
English
1
0
3
176
Pablo Vela
Pablo Vela@pablovelagomez1·
I've been on a SLAM/SFM kick. It's one of the more underexplored and lacking areas when it comes to human teleop/data collections, so I've brought over Deep Patch Visual Odometry/SLAM to @rerundotio and @Gradio. With this example, we now have 1. pycuvslam 2. pycolmap/glomap 3. mast3r-slam 4. dpvo/slam all integrated into rerun. The question becomes, which method should be used in what situations? They all make different trade-offs with different camera requirements and throughput/accuracy. What about when a new method comes out? Now that I have several different methods, I plan to use VSLAM-LAB for evaluation. It uses @prefix_dev to isolate all the dependencies of each of these methods and easily compare them against each other. In particular, I'll be converting the data preprocessing, algorithm outputs, and evaluation into rerun recordings (rrd files). This will allow both programmatic querying of anything stored in the files (which method had the highest ATE-to-FPS ratio? Which dataset/sequence caused the most difficulty? etc. etc.), all with easy visual inspection using the rerun server to link them all together. Another really important side effect of this is how it impacts agents. As Karpathy said ``` LLMs are exceptionally good at looping until they meet specific goals, and this is where most of the "feel the AGI" magic is to be found. Don't tell it what to do, give it success criteria, and watch it go. ``` by having accuracy and throughput metrics deeply tied with human inspectable artifacts. One can really accelerate agentic development with an actual understanding of how the method/data performs. I think this is another killer use case that I'll be really leaning into to make ingestion of new datasets/methods trivial with an agent. I'm making it my mission for folks to understand that rerun as a visualization tool only scratches the surface of what its true benefit is. Deep integration between data and visuals, with powerful query capabilities. I'll be focusing on the SLAM use case first and then bringing this into the full egocentric/exocentric data collection domain!
Pablo Vela@pablovelagomez1

I've migrated the old Mast3r-SLAM example I had made last year to the latest version of @rerundotio and made a bunch of improvements! I wanted to spend some time with agents to modernize it. Here's an example of me walking around with my iPhone and getting a dense reconstruction at about 10FPS on a 5090. Heres the following improvements I made. Brought it into the monorepo with proper packaging: • Using @prefix_dev pixi-build to get rid of all the mast3r/asmk/lietorch vendored code with just a few small patches. This let me remove so 60k lines of code from the repo! • Don't have to build the lietorch code on my machine anymore, which was taking ~10 minutes to compile (and also made it work on blackwell when it previously did not) Rebuilt the @Gradio interface: • Fixed incremental updates, .MOV uploads, and stop behavior • Made the CLI + Gradio interface share the same entry point so updates automatically propagate Upgraded the @rerundotio integration: • Switched to a multiprocessing async logging strategy • Added video/pointmap/confidence logging • Improved blueprint layout and hid noisy entities from 3D view • Biggest perf win was the async background logger - documented about a ~2.5x speedup from decoupling logging from tracking The newest and most interesting part was my attempt to replace the CUDA kernels for Gauss-Newton ray matching with a @Modular Mojo backend. As a Python dev, every time I look at CUDA code I basically shy away as it's pretty difficult for me to understand. Mojo let me rewrite the matching logic in a syntax I'm more comfortable with while still getting near-CUDA performance. Mojo is now the default matching backend with CUDA fallback. One major piece that's missing is the custom PyTorch op path, but I'll eventually do that as well. I heavily leaned on Claude Code to do the CUDA → Mojo migration, and I have no doubt it's not the cleanest or most idiomatic, BUT it's way more readable for me and helps me better understand the underlying algorithm. This was a ton of work, and a large part of why I'm doing it is how the monorepo compounds. This becomes an artifact for the next example I want to build with Claude that I can point to, which will make it even faster to implement. The compounding nature of this is really interesting and part of why I'm spending so much time trying to make things nice and readable.

English
6
23
263
19.1K
ud
ud@uddupa·
@annanimous4 @astrm_labs not super crazy possibility... we are on it. :) abt range... depends a lot on how crowded the env is, how big is drones battery, etc.
English
0
0
2
176
Anything
Anything@annanimous4·
@uddupa @astrm_labs does it have any certain range for signal to the operator when it actually has to fly with a guy doing it or idk adding a automatic flying by the drone analyzing by itself and flying would be crazy(possibility is there ig) ,but yeah.
English
1
0
1
308
ud
ud@uddupa·
@duke000083 @astrm_labs great idea! thx for inspiration. btw... u have warehouse for testing?
English
0
0
1
144
TheDuke
TheDuke@duke000083·
@uddupa @astrm_labs Throw 5 micro drones with object avoidance and map a warehouse in 30 minutes
English
1
0
1
161
ud
ud@uddupa·
@ben_sdl @astrm_labs we wrote a custom pipeline for occupancy grid based mapping...
English
1
0
3
209