Daniel DeTone

@nickkarpov Feel free to file a GitHub issue if you have any problems! Will do my best to answer them quickly

@ddetone Awesome work, going to use this

Today we release Boxer, a new lightweight approach that lifts open-world 2D bounding boxes to *metric* 3D: facebookresearch.github.io/boxer/ Here we show Boxer in action on an egocentric sequence captured from smart glasses:

For more details, check out the arxiv paper here: arxiv.org/abs/2604.05212

BoxerNet runs FAST 🔥🔥, taking roughly 20ms on a 4090 with bfloat16 for ALL prompts in an image (e.g. 30 boxes in parallel)

Giving Claude new skills is the closest thing I’ve felt to this Matrix moment

Check out our recent work ShapeR, a generative model which gets high quality object 3D meshes from Aria glasses

For those interested in 3D perception, check out the Sonata pre-trained backbone. I don’t think I’ll ever co-author another paper that triples performance (22% -> 72%) on a commonly used benchmark (linear probe scannet 3D sem seg). Released with a permissive license too

📢Sonata: Self-Supervised Learning of Reliable Point Representations📢 Meet Sonata, our"3D-DINO" pre-trained with Point Transformer V3, accepted at #CVPR2025! 🌍: xywu.me/sonata 📦: github.com/facebookresear… 🚀: github.com/Pointcept/Poin… 🔹Semantic-aware and spatial reasoning representations learned with no label; 🔹3x linear probing accuracy (from 21.8% to 72.5%) on ScanNet; 🔹2x data efficiency performance with only 1% of the data compared to previous approaches; 🔹As always, establish new SOTA results across indoor and outdoor 3D perception tasks. Our author team: @HengshuangZhao, @jstraub6, @rapideRobot, @ddetone, @NinjaDuncan, @TianweiS, @Christopher_Xie, @NanYang719.

The first generation of Aria glasses have made a big impact in the research community, can't wait to see all the new possibilities these will unlock meta.com/blog/project-a…

@ZMurez @gazorp5 heres more details on the EVL model you were asking about

cc @ZMurez remember 3D surfaces? :)

What is the right benchmark for a 3D Egocentric Foundation model? We recently open-sourced a small, high quality egocentric benchmark consisting of 1) 3D surfaces 2) 3D objects. We released a simple 3D CNN baseline model called EVL: projectaria.com/research/efm3d/ Try to beat our model!

@gazorp5 yes, will make a post on that today

@ddetone Do you plan on releasing the EVL model as well?

Interested in 3D object detection and egocentric vision? We open sourced a small yet challenging dataset called Aria Everyday Objects (AEO). We use it as one of the tasks for benchmarking a new class of Egocentric 3D Foundation Models we are working on: arxiv.org/abs/2406.10224

this work was done in collaboration with @jstraub6, @tianweis , @NanYang719,@the_shweenz, @rapideRobot

for more details on the dataset, check out projectaria.com/datasets/aeo/