we open-sourced the capture stack for Egocentric RGB-D data
Now anyone can build their own stack on top!
FPV Labs@fpv_labs
English
Shubhanshu Khatana
159 posts
Shubhanshu Khatana
@hatebunnyplzzz
building @fpv_labs, ex-Builder @Lossfunk, ex-LCS2 @IITD, ex-@SHLglobal Robots--Vision--RL
Katılım Mayıs 2021
396 Takip Edilen130 Takipçiler
@DozenDucc @yiding_song @theWaddleLabs is the idea inspired by Cap-X study from NVIDIA? looks impressive
Interested to see what is the performance ceiling on some close loop evaluations including approaches involving world models as well. or even simulation like RoboCasa365
English
Introducing Waddle Labs: Claude Code for robots.
Connect our API to your robot and enter a prompt, then our agents write code to achieve the task in 20 minutes.
@yiding_song @theWaddleLabs
English
Vidur is out!!
Best in the world at turning raw video into dense, timestamped action labels. every metric, both benchmarks, against every frontier model we tested.
FPV Labs@fpv_labs
1/ Announcing Vidur, an end-to-end system to generate fine-grained action labels on raw robot and human videos. Vidur leads across all metrics on WGO and EgoExoLearn Bench across temporal segmentation, semantic precision, semantic recall, and end-to-end action-label quality 🧵
English
@paradite_ a stanford study found llm-generated research ideas were judged **significantly more novel** than those from human experts, though slightly less feasible. LLMs already help frontier ideation via recombination + exploration at scale. ctstate.edu/images/Forms-D…
English
Shubhanshu Khatana retweetledi
Ever since we released Stera, the question we kept hearing was the same: Is an iPhone’s tracking actually good enough? So instead of arguing about it, we decided to test ARKit poses against ground truth.
We benchmarked an iPhone Pro running ARKit against a 24-camera Vicon optical motion-capture system (sub-millimeter accuracy at 200 Hz), across eight sequences spanning motions that dominate real egocentric tasks like walking, manipulation, fast motion, in-place and aggressive rotation, height change, and more.
Across all eight sequences, absolute trajectory error (ATE) ranged from 6.0 to 12.5 cm while translational RPE remained under 5 cm throughout.
Our takeaway - the barrier to high-fidelity egocentric data might be lower than everyone assumed. It doesn’t require a research-grade rig or a device most people will never own. It can be collected with commodity hardware, substantially lowering the barrier to large-scale data for robotics.
The full numbers, details of every sequence, and how we measured them against ground truth are in the essay below 👇
English
Am I the only one who is so generalist-pilled?
Like even while hiring for deep AI roles, I look for 'AI Generalists' - folks who can do interpretability one week or RL the next.
Where do I find such people in India?
English
Best Conference Paper Award and the Best Paper Award on Robot Manipulation and Locomotion at #ICRA2026!
Guanya Shi@GuanyaShi
High-quality motion reference data is key for humanoid skill learning 🤖🕺💃 A natural idea is to leverage human motions and “translate” them to humanoid motions, a process known as retargeting. For interaction-rich tasks such as scene interaction and loco-manipulation, retargeting is challenging: it must ensure motion consistency, smoothness, kinematic feasibility (no artifacts like penetration or foot skating), and scalability (one framework can handle thousands of motions). Excited to release OmniRetarget — a scalable retargeting method with a 4-hour high-quality humanoid motion dataset for interaction-rich tasks. OmniRetarget takes an interaction-preserving perspective: we optimize Laplacian deformation between source and target interaction meshes while enforcing kinematic constraints, producing consistent, smooth, and feasible trajectories at scale. Even better, OmniRetarget can efficiently augment motions by varying terrains, objects, and initial poses. This high-quality interaction-preserving retargeting enables a minimal RL setup to execute long-horizon (up to 30s) agile, interaction-rich skills. All tasks in the video share just 5 rewards, 4 domain randomization terms, and rely only on proprioception. More details: omniretarget.github.io
English
Excited to announce we are presenting our recent work "MobileEgo Anywhere" ICRA 2026 Beyond-Teleop Workshop poster-presentation.
#ICRA2026 #Egocentric #Robotics
FPV Labs@fpv_labs
Can a commodity smartphone replace a $10,000 data rig? We're presenting MobileEgo Anywhere at ICRA 2026 - open infrastructure for collecting hour-long egocentric trajectories on commodity hardware for downstream robot policies. 📍 Strauss 3 | 🕒 3:00 - 4:00 PM today Come say hi 👋
English
@HappyyPablo on solving one of the biggest bottleneck in video pseudo-labelling for all the downstream tasks from large scale video understanding to training instruction-tuned robot policies!
Shubham Sharma@HappyyPablo
open sourcing Marlin-2B 🐟 a tiny VLM to extract structured information from videos Marlin is finetuned for two questions devs want to ask in their videos: what is happening, and when? Best open model in its weight class, competitive with Gemini-2.5-flash at only 2B params 🧵
English
Shubhanshu Khatana retweetledi
Dedicating the first one to @ParasChopra and @lossfunk for cultivating a research-first mindset during my residency days - knowingly or unknowingly, we are carrying over that culture at @fpv_labs
English
@PrimeIntellect great step towards "autonomous training"
English
The next step toward automating AI is automating RL environments
Introducing General-Agent: A fully synthetic environment whose task corpus self-evolves and grows harder over time
4,504 tool-use tasks · 1,040 domains · 8,159 unique tools
GIF
English
@SpencerHewett Thrilled to announce that our recent work MobileEgo Anywhere, has been accepted in ICRA 2026 workshop (Beyond Tele-op)!!
Democratising large-scale data collection for Physical AI
Read More in🧵below.
x.com/fpv_labs/statu…
FPV Labs@fpv_labs
🚨 New Paper (ICRA Workshop, 2026) MobileEgo Anywhere: Open infrastructure for long-horizon egocentric data on commodity hardware. Existing robotics egocentric datasets are limited by short episode durations and rely on gated hardware. We release a framework designed to facilitate the collection of robust, hour-plus egocentric trajectories on a commodity iPhone.
English
Today, RADAR announced a $170 million Series B, bringing our valuation to more than $1 billion.
We believe Physical AI can transform the 80% of global commerce that still happens in stores.
Retailers lose an estimated $1 trillion each year because their stores lack real-time visibility into what they have and where it is. RADAR is helping close that gap with 99% item-level inventory accuracy in real time, already deployed in more than 1,400 stores with leading retailers including American Eagle Outfitters and Gap Inc. brands such as Old Navy.
We are just getting started.
A big thank you to our investors, including @nimble_partners, @gideonstrategic, @AlignVentures, @sound_ventures_, @ycombinator, our customers, including @AEO and @Gap Inc., @OldNavy and the entire RADAR team for helping bring us to this moment.
English
Thrilled to announce that our recent work MobileEgo Anywhere, has been accepted in ICRA 2026 workshop (Beyond Tele-op)!!
Amazing work by the entire team
Read More in🧵below.
FPV Labs@fpv_labs
🚨 New Paper (ICRA Workshop, 2026) MobileEgo Anywhere: Open infrastructure for long-horizon egocentric data on commodity hardware. Existing robotics egocentric datasets are limited by short episode durations and rely on gated hardware. We release a framework designed to facilitate the collection of robust, hour-plus egocentric trajectories on a commodity iPhone.
English
Introducing Project Stera by FPV Labs, an open data infra for embodied AI research.
Project Stera includes Stera-10M, with 10M+ frames of long-horizon data with persistent state tracking, and an open-source pipeline that converts raw data into training-ready formats.
English
ANNOUNCEMENT! 🚨
We have open-sourced Project Stera.
Democratising training ready egocentric data from collection to large scale processing.
---
putting out research paper, app, data, sdk, platform- everything to use out there.
FPV Labs@fpv_labs
Introducing Project Stera by FPV Labs, an open data infra for embodied AI research. Project Stera includes Stera-10M, with 10M+ frames of long-horizon data with persistent state tracking, and an open-source pipeline that converts raw data into training-ready formats.
English
Shubhanshu Khatana retweetledi
Human to robot transfer involves 2 problems that are tightly coupled - capturing how humans interact with the physical world and translating that knowledge into actions a robot can execute. Both depend on one thing - knowing precisely where the camera was in 3D space at every frame.
English
introducing metronis lab's first product.
aegis: autoresearch for evals and rl envs with ai native memory.
> beats SOTA on toolathon (for tooling) and legalbench (for vertical benchmark)
> vertical agnostic
> agents handle their own memory
> hermes-like rl-envs spin ups for your context
aegis is the closed-loop improvement engine for shipped AI agents.
book an intro call through the links on the website, or dm me.
check out metronis : www[dot]metronis[dot]space
English