Shubham Sharma

Go read our work on Hybrid Offline-and-online Reinforcement Learning at #NeurIPS2022 🤗🤗

We are publishing our second deep dive today as a follow-up post on SLAM and VIO in egocentric tracking. We go deep into the sensor tradeoffs b/w global shutter and rolling shutter and their implications on SLAM / VIO - specifically how the way the camera reads each frame can introduce significant tracking errors before our SLAM pipeline even starts processing. We break down why global shutter is the obvious fix but the wrong default, the physics of why rolling shutter dominates every consumer device, and where the fundamental limits lie.

After 8 months of building in stealth and testing our infrastructure on 10000+ hours of real-world data and hundreds of unique environments, we're bringing @fpv_labs into the open today. FPV Labs started with the following bet - if human data proves to be the underlying factor that determines scaling laws in general-purpose robotics, it will trigger the largest economic transformation in human history, and the underlying infrastructure that captures that data will determine how fast we get there. We will achieve this by building the full-stack infrastructure for capturing, processing, transferring, and evaluating human experience into spatial, temporal, and semantic knowledge for machines. Despite all the research novelty behind ChatGPT, its success can be attributed to one foundational fact - the scaling law of transformers. We believe the same dynamics have made their way into robotics. Recent studies showed task completion rates jumping from 30% to 70% when human demonstration data scaled from 1,000 to 20,000 hours, a log-linear trend that mirrors exactly what we saw in language and vision. Seeing these emergent signs of scaling law curves in robotics, we believe we are entering the era of general-purpose robotics policies, which makes the next few years the most exciting time in the history of this field. But the library of physical interactions required to train general-purpose robot policies does not exist yet. Over the last 8 months, we've seen dozens of companies emerge in this space. We were really happy to see new companies pushing this space forward, but we also saw the same pattern repeat: every egocentric data company was making some tradeoffs between quality, scale, and diversity. We have built FPV labs on the core principle that high-quality data is orders of magnitude more valuable than sheer volume. Case in point, self-driving cars collect thousands of hours of data per day, but only a small fraction of that data is actually useful for training better models. Several studies, like RT-2, have shown that as little as 1% of data improves as much as 25% on task success. The quality and diversity of data matter a lot more than scale, so there is clearly a power law curve in the downstream impact of data. We've spent months obsessing over data quality by building our stack, discarding it, rebuilding it, and iterating until we found a formula that doesn't compromise downstream quality at scale. We believe the downstream impact here is far more profound than most people realize. Workers globally are paid around $60 trillion per year in aggregate, and a lion's share of that compensation goes to physical labor - tasks that require navigating real spaces, manipulating real objects, and negotiating the infinite variability of the physical world. Human-to-robot transfer will be one of the most important infrastructures that will shape our society in the near future, and if it works, the economic impact will dwarf every technology transition that came before it in an exponential manner and lead to the creation of goods and services we can’t imagine today. Our mission is to lay the groundwork for us to transition into this future - the future of abundance. We are deeply grateful to our earliest believers, @paraschopra and @lossfunk, who played a critical role in shaping our thinking.

My mam finally coming out of stealth mode 🚀🚀

@1littlecoder Applied!! Ps: please open up your DMs so that i can beg you to let me in 🥰🙏

Dear Builders of Bengaluru, I'm hosting a workshop on AI media generation powered by @fal this saturday! It's a free event, but invite-only! Register @ Link 👇🏽 Thanks to the venue partner @c_engines

Poured everything into making this one. ❤️ Behrupiya is out now. Some games of pretend, never end. Here's Episode 1:

@amey_sharma Crazy work!!!

@NayrhitB @SusquehannaVC @LightspeedIndia @BCapitalGroup @beenextVC @sparrowcapvc @2point2club Deck Thanks for sharing!

The exact pitch deck that helped us raise a $9M Seed Round copy whatever you want VCs that invested: → @SusquehannaVC (led) → @LightspeedIndia → @BCapitalGroup → Seaborne Capital → @beenextVC → @sparrowcapvc → @2point2club joined. fundraising is hard enough without guessing what investors want to see. so - I'm making our deck public. if you're raising right now, take it and make it yours. Reply 'deck' + follow (so I can DM it over)

Memory Is All You Need. Traditional RAG relies on stateless, one-shot retrieval leading to temporal drift, outdated facts, and 40-80% failure rates in multi-agent coordination. True agentic systems demand stateful, persistent long-term memory: delta updates, episodic/semantic consolidation, and RL-optimized writes. Deep dive into why retrieval falls short and memory-augmented architectures win: x.com/aifunctor/stat… Building stateful agents @aifunctor → aifunctor.com Sneak-peaks from the article 👇 #AI #LLM #Memory #AgenticAI #LongTermMemory

Craziest party in Bangalore 🥸🤙🏼

The @NotionHQ Series A pitch was the most compelling story I have heard in my time in venture. This post gives a glimpse into why.

@anindyadeeps It’s in harlur 😭

@HappyyPablo damnn where is this?

Got a new drape for our hackerhouse Vibemaxing 🥰🥰

@tofufomax Currently FAFO in a few ideas storytelling and games. Figured i should keep posting here before launching lol

@HappyyPablo oh nice, what are you working on, shubham ?

@RitvikRastogi19 Congrats man! @sachdh 🚀🚀🚀

My first research paper!! and it got accepted at NeurIPS!! Still hard to believe. Cheers to the sleepless nights, the imposter syndrome, and the moments I almost gave up. To all the “it’s so over” and “we are so back.” Fuck it, we ball.

@jsuarez I’ve used "device=" on every tensor since I started. purely because of aesthetics lol

The second one is way faster. Apparently torch will move gradients back to CPU or something. Thanks, great way to spend an hour of my Saturday night.

Facebook groups are not ready for my game with @NanoBanana

Why does this look like something made with grok imagine ??

@levelheaded_94 super interesting experiment!!

For over a century, we believed evolution was just natural selection and mutation. We were wrong. The third, hidden pillar of evolution is cooperation. We tested whether SOTA language models possess this trait or if they default to greed. Introducing Commons Keeper v1.0 🧵

🧑‍🍳 🔥

@und3rdg here's their recent live action trailer: youtube.com/watch?v=8i2njo…

After the recent RMG ban, i hope more capital goes into making world class games from India. The new gen gamers have taste and want to experience more original stories so it's a great time to back Indian IPs for a global audience Recently chatted with @und3rdg where i learned about Mukti, their narrative game set in an Indian museum. Exactly the kind of Indian IP I want the world to experience 🌸