Andy Tang

How can robot policies be trained to best leverage VLMs' CoT reasoning and in-context learning for generalization? The key is Steerable Policies: vision-language-action models that can be flexibly controlled in many ways! steerable-policies.github.io 1/9

Exploration is fundamental to RL. Yet policy gradient methods often collapse: during training they fail to explore broadly, and converge into narrow, easily exploitable behaviors. The result is poor generalization, limited gains from test-time scaling, and brittleness on tasks where strategic exploration is necessary. We introduce a framework for training a policy over sets of generations and use it to induce exploration. Work with @ifdita_hasan (co-lead), @ellenjxu_ , @chelseabfinn and @DorsaSadigh at Stanford 🧵

Very happy to share that our work on learning long-history policies received the Best Paper Award from the Workshop on Learned Robot Representations @RoboticsSciSys ! 🤖🥳 Check out our paper if you haven't already! long-context-dp.github.io Thank you to all the organizers and the amazing collaborators @tangerinecoder, @liu_yuejiang and @chelseabfinn!

Even the smartest LLMs can fail at basic multiturn communication Ask for grocery help → without asking where you live 🤦‍♀️ Ask to write articles → assumes your preferences 🤷🏻‍♀️ ⭐️CollabLLM (top 1%; oral @icmlconf) transforms LLMs from passive responders into active collaborators. Website: aka.ms/CollabLLM Github: github.com/Wuyxin/collabl… Blog: #blog" target="_blank" rel="nofollow noopener">wuyxin.github.io/collabllm/#blog Paper: arxiv.org/pdf/2502.00640 🎯 Key insight: Rewards responses not by immediate helpfulness, but by their long-term impact on the conversation trajectory. @MSFTResearch @StanfordAILab @stanfordnlp

How can robots autonomously handle ambiguous situations that require commonsense reasoning? *VLM-PC* provides adaptive high-level planning, so robots can get unstuck by exploring multiple strategies. Paper: anniesch.github.io/vlm-pc/

How do we make a scalable RL recipe for robots? We study batch online RL w/ demos. Key findings: - iterative filtered imitation is insufficient - need diverse policy data, eg using diffusion policy - policy extraction can hinder data diversity Paper: pd-perry.github.io/batch-online-r…

🧠Memory is crucial for robots — to handle occlusions, track progress, stay coherent, etc. Yet, most VLA truncate context. 🤔Why is long-context hard for robot policies? And how can we fix it? 📄Our new paper: Learning Long-Context Diffusion Policies via Past-Token Prediction

Was super fun exploring this! Most modern policies don't use history -- Diffusion Policy in particular gets a lot worse. We identify a simple ingredient for history improvement, and use it to improve efficiency and performance of long-context policies.

@sanjehorah i have an ad-hoc version in LaTeX annotated with date added/finished, stage (to read, annotate, done), priority, and stream (from Twitter, robotics, neuro, etc.) -- lmk if ppl get together to build as i have opinions

i am BEGGING

I’m excited to share a project I’ve been working on for over a year, which I believe will fundamentally change our approach to language models. We’ve designed a new architecture, which replaces the hidden state of an RNN with a machine learning model. This model compresses context through actual gradient descent on input tokens. We call our method “Test-Time-Training layers.” TTT layers directly replace attention, and unlock linear complexity architectures with expressive memory, allowing us to train LLMs with millions (someday billions) of tokens in context. Our instantiations, TTT-Linear and TTT-MLP, both match or beat the strongest Transformers and Mamba. Arxiv: arxiv.org/abs/2407.04620

@adriana0nline CONGRATSSS!!!

Graduated btw :-)

@TheBookie0 seems like they might be in need of a designer ;)

for some reason my teacher didn't stick to the default templates and added these random background colors to the slides?!

@giansegato do this as well! don't have a voice recorder but turn off wifi/cellular, open voice memos, go pace around a basement or run

a couple of weeks ago i bought an old-school portable voice recorder, like the ones we used to use to record classes before the iphone been using it while going on smartphones-less walks. i talk to it with no structure, just following my train of thoughts as they come, and when i come back home i transcribe my rambles with whisper, summarize/synthesize with an llm, and work on the very few good ideas in a more focused mental state it's well understood that movement improves thinking, and that the unfocused, bored state of mind can foster high quality ideas. the issue is that every time i go on a walk, my smartphone puts me in a focused, cognitively engaged state. no matter what i try. i'm addicted to mental stimulation, and constantly having the internet in my pocket simply hurts my ability to think crisply and *widely* i don't think it's a coincidence that @paulg always preaches the benefits of walks, but doesn't bring the internet with him. anecdotal, but telling on the other hand though, issue with smartphone-less walks is that i just forget stuff. i feel the constant need to take notes. probably a byproduct of my internet addiction. and so back to square 1 until today there was not really a good solution to my problem. luckily, not anymore whisper + any decently sized llm is already good enough to bring order to my incoherent rambling, making the transition from unfocused to focused thinking *seamless*. can't overstate how huge this is for me this *feels* artificial intelligence. it's an intelligent tool, making sense of my voice memos. it's kinda wild there may be a hardware side project somewhere in there (could be fun to put together with a raspberry), but with a modest $30 investment and open source tools i can run locally i already solved an issue that has been bothering me for years i think we just began to explore the impact of deep learning on tools for thought

@TheBookie0 lfggggggggggggg

i'm going to Cornell!!!

We've had over a thousand new engineers try Quilter in the last few weeks submitting some really interesting designs. We really want to see some of these come to life, so we're subsidizing board builds! If you want to build a Quilter design in real life, we'll cover the cost of the PCB! More about this in the link. Open hardware community: this one is especially for you ;)

Here is Playground v2.5, our latest model.

Very excited to introduce ROAM, our new work that allows a robot to *adapt on-the-go* as it faces OOD situations during deployment, drawing on pre-trained behaviors. See as ROAM enables our Go1 to roller skate zero-shot 🤖🐕🛼 (without any lessons!) 🧵(1/9)

We’ve had a flurry of product launches over the past week. Unless you’ve been on X every day, you likely missed a couple. Here’s a recap of every launch so you can get up to speed👇

@itsarnavb @0xDak my fav is the triple back tap on iPhone, I feel like no one knows about it and it's so nice to switch my phone to grayscale and back

@0xDak there’s actually a bunch of these available buried somewhere under accessibility! i guess “double pinch” was the most reliable one so they shipped that

@JimmyAustin you dropped this👑