Jenny Pan

15 posts

Jenny Pan

Jenny Pan

@jenpan_

visiting researcher @Stanford

Katılım Haziran 2021
42 Takip Edilen314 Takipçiler
Sabitlenmiş Tweet
Jenny Pan
Jenny Pan@jenpan_·
Robots need memory to handle complex, multi-step tasks. Can we design an effective method for this? We propose MemER, a hierarchical VLA policy that learns what visual frames to remember across multiple long-horizon tasks, enabling memory-aware manipulation. (1/5)
English
3
32
174
70.6K
Jenny Pan retweetledi
Charles Xu
Charles Xu@CharlesXu0124·
Pushing online RL to the next level -- exposing RL Token from the π-0.6 model for online RL achieves superhuman performance with as little as 15 minutes of data.
Physical Intelligence@physical_int

We developed an RL method for fine-tuning our models for precise tasks in just a few hours or even minutes. Instead of training the whole model, we add an “RL token” output to π-0.6, our latest model, which is used by a tiny actor and critic to learn quickly with RL.

English
7
17
99
14.1K
Jenny Pan retweetledi
Marcel Torné
Marcel Torné@marceltornev·
We equipped PI policies with memory! And taught our robots to do long-horizon real world tasks such as preparing the items for a recipe, cooking a grilled cheese and cleaning the kitchen!
Physical Intelligence@physical_int

We’ve developed a memory system for our models that provides both short-term visual memory and long-term semantic memory. Our approach allows us to train robots to perform long and complex tasks, like cleaning up a kitchen or preparing a grilled cheese sandwich from scratch 👇

English
7
15
84
8.7K
Jenny Pan retweetledi
RoboPapers
RoboPapers@RoboPapers·
Most robot policies today still largely lack memory: they make all their decisions based on what they can see right now. MemER aims to change that by learning which frames are important; this lets it deal with tasks like object search. @ajaysridhar0, @jenpan_, and @satviks107Sharma tell us about how to achieve this fundamental capability for long-horizon task execution. Watch Episode #54 of RoboPapers with @micoolcho and @chris_j_paxton to learn more!
English
7
14
86
17.8K
Jenny Pan retweetledi
Yanjiang Guo
Yanjiang Guo@GYanjiang·
Rollouts in the real world are slow and expensive. What if we could rollout trajectories entirely inside a world model (WM)? Introducing 🚀Ctrl-World🚀, a generative manipulation WM that can interact with advanced VLA policy in imagination. 🧵1/6
English
5
40
211
95K
Jenny Pan retweetledi
Ajay Sridhar
Ajay Sridhar@ajaysridhar0·
VLAs are great, but most lack long-term memory humans use for everyday tasks. This is a critical gap for solving complex, long-horizon problems. Introducing MemER: Scaling Up Memory for Robot Control via Experience Retrieval. A thread 🧵 (1/8)
English
6
43
318
42.8K
Jenny Pan
Jenny Pan@jenpan_·
We design 3 tasks that entail using memory in distinct ways, including recalling object locations, keeping track of completed actions, and counting repeated steps. MemER significantly outperforms baselines that naively scale history, while retaining low-latency inference. (4/5)
Jenny Pan tweet media
English
1
0
8
1.1K
Jenny Pan
Jenny Pan@jenpan_·
Robots need memory to handle complex, multi-step tasks. Can we design an effective method for this? We propose MemER, a hierarchical VLA policy that learns what visual frames to remember across multiple long-horizon tasks, enabling memory-aware manipulation. (1/5)
English
3
32
174
70.6K
Jenny Pan
Jenny Pan@jenpan_·
Long histories are computationally expensive, while naive subsampling misses crucial context. We train a high-level policy to select & track relevant past keyframes from its experience, and condition on this memory to generate a subtask for a low-level policy to execute. (2/5)
Jenny Pan tweet media
English
0
0
0
176

Keşfet

@ajaysridhar0 @micoolcho @chris_j_paxton @satviks107 @chelseabfinn @jessicac_09 @elonmusk @BarackObama