Le Chen

Introducing LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Model 📖 Paper: arxiv.org/abs/2603.27449 🌐 Project: zerg-overmind.github.io/LOME.github.io/

🚀 Meet OFTv2 — Orthogonal Finetuning made scalable, finally. ⚡️ 10× faster 💾 3× less GPU memory 🤖 Quantized OFT: plug-and-play on quantized LLMs, better than QLoRA Try it now on Hugging face PEFT: tinyurl.com/ycxswfe7 Website: spherelab.ai/oftv2/ #AI #LLM 🧵1/6

@lgraesser3 Thanks a lot!

@clthegoat Congratulations Li! This is great work and I really enjoyed reading your paper.

Learning to play the piano with two robot hands is super challenging, even in simulation! It requires coping with bimanual coordination at high speed to achieve human-level dexterity. We introduce RP1M, a large-scale robot piano-playing motion dataset, featuring ~1M trajectories over 2k music pieces. Website: rp1m.github.io Paper: arxiv.org/abs/2408.11048

@kevin_zakka Thank you so much, Kevin!

@clthegoat Amazing work @clthegoat! Super happy to see you got rid of the need for fingering labels. Unlocking basically any MIDI file makes RoboPianist even more useful to the community :)

@BlockyBard 🔥

@YagaoDirac For each RL agent, we use DroQ to train an MLP.

@clthegoat What layer or model is used for this? 用的什么层？全连接还是tfm还是什么别的？

@AuroraDevEco Thanks a lot!

@yaru_niu yes finally🤣

@clthegoat Glad to see you here🤩

RP1M contains various piano-playing motions, including highly dynamic motions and long-distance movements. Here is an example of the song Flight of the Bumblebee:

🌟Our OT-based method: 1️⃣ Low-cost: No need for human demonstrations or annotations. 2️⃣ Cross embodiments: Supports diverse hand morphologies and robot platforms. 3️⃣ Superior performance: Allows robots to discover optimal fingering aligned with their unique morphology.

Gradient subspace optimization unlocked for RL 🔒➡️🔓 Used only for supervised learning so far, our #ICLR2024 paper illustrates that policy gradients evolve in a small, slowly-changing subspace, opening up many opportunities for more efficient RL. arxiv.org/abs/2401.06604