xintong hu

Current robot policies overfit specific language templates, handling 'pick and place' but freezing on 'drag it to me ' or 'push it closer to me.' They also lack control over execution: which hand, what approach angle, where to grasp, which path to follow. 🤖 FineVLA make robots steerable : changing instruction alters execution; same task, different phrasing, distinct actions — all faithfully done. 🏠 Homepage: finevla.xlang.ai 📄 Paper: huggingface.co/papers/2605.27… 💻Codebase: github.com/xlang-ai/FineV… 🧵[1/6]

Two years ago, we built OSWorld 1.0 — the benchmark that became the standard for computer-use agents. Agents now score 83.5% on it. Problem solved? Not even close. 🚀Today we introduce OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks. What's new: 🎯 108 real-world workflows, each ~1.6 hours ⏱️ for a skilled human ⚙️ ~318 tool calls/task vs. ~30 in OSWorld 1.0 🌍 Grounded in authentic artifacts & stateful user profiles ⚡ Captures real phenomena: dynamic environments, streaming interaction, cross-source reasoning, implicit-state inference & more 📊 Best results: Claude Opus 4.8 reaches the highest accuracy at 20.6%, while GPT-5.5 is far more token-efficient but plateaus near 13%. No one is close to solving real computer use. 🏠 Homepage: osworld-v2.xlang.ai 📄 Paper: github.com/xlang-ai/OSWor… 💻 Code: github.com/xlang-ai/OSWor… 🤗 Dataset: huggingface.co/datasets/xlang… 🧵 [1/8]

@taoooo917 @cai_xu77719 thanks, Yitao!

Fine-grained instruction following is a key missing piece for building more capable generalist VLA policies. Glad to be involved in this project. Thanks @Erics_Tong and @cai_xu77719 for leading this excellent work!

Current robot policies overfit specific language templates, handling 'pick and place' but freezing on 'drag it to me ' or 'push it closer to me.' They also lack control over execution: which hand, what approach angle, where to grasp, which path to follow. 🤖 FineVLA make robots steerable : changing instruction alters execution; same task, different phrasing, distinct actions — all faithfully done. 🏠 Homepage: finevla.xlang.ai 📄 Paper: huggingface.co/papers/2605.27… 💻Codebase: github.com/xlang-ai/FineV… 🧵[1/6]

🧵[6/6] FineVLA makes robot policies steerable by specifying how to act, not just what to do. We open-source the full stack: ✅ Fine-grained data annotation pipeline ✅ RoboFine-VLM annotator ✅ RoboFine-Bench benchmark 🏠 Homepage: finevla.xlang.ai 📄 Paper: huggingface.co/papers/2605.27… 💻Codebase: github.com/xlang-ai/FineV… 🤗Annotator: huggingface.co/xlangai/RoboFi… 🤗Benchmark : huggingface.co/datasets/xlang…

🧵[5/6] Key findings: 🔬 i. No Sacrifice — Fine-grained data doesn't hurt goal-level success. FG-only consistently outperforms Raw-only by +1.4 to +8.1 pts. The OFT-vs-GR00T architecture gap shrinks from 6.4 to just 0.8 — showing strong cross-architecture generalization. 🔬 ii. Complementary — FG and raw instructions are complementary, not competing. Performance follows a clear inverted-U, peaking at FG:Raw = 1:1. Best mix: 86.8%/82.5% in simulation(+15/+11.1 over Raw-only ), 62.7 in real-world (+12.8 over Raw-only). 🔬 iii. Steerable — Fine-grained language gives robots true factor-level controllability. Same task, different instructions → different execution: - Object pose: 24 → 47 (+23) - Approach direction: 60 → 78 (+18) - Target color: 22 → 40 (+18) - Rotation: 76 → 86 (+10)

RLVR has become the recipe for agentic post-training. But for Computer-Use Agents, the bottleneck is not the algorithm, it is the data. 🐌 🚀 We introduce CUA-Gym: a scalable, lightweight synthesis engine that turns arbitrary task queries into verifiable RLVR data for computer-use agents. The largest open CUA RLVR dataset to date: 🎯 32,122 verifiable RLVR tasks with programmatic setup scripts + rewards 🌐 110 environments: 16 desktop apps + 94 synthesized mock web apps 🏆 Qwen3.5-based CUA models trained with GSPO reach 72.6% on OSWorld-Verified and 56.6% on WebArena 📄 Paper: huggingface.co/papers/2605.25… 🏠 Homepage: cua-gym.xlang.ai 🤗 Dataset: huggingface.co/datasets/xlang… 💻 Codebase: github.com/xlang-ai/CUA-G… 🧩 Environments: github.com/xlang-ai/CUA-G… 🧵[1/6]

@OWW Thank you for your support. Please feel free to communicate with us if you have any questions.👏

FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies Xintong Hu, Xuhong Huang, Jinyu Zhang, Yutong Yao, Yuchong Sun, Qiuyue Wang, Mingsheng Li, Sicheng Xie, Yitao Liu, Junhao Chen, Yixuan Chen, … arxiv.org/abs/2605.27284 [𝚌𝚜.𝚁𝙾 𝚌𝚜.𝙰𝙸]

Super stoked to be part of Qwen-vla! So glad to meet such team member who share the same interests.

Excited to share Qwen-VLA paper, our exploration of generalist Vision-Language-Action models. It extends Qwen’s multimodal backbone from visual understanding and reasoning to continuous action generation and trajectory prediction. Paper: arxiv.org/pdf/2605.30280