Yitao Liu

OSWorld 2.0 is finally out! CUA on long-horizon tasks still has a long way to go. Those tasks are pretty impressive, and the website is insightful. Please check them out! It will definitely accelerate CUA development, and I think we will see Jarvis on your computer soon!

Fine-grained instruction following is a key missing piece for building more capable generalist VLA policies. Glad to be involved in this project. Thanks @Erics_Tong and @cai_xu77719 for leading this excellent work!

Happy to share this journey with excellent guys in Qwen team. We have explored a lot about simulation data scaling for VLA pre-training and post-training. This model also shows great generalization ability. Still working on simulation evaluation and data scaling. More on the way!

Zero teleoperation. Zero real-world data. ➔ Autonomous humanoid loco-manipulation in reality. Introducing VIRAL: Visual Sim-to-Real at Scale. We achieved 54 autonomous cycles (walk, stand, place, pick, turn) using a simple recipe: 1. RL 2. Simulation 3. GPUs Website: viral-humanoid.github.io Arxiv: arxiv.org/abs/2511.15200 Deep dive with me: 🧵

📣Introducing VideoAgentTrek: a human-free, web-scale pipeline that turns screen-recorded tutorials into training data for computer-use agents, powered by specially trained VLMs. 🔗 [Website] videoagenttrek.github.io 📄 [Paper] arxiv.org/abs/2510.19488

in case u don't know, i set up a small team for robotics and embodied ai inside qwen. multimodal foundation models are now being transformed to foundation agents that can leverage tools and memory to perform long-horizon reasoning thanks to reinforcement learning. they should definitely step from virtual world to physical world!

🚀 We're thrilled to unveil Qwen3-VL — the most powerful vision-language model in the Qwen series yet! 🔥 The flagship model Qwen3-VL-235B-A22B is now open-sourced and available in both Instruct and Thinking versions: ✅ Instruct outperforms Gemini 2.5 Pro on key vision benchmarks ✅ Thinking achieves state-of-the-art (SOTA) performance on multimodal reasoning tasks ✨ Key breakthroughs: 🖥️ Visual Agent: Operates GUIs on PC/phone — understands buttons, calls tools, and completes real-world tasks (SOTA on OS World) 💻 Visual Coding: Transforms screenshots into code (HTML/CSS/JS, Draw.io) — true "what you see is what you get" development 📚 256K+ context (scalable to 1M) → supports 2-hour videos and multi-page long PDFs 🌍 32-language OCR with enhanced robustness for blurry, tilted, or rare characters 📐 Advanced spatial reasoning: 2D → relative coordinates, 3D grounding, occlusion handling, and perspective understanding 🧠 Thinking Mode: Leading performance in STEM/Math — enables deep causal reasoning 🔤 Text capabilities rival top-tier LLMs — a solid language foundation powering its multimodal excellence From "seeing" to "understanding", from "recognizing" to "reasoning & acting" Qwen Chat: chat.qwen.ai/?models=qwen3-… API: #5540e6e52e1xx" target="_blank" rel="nofollow noopener">alibabacloud.com/help/en/model-… Blog：qwen.ai/blog?id=99f033… ModelScope: modelscope.cn/collections/Qw… HuggingFace: huggingface.co/collections/Qw…

Scalable data is always the key to generalization. Simulation is absolutely much more scalable than real world. So what's next😎

@xhluca hardly to imagine this when using it in my last project last year lol

Despite having 873 stars on GitHub, BrowserGym's core library is installed 2610335 times a month on PyPI. The impact of the library on web agent research is severely underestimated & underappreciated.

🤖 If you want to evaluate your CUA on a real and challenging benchmark, definitely check out the updated OSWorld-Verified which provides more convincing results and easier evaluation. Also, we can get more valuable insights from the evaluation results.

✨(3/n) cont'd 2. Prompting is cheap but also has its upper bound, e.g. we found that although experiences provide some insights, the model sticks to its own thoughts sometimes. How to turn such NL experiences into parameter change efficiently could be an important direction.

✨(2/n) key takeaways from this work: 1.Learning from past experiences is obviously important for self-improvement and efficient domain adaptation for agents. We show that prompting-based experience distillation and utilization is efficient for web agents (15%△tokens→50% △SR)

I am excited to share my work about self-improving language agents through experience distillation in ACL poster session at Hall 4X Board 332! Thanks my brilliant collaborators: @ShunyuYao12 @ChengleiSi @karthik_r_n Paper here: arxiv.org/abs/2506.06698

>>> Qwen3-Coder is here! ✅ We’re releasing Qwen3-Coder-480B-A35B-Instruct, our most powerful open agentic code model to date. This 480B-parameter Mixture-of-Experts model (35B active) natively supports 256K context and scales to 1M context with extrapolation. It achieves top-tier performance across multiple agentic coding benchmarks among open models, including SWE-bench-Verified!!! 🚀 Alongside the model, we're also open-sourcing a command-line tool for agentic coding: Qwen Code. Forked from Gemini Code, it includes custom prompts and function call protocols to fully unlock Qwen3-Coder’s capabilities. Qwen3-Coder works seamlessly with the community’s best developer tools. As a foundation model, we hope it can be used anywhere across the digital world — Agentic Coding in the World! 💬 Chat: chat.qwen.ai 📚 Blog: qwenlm.github.io/blog/qwen3-cod… 🤗 Model: hf.co/Qwen/Qwen3-Cod… 🤖 Qwen Code: github.com/QwenLM/qwen-co…

🎮 Computer Use Agent Arena is LIVE! 🚀 🔥 Easiest way to test computer-use agents in the wild without any setup 🌟 Compare top VLMs: OpenAI Operator, Claude 3.7, Gemini 2.5 Pro, Qwen 2.5 vl and more 🕹️ Test agents on 100+ real apps & webs with one-click config 🔒 Safe & free access on cloud-hosted machines Page: arena.xlang.ai Leaderboard (tentative): arena.xlang.ai/leaderboard Blog: arena.xlang.ai/blog/computer-… Data & Code (coming soon): github.com/xlang-ai/compu… ⭐️Why Computer Agent Arena? 1️⃣Beyond Static Benchmarks: We use computers to perform enormous tasks and workflows every day, and AI agents have the potential to automate these tasks. However, existing benchmarks are very limited (e.g., only 369 tasks in OSWorld and 812 tasks in WebArena). To better measure their capabilities, we introduce Computer Agent Arena for users to easily compare & test AI agents on all kinds of crowdsourced real-world computer use tasks. 2️⃣Cloud Testing, Simplified: As agents like OpenAI’s Operator and Claude 3.7 sonnet release, users face configuration challenges and privacy hurdles to deploy on their own computers. Our platform integrates these agents with cloud-hosted machines, providing users with quick and secure access. 3️⃣Unified Embodied Digital Environment: Unlike Chatbot Arena, we provide users with a real embodied environment—computers—where all agents are grounded in real computer tasks and environments. Led by @XLANG_Lab [1/🧵]

What happens when humans can’t reliably supervise LLMs during RLHF? In a new paper, we find that unreliable supervision can cause DPO to fail completely. Instead of DPO/RLHF, we propose using human feedback to update the *SFT dataset* and show this works much better! 🧵

Google presents: Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments

NeurIPS pile 🗂️ +1. See y'all in december and look forward to having more models challenged and confronted (lol) by CharXiv 😛

OSWorld has been accepted by NeurIPS 2024 D&B track! 🎺✌️ Again, graceful thanks to all of our collaborators for their invaluable contributions to the project: @_zdy023, @chenjx210734, @xiaochuanlee, @SihengZhao, @RuishengC49326, @nikushii_, @ChengZhoujun, @dongchan, @fangyu_lei, @taoooo917, @yihengxu_, @shuyanzhxyc, @silviocinguetta, @CaimingXiong, @hllo_wrld, @taoyds; and @sidawxyz, @ptshaw2,@ChenHenryWu,@pengchengyin,@ShunyuYao12,@xhluca,@sivareddyg,@ruoxi_cc ,@LukeZettlemoyer, @ZhiyuanZeng_, @_TobiasLee, @zywu_hku, Chengyou Jia for their helpful feedback on this work!! Also thanks to contributors who help with improving this ecosystem and trust it, good and still behind~ Let's go over this video again!!!