Forrest Huang

@richard_qiuyi_z Any activity with Richard is a good time. If you're in Rio this week, swing by and learn about Elorian's new patio fountain luma.com/dwdpic19

Super excited for the first Elorian cohosted dinner at ICLR! Hope to see you there :)

Excited to be part of this journey, looking forward to building the next generation of models that can truly see, listen, and think.

🤔 Small GUI agents can be important for on-device deployment, but how far can such small agents go? 💡 Instead of scaling up, in Ferret-UI Lite, we focused on scaling down, and present our lessons from building small on-device GUI agents. 🔗 arXiv: arxiv.org/abs/2509.26539 📔 Ferret-UI Lite is 3B in size. We conduct both SFT and RL, for both grounding and navigation, and across mobile, web and desktop platforms. 1️⃣ Ferret-UI Lite achieves strong results on GUI grounding (e.g., 53.3% on ScreenSpot-Pro and 61.2% on OSWorld-G), surpassing many larger models. We found that RL with zoom-in is effective to improve performance. 2️⃣ For navigation, Ferret-UI Lite achieves success rates of 28.0% on AndroidWorld and 19.8% on OSWorld, outperforming other 3B agents. But its performance can still be limited. We found carefully designed RLVR reward is important for training such small agents. We hope our lessons will inform future research on building efficient, capable, and practical on-device AI agents. Led by Zhen Yang, @ZiYiDou, Di Feng, Forrest Huang, Anh Nguyen @alexttoshev @yinfeiy etc.

youtu.be/BOIfrxNzsQE Announcing the @ACMUIST Student Innovation Contest CFP! Thanks to @AnthropicAI's sponsorship, selected teams will receive free access to the Claude LLM APIs for development. Looking forward to wild submissions with @jasonwuishere! #student-innovation-contest" target="_blank" rel="nofollow noopener">uist.acm.org/2023/cfp/#stud…

AI&HCI Workshop (sites.google.com/view/aihci/home) at #ICML2023 had a great turnout, with 5 in-person and 1 recorded keynotes, 60 papers (from 207 authors and 63 institutions), 2/3 of participants from AI/ML and 1/3 from HCI.

Check out our upcoming ICML paper on layout generation using latent diffusion: openreview.net/forum?id=2jvwy…. Our work generates UI or document layout from guidelines in vector space, which is editable through direct manipulation and brings an interactive experience to layout design.

Excited to share our latest work on Spotlight, a foundational model for UI understanding ai.googleblog.com/2023/02/a-visi…

PLay: Parametrically Conditioned Layout Generation using Latent Diffusion abs: arxiv.org/abs/2301.11529

Check out our most recent paper out now in AIEDAM! Enabling multi-modal search for inspirational design stimuli using deep learning bit.ly/3zF725v

PSA: The world’s smartest artificial intelligence (aka @OpenAI) made this magazine cover (yes, really!)—its first EVER.

@Eurostar Any ideas will train 9063 from Paris to London be delayed tonight?