Howard Zhou

True multimodal AI needs to understand the world spatially 🎯 🚀 Excited to release #CVPR2026 TIPSv2 from @GoogleDeepMind, a foundational image-text encoder with spatial awareness, leading to strong overall results and massive gains on patch-text alignment. 🔥 1/N

Text, image, video, audio, all embedded into one unified space. Congratulations to our team for this amazing accomplishment!

Gemini 3 Pro is the frontier of multimodal AI, delivering SOTA performance across document, screen, spatial, and video understanding. Read our deep dive on how we’ve pushed our core capabilities to power hero use cases across: + Docs: "derender" complex docs into structured code (HTML/LaTeX) + Screen: build robust computer agents that automate complex tasks + Spatial: generate collision-free trajectories for robotics & XR + Video: analyze sports footage using high-FPS processing with "thinking" mode See how these capabilities are transforming workflows in education, biomedical, and law/finance → goo.gle/3Mt3UlT

Please check out this cool work from @IanHuang3D and our team within @GoogleDeepMind Website: fireplace3d.github.io Paper: arxiv.org/abs/2503.04919

Last Call: Learn GenAI and help us break the GUINNESS WORLD RECORDS™ for Largest Virtual AI Conference! Join Google & Kaggle's GenAI Intensive: No cost, live sessions, hands-on labs. Registration closes this Friday! #GenAI #GuinnessWorldRecords #Kaggle #GoogleAI

BREAKING: Gemini 2.5 Pro is now #1 on the Arena leaderboard - the largest score jump ever (+40 pts vs Grok-3/GPT-4.5)! 🏆 Tested under codename "nebula"🌌, Gemini 2.5 Pro ranked #1🥇 across ALL categories and UNIQUELY #1 in Math, Creative Writing, Instruction Following, Longer Query, and Multi-Turn! Massive congrats to @GoogleDeepMind for this incredible Arena milestone! 🙌 More highlights in thread👇

Multimodal AI encoders often lack spatial understanding… but not anymore! Our #ICLR2025 TIPS model (Text-Image Pretraining with Spatial awareness) from @GoogleDeepMind can help 💡🚀 Check out our strong & versatile image-text encoder 💪 Paper & code: arxiv.org/abs/2410.16512

@nagaviperszy 你是准备把熊吓跑吗？

太太说，这样更能加强安全意识。

Want some TIPS? Well, then check out “Text-Image Pretraining with Spatial awareness” :) TIPS is a general-purpose image-text encoder, for off-the-shelf dense and image-level prediction. Finally image-text pretraining with spatially-aware representations! arxiv.org/abs/2410.16512

Modeling Collaborator Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use From content moderation to wildlife conservation, the number of applications that require models to recognize nuanced or subjective visual concepts is growing.

We are excited to share our work on our Pathways Autoregressive Text-to-Image model, Parti! #Parti achieves high-fidelity photorealistic image generation and supports content-rich synthesis involving complex compositions and world knowledge. parti.research.google

Andrew Marmon and I rounded up all #CVPR2022 papers on NeRF/Neural Radiance Fields we could find in a new blog post here: dellaert.github.io/NeRF22/

New work from @GoogleResearch by @JHYUXM, @MrZiruiWang, @Spezzer, @LeggYeung, Mojtaba Seyedhosseini and Yonghui Wu: CoCa is a new way of combining image and text representations that achieves SOTA results on a large number of tasks of different kinds.

In anticipation of the Intl. Conf. on Computer Vision (#ICCV2021) this week, I rounded up all papers that use Neural Radiance Fields (NeRFs) represented in the main #ICCV2021 conference here (1/N): dellaert.github.io/NeRF21

@docmilanfar Hell yeah!

Whatever your allegiance it’s hard not to be happy for Messi 🇦🇷

Training NeRFs per-scene is so 2020. Inspired by image based rendering, IBRNet does amortized inference for view synthesis by learning how to look at input images at render time. 15% drop in error, 80% fewer FLOPs than NeRF. Great work @QianqianWang5! ibrnet.github.io

Work from our team, yeah!