Zhengzhong Tu

1.3K posts

Zhengzhong Tu banner
Zhengzhong Tu

Zhengzhong Tu

@_vztu

Assistant Professor in AI@TAMU | AE @ IEEE-TIP/TMLR | AC @ CVPR/NeurIPS/ICLR | ex-@GoogleAI | PhD @UTAustin | BS @FudanUni | Intern @GoogleAI (x3)

Mountain View, CA Katılım Aralık 2021
2.6K Takip Edilen7.4K Takipçiler
Sabitlenmiş Tweet
Zhengzhong Tu
Zhengzhong Tu@_vztu·
🎉 [𝗡𝗲𝘂𝗿𝗜𝗣𝗦 𝟮𝟬𝟮𝟱] 𝗣𝗮𝗽𝗲𝗿 𝗔𝗰𝗰𝗲𝗽𝘁𝗲𝗱 𝗮𝗻𝗱 𝗖𝗼𝗱𝗲 𝗥𝗲𝗹𝗲𝗮𝘀𝗲! Not everyone is a photo pro — even with advanced AI tools. Restoring an image to professional grade often requires either 𝘥𝘦𝘦𝘱 𝘱𝘩𝘰𝘵𝘰𝘨𝘳𝘢𝘱𝘩𝘺 𝘦𝘹𝘱𝘦𝘳𝘵𝘪𝘴𝘦 or 𝘣𝘳𝘰𝘢𝘥 𝘬𝘯𝘰𝘸𝘭𝘦𝘥𝘨𝘦 of how to combine many different AI models. 🙋 𝗦𝗼 𝘄𝗲 𝗮𝘀𝗸𝗲𝗱 𝗼𝘂𝗿𝘀𝗲𝗹𝘃𝗲𝘀: Can we build a restoration agent that fully automates this process — no domain expertise required? To answer this, we @TAMU partnered with @Topaz and scholars from @Stanford, @Caltech, @UTAustin, @Snap, @ucmerced, @CUBoulder to develop the 𝘄𝗼𝗿𝗹𝗱’𝘀 𝗳𝗶𝗿𝘀𝘁 𝗮𝗴𝗲𝗻𝘁𝗶𝗰 𝗽𝗵𝗼𝘁𝗼 𝗿𝗲𝘀𝘁𝗼𝗿𝗮𝘁𝗶𝗼𝗻 𝘀𝘆𝘀𝘁𝗲𝗺. Our system brings together over 𝟱𝟬 𝘀𝗽𝗲𝗰𝗶𝗮𝗹𝗶𝘇𝗲𝗱 𝗔𝗜 𝗺𝗼𝗱𝗲𝗹𝘀 — for denoising, deblurring, upscaling, face recovery, detail enhancement, and more. 🔬 It diagnoses the input image. 🦾 It plans and executes an action graph (just like a human editor). ☑️ After each step, it evaluates the output and adapts the plan if needed. The results are phenomenal — as you’ll see in the video. We believe democratizing professional photography will empower anyone to create images suitable for professional use cases. And because we see this as a responsibility, we are 𝗳𝘂𝗹𝗹𝘆 𝗼𝗽𝗲𝗻-𝘀𝗼𝘂𝗿𝗰𝗶𝗻𝗴 𝘁𝗵𝗲 𝘀𝘆𝘀𝘁𝗲𝗺 — to accelerate progress in agentic photo restoration and support the broader community. 👉 𝗖𝗵𝗲𝗰𝗸 𝗼𝘂𝘁 𝘁𝗵𝗲 𝗰𝗼𝗱𝗲: github.com/taco-group/4KA… 📄 𝗣𝗿𝗼𝗷𝗲𝗰𝘁 𝘄𝗲𝗯𝗽𝗮𝗴𝗲: 4kagent.github.io 𝗜'𝗺 𝗹𝗼𝗼𝗸𝗶𝗻𝗴 𝗳𝗼𝗿𝘄𝗮𝗿𝗱 𝘁𝗼 𝘀𝗲𝗲𝗶𝗻𝗴 𝘄𝗵𝗮𝘁 𝘆𝗼𝘂 𝗰𝗮𝗻 𝗯𝘂𝗶𝗹𝗱 𝘂𝘀𝗶𝗻𝗴 𝗼𝘂𝗿 𝗼𝗽𝗲𝗻–𝘀𝗼𝘂𝗿𝗰𝗲 𝘀𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗮𝗻𝗱 𝘁𝗼 𝗿𝗲𝘃𝗶𝗲𝘄𝗶𝗻𝗴 𝘆𝗼𝘂𝗿 𝗱𝗲𝗺𝗼 𝗮𝗻𝗱 𝗳𝗲𝗲𝗱𝗯𝗮𝗰𝗸!
English
9
39
368
41.9K
Wenbo Guo
Wenbo Guo@WenboGuo4·
I will be on-leaving from UCSB and joining Meta MSL as an AI Research Scientist, as part of the Virtue AI team! I feel extremely lucky and deeply honored to work with our amazing Virtue AI team. I still remember the first time discussing with Bo and Dawn about building an end-to-end agent security platform. Back then, AI agents were still in their infancy—a very new idea. We shared the vision that agents are the future, given their ability to take real actions and complete real tasks rather than simply generating content. We believed that security and compliance would be an essential, must-have layer for agents to truly take off. So we started and built everything from the ground up: real-time guardrails for agent prompts and actions, static scanning of MCP source code and tool descriptions, shadow AI detection, and both static and dynamic observability of agent structures and trajectories. Beyond that, we designed multiple integration approaches, including gateway, hook/harness, and SDK. Earlier this year, we officially released AgentSuite, our unified agent security and compliance platform that integrates all of these capabilities. We then continued building Agent ForgingGround, a set of simulated environments for agent red-teaming, along with a suite of novel red-teaming algorithms. AgentSuite and Agent ForgingGround have been well received by the market, tested by many Fortune 500 enterprises, and are expected to bring us millions in ARR. Throughout this process, our team worked incredibly hard—designing and implementing the product, supporting customers, and responding to their requests with remarkably fast turnaround. As part of this effort, I again felt extremely lucky to contribute to the entire journey. Along the way, I learned so much: how to build a strong tech team, how to talk to customers, how to digest and distill their needs, how to design products, how to implement full-stack solutions, how to handle on-prem deployments, and more. These end-to-end experiences are something I would never have gained without Virtue. Most gratefully, our effort and product have been recognized by many great customers, and ultimately, by Meta MSL, one of the top frontier AI labs. Looking forward, I am super excited to continue with our Virtue team and many other great colleagues at Meta. We are deeply impressed by the vision that Alexandr Wang, Nat Friedman, and Prashant Ratanchandani share. I look forward to building the super security layer for superintelligent agents, with the potential to impact billions of users and society as a whole. A heartfelt thanks to our team, customers, partners, advisors, and investors for all the hard work and support along the way. Very exciting but challening jounry ahead! Way to go~
English
6
0
100
10.1K
Zhengzhong Tu
Zhengzhong Tu@_vztu·
@jxmnop seems to show that the recent hires from Arxiv are worth it
English
0
0
1
129
Jack Morris
Jack Morris@jxmnop·
BREAKING: ArXiv changes toolbar color from #AE2A24 to #1C1A17 RIP "ArXiv Crimson Red" (1991 – 2026) 🟥 🔴 ❤️
Jack Morris tweet mediaJack Morris tweet media
English
87
175
2K
257.7K
Zhengzhong Tu
Zhengzhong Tu@_vztu·
🔥 Thrilled to share that #SparkVSR was accepted to #ECCV2026! But before that, our GitHub repo already gathered more than 600+ stars - greatly appreciate the support from the community 👏 As a fully open-source project, we're glad to see that our model was directly supported in-one-click by AI workflow platforms like #RunningHubCNAPS.AI, and Runpod, enabling millions of AI developers and designers to customize their own AI pipelines. We're also super grateful for the media coverage by AIFilm.studio, Hackernews, #Tencent News, to name a few. We're continuing the R&D of this project and, in the future, will release much lighter versions to support consumer GPUs or mobile devices. Please stay tuned for our latest updates! ☕ Links 1. SparkVSR github: lnkd.in/gFgsFwJm 2. CNAPS AI: cnaps.ai 3. Runninghub: lnkd.in/gaQtAA2k 4. AI Film: lnkd.in/g_nMe3Q4 5. Tencent: lnkd.in/gEjZYRmP
English
2
6
39
3.8K
Zhengzhong Tu retweetledi
Wildminder
Wildminder@wildmindai·
some cool stuff from Google! Dropped an Interactive Video Super-Resolution - SparkVSR - CogVideoX1.5-5B; - propagates high-quality textures from edited keyframes across sequences; - tops DOVE and STAR, +24.6% on CLIP-IQA and +21.8% on DOVER; - zero-shot restoration, colorization, style transfer! - beats FlashVSR sparkvsr.github.io
English
3
33
233
19.9K
Higgsfield AI 🧩
Higgsfield AI 🧩@higgsfield·
Unlimited Seedance for 30 days, starting today. Higgsfield is the only place with real unlimited Seedance, powered by an official @BytePlusGlobal partnership. Purchase now and get unlimited Seedance for 30 days.
English
530
396
2.4K
1.5M
Zhengzhong Tu
Zhengzhong Tu@_vztu·
We often hear that "computer vision has been solved.” But is it really so? 🚀 Excited to share our new work: 𝗖𝗩-𝗔𝗿𝗲𝗻𝗮: 𝗔𝗻 𝗢𝗽𝗲𝗻 𝗕𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸 𝗳𝗼𝗿 𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻𝗮𝗹 𝗖𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝗩𝗶𝘀𝗶𝗼𝗻 𝗣𝗿𝗼𝗯𝗹𝗲𝗺 𝗦𝗼𝗹𝘃𝗶𝗻𝗴 𝘄𝗶𝘁𝗵 𝗛𝘂𝗺𝗮𝗻-𝗔𝗜 𝗖𝗼𝗹𝗹𝗮𝗯𝗼𝗿𝗮𝘁𝗶𝘃𝗲 𝗣𝗿𝗲𝗳𝗲𝗿𝗲𝗻𝗰𝗲𝘀. In this paper, we define 𝗶𝗻𝘀𝘁𝗿𝘂𝗰𝘁𝗶𝗼𝗻𝗮𝗹 𝗰𝗼𝗺𝗽𝘂𝘁𝗲𝗿 𝘃𝗶𝘀𝗶𝗼𝗻 𝗽𝗿𝗼𝗯𝗹𝗲𝗺 𝘀𝗼𝗹𝘃𝗶𝗻𝗴 𝗶𝗖𝗩𝗣𝗦 as a broader formulation of image editing: given a real input image and a natural-language instruction, a system must produce an edited output that realizes the requested transformation while satisfying explicit preservation, geometric, physical, and usability constraints. 🧩 To support this direction, we introduce 𝗖𝗩-𝗔𝗿𝗲𝗻𝗮, an open benchmark designed for professional-grade visual editing and problem solving. 𝗖𝗩-𝗔𝗿𝗲𝗻𝗮 contains: ✅ 12K high-resolution real-image instruction pairs ✅ 16 instruction-based visual task types ✅ Tasks spanning restoration, enhancement, computational photography, physically grounded object insertion, semantic manipulation, geometry-driven structural editing, and typography recovery ✅ Real-world images with native aspect ratios and high-resolution details 🔍 We also introduce 𝗖𝗼𝗴𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗲𝗿, a dual-track retrieval and curation pipeline that combines targeted web search, agentic query refinement, verification, and traceability to construct diverse and legally traceable benchmark data. ⚖️ For evaluation, we propose 𝗔𝗰𝘁𝗶𝘃𝗲 𝗘𝗹𝗼, a human-AI collaborative preference protocol. Instead of relying purely on automatic metrics or fully human annotation, Active Elo combines: 1. 𝗖𝗩-𝗝𝘂𝗱𝗴𝗲, a logic-gated, multi-dimensional VLM evaluator 2. selective routing of ambiguous high-quality comparisons to expert human raters 3. reliability-weighted Elo updates to aggregate mixed human and AI supervision This allows us to evaluate models at scale while preserving alignment with expert human preferences. 📊 We benchmark 21 systems, including proprietary, open-source, and agentic models. Our results reveal persistent gaps in instruction adherence, physical reasoning, structural control, and fine-grained detail preservation. 🤖 Finally, we develop 𝗖𝗩-𝗔𝗴𝗲𝗻𝘁, a lightweight agentic baseline that combines planning, editing, and verification. The results suggest that closed-loop reasoning is a promising direction for professional-grade instruction-following visual editing. 💡 The main takeaway: as visual AI moves toward real workflows, the challenge is no longer only to generate visually plausible images. Models must also understand intent, preserve constraints, reason about structure and physics, and verify whether the edit actually solves the requested visual problem. 𝗣𝗿𝗼𝗷𝗲𝗰𝘁: ark1234.github.io/cv-arena 𝗖𝗼𝗱𝗲: github.com/taco-group/CV-… #ComputerVision #GenerativeAI #MultimodalAI #ImageEditing #AIAgents #Benchmarking #CVArena #TAMU
Zhengzhong Tu tweet media
English
2
12
52
7K
Zhengzhong Tu
Zhengzhong Tu@_vztu·
𝗘𝘃𝗲𝗿𝘆𝗼𝗻𝗲 𝘄𝗮𝗻𝘁𝘀 𝟰𝗞 𝗔𝗜. But true native-4K data is still surprisingly scarce. 🚀 Excited to share our new work: 𝟰𝗞𝗟𝗦𝗗𝗕: 𝗔 𝗟𝗮𝗿𝗴𝗲-𝗦𝗰𝗮𝗹𝗲 𝗗𝗮𝘁𝗮𝘀𝗲𝘁 𝗳𝗼𝗿 𝟰𝗞 𝗜𝗺𝗮𝗴𝗲 𝗥𝗲𝘀𝘁𝗼𝗿𝗮𝘁𝗶𝗼𝗻 𝗮𝗻𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻, accepted to CVPR 2026 DataCV. Most public datasets are built around sub-1K, HD, or 2K images. But at 4K resolution, small artifacts become big problems: blurry textures, distorted boundaries, repeated patterns, and missing fine details. To address this gap, we introduce 𝟰𝗞𝗟𝗦𝗗𝗕, a large-scale native-4K dataset and benchmark for high-resolution restoration and generation. 📌 𝟰𝗞𝗟𝗦𝗗𝗕 𝗶𝗻𝗰𝗹𝘂𝗱𝗲𝘀: ✅ 129K native-4K training images ✅ 2K validation images and 1,984 test images ✅ Diverse categories: nature, urban scenes, people, food, artwork, CGI, and more ✅ Aligned 4K image–text pairs for generative modeling ✅ Paired LR/HR evaluation sets for super-resolution We also build a multi-stage curation pipeline combining resolution filtering, LMM-based quality scoring, texture-richness filtering, and human verification. Across classical SR, real-world blind SR, and 4K text-to-image generation, fine-tuning on 4KLSDB consistently improves fidelity, local detail, perceptual quality, and human preference. 💡 Main takeaway: 𝗻𝗮𝘁𝗶𝘃𝗲-𝟰𝗞 𝘀𝘂𝗽𝗲𝗿𝘃𝗶𝘀𝗶𝗼𝗻 𝗺𝗮𝘁𝘁𝗲𝗿𝘀. As visual AI moves toward ultra-high-resolution restoration and generation, we need datasets and benchmarks that expose the fine-scale failures hidden by low-resolution evaluation. 📄 𝗣𝗿𝗼𝗷𝗲𝗰𝘁 𝗽𝗮𝗴𝗲: 4klsdb.github.io 💻 𝗚𝗶𝘁𝗛𝘂𝗯: github.com/taco-group/4KL… 💽 𝗗𝗮𝘁𝗮𝘀𝗲𝘁: huggingface.co/datasets/Singl… #ComputerVision #GenerativeAI #ImageRestoration #SuperResolution #TextToImage #DiffusionModels #Dataset #Benchmarking #TAMU
Zhengzhong Tu tweet media
English
1
8
60
5.2K
Zhengzhong Tu retweetledi
reactor
reactor@reactorworld·
Today Reactor is coming out of stealth. We’ve raised $59M in Seed and Series A funding, led by @lightspeedvp, with participation from @AmplifyPartners, @wndrco, @Sky9Capital, and @FPVventures. Reactor is the platform for building in the World Model era: the infrastructure that lets developers build with them at global scale for the first time. Stream from a frontier World Model to your app, in real time, all in under 10 lines of code. World Models represent the next major shift in AI: pixels, audio and actions are generated on the fly, in real-time, in response to user inputs, and to the environment. Every time computing has made a shift from passive to interactive, entire industries appeared that didn't exist before. We're standing in front of such moment again. Over the last 6 months, we’ve assembled an all-star team with alumni from Apple, Meta, Google, Luma AI, Netflix, and Replicate. We're already partnering with some of the biggest names and labs in the world, and hundreds of developers are already building on Reactor. The World Model era starts now.
English
177
230
2.3K
12.7M
Zhengzhong Tu
Zhengzhong Tu@_vztu·
🎉 Excited to share our new work accepted to #CVPR2026 “𝗡𝗲𝘅𝘂𝘀𝗙𝗹𝗼𝘄: 𝗨𝗻𝗶𝗳𝘆𝗶𝗻𝗴 𝗗𝗶𝘀𝗽𝗮𝗿𝗮𝘁𝗲 𝗧𝗮𝘀𝗸𝘀 𝘂𝗻𝗱𝗲𝗿 𝗣𝗮𝗿𝘁𝗶𝗮𝗹 𝗦𝘂𝗽𝗲𝗿𝘃𝗶𝘀𝗶𝗼𝗻 𝘃𝗶𝗮 𝗜𝗻𝘃𝗲𝗿𝘁𝗶𝗯𝗹𝗲 𝗙𝗹𝗼𝘄 𝗡𝗲𝘁𝘄𝗼𝗿𝗸𝘀” In textbooks and benchmarks, datasets are often neatly annotated for every task. In the real world, they rarely are. Data is collected at different times, in different places, and for different purposes. One dataset may contain labels for mapping, another for tracking, another for depth or segmentation. Does that mean fragmented data has to be discarded? 💪 𝗢𝘂𝗿 𝗮𝗻𝘀𝘄𝗲𝗿: 𝗻𝗼. We show that partially supervised, heterogeneous data can still be highly valuable—and in some cases, can even outperform fully annotated data. How do we learn across structurally different tasks when labels are only partially available? 💡 𝗢𝘂𝗿 𝗦𝗼𝗹𝘂𝘁𝗶𝗼𝗻: 𝗡𝗲𝘅𝘂𝘀𝗙𝗹𝗼𝘄 NexusFlow is a lightweight, plug-and-play framework that aligns disparate tasks in a shared latent space. What makes it work: • 🔄 𝗜𝗻𝘃𝗲𝗿𝘁𝗶𝗯𝗹𝗲 𝗳𝗲𝗮𝘁𝘂𝗿𝗲 𝗮𝗹𝗶𝗴𝗻𝗺𝗲𝗻𝘁. Invertible coupling layers map task features into a unified canonical space. Since the mapping is bijective, task information is preserved, helping avoid the representational collapse often seen in vanilla alignment methods. • 🔌 𝗣𝗹𝘂𝗴-𝗮𝗻𝗱-𝗽𝗹𝗮𝘆 𝗱𝗲𝘀𝗶𝗴𝗻. No need to modify task heads or losses. NexusFlow can be added to BEV-based backbones with a simple alignment loss. • 📈 𝗦𝗰𝗮𝗹𝗮𝗯𝗹𝗲 𝘁𝗼 𝗺𝘂𝗹𝘁𝗶𝗽𝗹𝗲 𝘁𝗮𝘀𝗸𝘀. The method scales as O(N) with one surrogate branch per task, making extension to 3+ tasks straightforward. • 📐 𝗧𝗵𝗲𝗼𝗿𝗲𝘁𝗶𝗰𝗮𝗹 𝗴𝗿𝗼𝘂𝗻𝗱𝗶𝗻𝗴. Invertibility provides a provable bound that connects the alignment loss to cross-task knowledge transfer. 🏆 𝗥𝗲𝘀𝘂𝗹𝘁𝘀 NexusFlow sets a new state of the art on nuScenes for domain-partitioned autonomous driving, where online map reconstruction and multi-object tracking are supervised in different geographic regions. It also delivers consistent gains across all three NYUv2 tasks: semantic segmentation, depth estimation, and surface normal prediction. 📎 𝗣𝗿𝗼𝗷𝗲𝗰𝘁 𝗽𝗮𝗴𝗲: ark1234.github.io/nexusflow_web 🤝 This work was conducted in collaboration across Worcester Polytechnic Institute, Texas A&M University, Tohoku University, University of Michigan, and Bosch Research. Huge thanks to collaborators: Fangzhou Lin, Yuping Wang, Yuliang Guo, Zixun Huang, Xinyu Huang, Haichong Zhang, Kazunori Yamada, Zhengzhong Tu, Liu Ren, and Ziming Zhang. #CVPR2026 #ComputerVision #MultiTaskLearning #AI #GenAI #AutonomousDriving #DeepLearning #RepresentationLearning
Zhengzhong Tu tweet media
English
0
1
18
2.2K
Higgsfield AI 🧩
Higgsfield AI 🧩@higgsfield_ai·
18-minute breakdown of Claude + Higgsfield MCP. Here's what the video walks through. 1. Research across TT, IG, YT 2. Production calendar by the agent 3. UGC, motion design, carousels 4. Approval gate before spend 5. Integrate Meta Ads MCP Marketing agency in your laptop.
English
143
231
1.8K
203.4K
Zhengzhong Tu retweetledi
Zhiwen(Aaron) Fan
Zhiwen(Aaron) Fan@zhiwen_fan_·
We are working together to launch the Rising Star Award for Spatial Intelligence. Applications are due in 10 days. Thanks to 2077AI for sponsoring a $30K research gift fund to support a PhD student or postdoc advancing spatial intelligence. Apply by May 22: e2e3d.github.io/rising_star.ht… Thanks to ChatGPT for helping turn a simple prompt into this poster :)
Zhiwen(Aaron) Fan tweet media
English
0
4
29
13.1K