Jie Lei

Today we’re excited to unveil a new generation of Segment Anything Models: 1️⃣ SAM 3 enables detecting, segmenting and tracking of objects across images and videos, now with short text phrases and exemplar prompts. 🔗 Learn more about SAM 3: go.meta.me/591040 2️⃣ SAM 3D brings the model collection into the 3rd dimension to enable precise reconstruction of 3D objects and people from a single 2D image. 🔗 Learn more about SAM 3D: go.meta.me/305985 These models offer innovative capabilities and unique tools for developers and researchers to create, experiment and uplevel media workflows.

Meet SAM 3, a unified model that enables detection, segmentation, and tracking of objects across images and videos. SAM 3 introduces some of our most highly requested features like text and exemplar prompts to segment all objects of a target category. Learnings from SAM 3 will help power new features in Instagram Edits and Vibes, bringing advanced segmentation capabilities directly to creators. 🔗 Learn more: go.meta.me/591040

I am humbled to be re-featured as Women in Computer Vision for the BEST of CVPR section of the Computer Vision News July Magazine. It was great chatting with Ralph Anzarouth. I hope my unconventional career path can encourage more female researchers. rsipvision.com/ComputerVision…

Knowledge vs Large Models? Welcome to our #CVPR23 tutorial "Knowledge-Driven Vision-Language Encoding" with @Xudong_Lin_AI @jayleicn @mohitban47 @cvondrick @Shih-Fu Chang @elgreco_winter Jun 19: 9:00-12:30 Loc: East 8 Website:blender.cs.illinois.edu/tutorial/Knowl… Zoom:cvpr2023.thecvf.com/virtual/2023/t…

Can pretrained vision models generalize to audio-visual data without any audio pretraining? In our #CVPR2023 paper "Vision Transformers are Parameter-Efficient Audio-Visual Learners," we show that they can! yanbo.ml/project_page/L… w/ @yblin98 @yilin_sung @jayleicn @mohitban47

What makes modern Video-Language (VidL) perform well? Check out our #CVPR2023 paper "VindLU: A Recipe for Effective Video-and-Language Pretraining" where we demystify the most critical factors in the VidL model design. klauscc.github.io/vindlu.html @fncheng2333 @jayleicn @mohitban47

What is the value of knowledge in the era of large-scale pretraining? Welcome to our #AAAI23 tutorial "Knowledge-Driven Vision-Language Pretraining" with @Xudong_Lin_AI @jayleicn @mohitban47 @Shih-Fu Chang @elgreco_winter Feb 8: 2-6pm Loc: Room 201 Zoom: underline.io/events/389/ses…

Self-attention for VL tasks (esp. video+text) is too expensive! Check out our #WACV2023 paper “Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention” arxiv.org/abs/2211.11701 github.com/zinengtang/Per… @ZinengTang* @jmin__cho* @jayleicn @mohitban47 🧵

YouTube

Do you still get Out-of-Memory error even when you've saved >95% params w. adapter/prompt-tuning? Try Ladder Side-Tuning (LST 🪜) for both Parameter & Memory Efficient Transfer Learning (in NLP+VL tasks)! arxiv.org/abs/2206.06522 github.com/ylsung/Ladder-… @jmin__cho @mohitban47 🧵

Can GPT-3 understand videos? Glad to share our new work VidIL on prompting LLMs to understand videos using image descriptors (frame caption + visual token). We show strong few-shot video-to-text generation ability WITHOUT the need to train on ANY videos: arxiv.org/abs/2205.10747

At Microsoft Research, we aim to empower the next generation of computing related research talent. Today, we're thrilled to announce and congratulate this year's Microsoft Research PhD Fellowship recipients from around the world. Meet the 2022 recipients: aka.ms/phdfellowship

A big congrats to 2022’s cohort of Google PhD Fellows, who are part of our program that recognizes and supports graduate students doing exceptional research in computer science and related fields. Read more about the experience from three alums ↓ goo.gle/3CR3AXf

TVLT: Textless Vision-Language Transformer abs: arxiv.org/abs/2209.14156 github: github.com/zinengtang/TVLT

🥳🥳 Check out our #ECCV2022 oral paper. We propose ECLIPSE 🌒 that integrates audio🔊🎵 into popular CLIP to have 2.92x faster and 2.34x memory-efficient for long-range video retrieval. arxiv.org/abs/2204.02874 yanbo.ml/project_page/e… w. @jayleicn @mohitban47 @gberta227 🧵👇