Pauline Luc

Scaling 4D Representations – new preprint arxiv.org/abs/2412.15212 and models now available github.com/google-deepmin…

So pleased and proud to share with you what our team has been up to, on an ambitious journey to build a video foundation model for scientific domains ! ✨ 🚀 🎞️ 🧪 #ICCV2025 #AI4Science

We're excited to release TAPVid-3D: an evaluation benchmark of 4,000+ real world videos and 2.1 million metric 3D point trajectories, for the task of Tracking Any Point in 3D!

We present a new SOTA on point tracking, via self-supervised training on real, unlabeled videos! BootsTAPIR achieves 67.4% AJ on TAP-Vid DAVIS with minimal architecture changes, tracks 10K points on a 50-frame video in 6 secs. Pytorch & JAX impl on Github. bootstap.github.io

Thrilled to present to you Gemma! A family of lightweight, state-of-the art and open models by @GoogleDeepMind. We provide both pre-trained and fine-tuned checkpoints for easy tuning, responsible development, and community-driven innovation! More info at ai.google.dev/gemma

Our latest release @MistralAI Mixtral 8x7B mixture of experts - performance of a GPT3.5 - inference cost of a 12B model - context length of 32K - speaks English, French, Italian, German and Spanish Blog post mistral.ai/news/mixtral-o…

Mixtral 8x7B is here, 11 weeks only after Mistral 7B. Outperforms Llama 2 70B and GPT 3.5 on most benchmarks, at the inference cost of a 12B dense model, with 32k tokens context size.

Thrilled to share #Lyria, the world's most sophisticated AI music generation system. From just a text prompt Lyria produces compelling music & vocals. Also: building new Music AI tools for artists to amplify creativity in partnership w/YT & music industry deepmind.google/discover/blog/…

Demystifying CLIP Data Reveals CLIP’s data curation approach and makes it open to the community repo: github.com/facebookresear… abs: arxiv.org/abs/2309.16671

Okay, so that AI letter signed by lots of AI researchers calling for a "Pause [on] Giant AI Experiments"? It's just dripping with #Aihype. Here's a quick rundown. >>

🦩 Introducing OpenFlamingo! A framework for training and evaluating Large Multimodal Models (LMMs) capable of processing images and text. More details below (including a multimodal LLaMA model!)⬇️ Blog: laion.ai/blog/open-flam… Demo: 7164d2142d11.ngrok.app

Introducing Vid2Seq, a new visual language model for dense video captioning. To appear at #CVPR2023. Work done @Google w/ @NagraniArsha P.H. Seo @antoine77340 @jponttuset I. Laptev J. Sivic @CordeliaSchmid. Page: antoyang.github.io/vid2seq.html Paper: arxiv.org/abs/2302.14115 🧵/5

In case you missed it...Flamingo 🦩 a new SOTA visual language model. Read more below ⬇️ Paper: dpmd.ai/dm-flamingo-pa… Blog: dpmd.ai/dm-flamingo

Chatting with Flamingo about images is definitely the most organic experience I’ve had with an ML model. The ability to readily describe output from e.g. DALL-E 2 might be the closest we’ve come to two independently-trained large-scale models having a conversation 👀

Finally able to share what I have been working on this year! 🦩 Tldr: We took our best LM (Chinchilla), froze it and added new visual layers to it and trained 🦩 on full webpages with images instead of just image-text pairs. Check out the visual dialogue examples from the paper!

10B extra parameters for adaptation and visual conditioning, new cross-modality data and a lot of love makes Chinchilla able to see !

A group of Flamingos is called “flamboyance” which could be an apt description for the family of vision-language models I’m thrilled to see out in the wild! I believe using large pre-trained models in creative ways will be key and hope our work is a step in the right direction.

Great to finally share our newest addition to the DeepMind large-scale model zoo! 🦩

Was a total pleasure to be part of the team for Flamingo. Lots of exciting capabilities that we are just beginning to explore.

A lot happenned in the last year ! I defended my PhD and joined @DeepMind where I worked with an incredible team on Flamingo🦩, a visual language model. Flamingos can fly, they can dance, and this one writes pretty well too !