Rowan Zellers

Excited to introduce GPT-4o. Language, vision, and sound -- all together and all in real time. This thing has been so much fun to work on. It's been even more fun to play with -- with moments of magic where things feel totally fluid and I forget I'm video chatting with an AI.

I find it amusing that Claude/Codex will describe race conditions as "racy"

🚀 Announcing CUA-Suite, a computer-use agent (CUA) training and evaluation ecosystem based on the largest open expert video corpus for desktop CUAs – VideoCUA. 55 hours of human demonstrations across 87 professional apps — 2.5× bigger than the previous largest dataset. 🌐 cua-suite.github.io

Building technologies for better human-AI collaboration on next gen hardware at scale. Exciting.

We’re bringing 1GW compute online, with more to come-

Really cool work by the SI team! It's impressive to see a low level inverse dynamics model for computer use work here... both for computer tasks as well as a bonus self driving car demo

I realized an interesting connection between climbing and research recently In climbing, sometimes you can send a boulder problem and then get 'stuck' with a certain beta, with a bunch of moves that seem critical but end up being arbitrary. Cleaning it up by doing boulder repeats is valuable but hard. You're fighting against your mind telling you certain moves are necessary. Same is true for frontier research. Any model training recipe probably has a bunch of pieces that are unnecessary, or perhaps even harmful to performance. To make progress requires constantly questioning old truths. Just like boulder repeats -- careful experimentation is hard but it can be fun. A healthy lab is one where this is encouraged 😀

I’ve been telling people this a lot today: I enjoy so much working with people who care about what they are building and craftsmanship. It is a privilege to have a chance to work on something I’m passionate about, beyond making a living. I cherish it and don’t take it for granted.

doesn’t the name ‘Shimpo VL-Whisper’ sound like a vision / language / audio model?

@BigMeanInternet fair but then isn’t the question ‘why aren’t capitalists investing one trillion dollars into iphone-of-ebikes like companies?’

@rown I bet if capitalists invested one trillion dollars he wouldn't have gone bankrupt.

Why build all these data centers rather than the iphone of electric bikes? How come no one wants to build the iphone of electric bikes?

Discrete or continuous tokens? Or even tokenizer-free? The visual modeling debate rages on, but for now, let me introduce L24SQ, a provably optimal, regularizer-free quantizer with a large codebook (~200k), achieving SoTA reconstruction-compression tradeoff and generative power!

@ziqiao_ma I might have missed this, do you have the ablation where you predict the next patch’s pixels via l2 loss instead of the embedding?

NEPA: Next-Embedding Predictive Autoregression A simple objective for visual SSL and generative pretraining. Instead of reconstructing pixels or predicting discrete tokens, we train an autoregressive model to predict the next embedding given all previous embeddings. Key ideas: - One self-supervised signal: cosine-style next-embedding prediction - Autoregression runs directly on the embeddings from a native encoder (no offline encoder) - No pixel decoder (and loss), no contrastive pairs, no task-specific heads, no random masks Scales into modern ViT backbones and stays competitive after supervised fine-tuning: - ImageNet-1K (Base 83.8%; Large 85.3%) - ADE20K Fully open-sourced with reproducibility verified: - Homepage: sihanxu.me/nepa/ - Paper: arxiv.org/abs/2512.16922 - Code: github.com/SihanXU/nepa - Weights: huggingface.co/collections/Si… This work is led by @6SihanXu and advised by @SLED_AI, @sainingxie, and Stella X. Yu. Contributors: me, @wenhaocha1, @ChenXuweiyi, and @JinWeiyang18434.

@bengoodger My computer crashed and I had to restart :/

@rown Before you do would you be able to show the details of the atlas processes in activity Monitor? We are hunting some leaks and any info is helpful. Thank you!!

I was playing around with ChatGPT atlas browser (congrats @bengoodger and team). A lot of cool ML + UX codesign! I had it go to amazon and help me purchase things. Curiously... ChatGPT atlas adds some prompt injection... which the latest model recognizes, flags, and ignores (!)

@bengoodger FYI more thing you might want to fix (I haven’t used it in the last few days… just kept the app open. Will uninstall now)

@rown Thanks for sharing! We're working on a fix.

Last year Molmo set SOTA on image benchmarks + pioneered image pointing. Millions of downloads later, Molmo 2 brings Molmo’s grounded multimodal capabilities to video 🎥—and leads many open models on challenging industry video benchmarks. 🧵

Also check out the cookbook example that @brandontrabucco put together for finetuning Qwen3-VL-235B on image classification github.com/thinking-machi… this is just scratching the surface, excited to see what the community can do with this!

Today we are releasing Tinker to everyone, and now with vision input! You can now finetune a frontier Qwen3-VL-235B on your own image+text data, bringing your own algorithm (sft, RL, something else?). We'll take care of the GPU infra. Full update: thinkingmachines.ai/blog/tinker-ge…

Such a cool showcase of powerful unified model, being able to interleavely generating text, thinking, and video! Great work, congrats!!