Ani Aggarwal

🧵 Your DiT, faster Introducing ECAD: we reframe diffusion model caching as multi-objective optimization and evolve Pareto-optimal schedules via a genetic algorithm—achieving 4.47 FID gain at 2.58× speedup, with no retraining or tuning. 🔗 aniaggarwal.github.io/ecad #MachineLearning

@LunjunZhang Agreed on this. I'm very bullish on using EAs in particular in areas that gradients are expensive / impossible to commute and RL would be slow or unstable (see my paper ECAD). Combining them is super clever :)

RL optimizes weights. Evolution optimizes contexts. What if we combine RL and Evolutionary Algorithm (EA) into a new paradigm of LLM self-improvement? In "Evolutionary System Prompt Learning for Reinforcement Learning in LLMs", we show that RL and EA are deeply synergistic.

@wimmer_th Congrats! Would love to chat in Rio!

AnyUp has been accepted to ICLR 2026 as an oral presentation! ⭐️ I'm looking forward to presenting it in Rio. If you're interested, please come by my talk or poster. 🇧🇷

🦖 Pixel 🦕 dense 🦖 DINO 🦕 features and image super resolution 🖼️!! #CVPR2026

@massiviola01 Low memory mode is live!

@AnirudAgg Ok nice, definitely will take a look again!

DINO features are the best in town and offer unmatched performance for dense vision tasks such as classification and detection. But in the current multimodal ML landscape, these features have a limitation: they lack a direct connection to natural language! Unlike CLIP-style models, DINO embeddings are not spatially aligned with text, and this means the model cannot understand queries, perform zero-shot classification/retrieval, or highlight all the patches matching a given concept. Fortunately for us, the DINO team addressed this issue with dino.txt, a simple approach to align frozen DINO backbones with a text encoder to solve tasks associated with a text prompt, both at the global and local levels. Think about CLIP and Segment Anything 3 abilities baked into a single model, and this was a couple of years ago! Curious to know how it's done and some practical takeaways for your everyday vision tasks? Check out the resources below!⏬ dino.txt: DINO Meets Text open.substack.com/pub/mlhonk/p/5… Paper: arxiv.org/abs/2412.16334…

@massiviola01 Yes! We have a low memory mode that is super efficient in memory, but adds some latency (by running some operations in sequence rather than in parallel). We will add this to the released code repo this weekend! Thanks for your interest in our work 😁

@AnirudAgg Yes, I am a bit familiar with this line of work, and it's great offline. The problem I noticed is that it eats a lot of GPU memory even for a 224 image, not ideal for production inference with lots of cameras! Do you know if there is anything that adds only a minimal overhead?

🎉 Thrilled to share that my first research paper has been accepted to #ICLR2026! 🎉 I’ll be attending in person in Rio and would love to connect with others! 🇧🇷

Technically more than one line of code 😁

Super excited to announce our paper! Upsample any latents (DINO, VAE, etc.) in linear time (compared to quadratic cross attention). Our models are all available on Hugging Face and Torch Hub with just one line of code! (please star the GitHub repo 🥺) huggingface.co/UPLiFT-upsampl…

@HrishbhDalal @MatthewWalmer @_sakshams_ @abhi2610 Agreed, but even without a larger size, I'm of the opinion that using a modern VAE and multi scale image generator for training (FLUX, maybe?, which has larger latents) would achieve really strong image super res, super efficiently.

@MatthewWalmer @_sakshams_ @AnirudAgg @abhi2610 i think if you added a layer of attention and 2-4 decoder layers of moderate width, you should be able to convert this to complete images quite fast...

We’re excited to announce UPLiFT, our lightweight, pixel-dense feature upsampler. UPLiFT boosts feature density, preserves semantics, and has better efficiency scaling than recent SOTA methods. See all links in the thread below. Coauthors: @_sakshams_ @AnirudAgg @abhi2610 🧵[1/6]

@prodarhan This is sick, love the website side by sides and clean animations 😍

Excited to introduce PolaRiS, a real-to-sim recipe for turning short real-world videos into high fidelity simulation environments for scalable and reliable zeroshot generalist policy evaluation. polaris-evals.github.io (1/N 🧵)

@QianqianWang5 Super interested and will definitely be applying! 😁

I'm recruiting multiple PhD students this cycle to join me at Harvard University and the Kempner Institute! My interests span vision and intelligence, including 3D/4D, active perception, memory, representation learning, and anything you're excited to explore! Deadline: Dec 15th.

@prodarhan @IamKyros69 Only if you first reconceptualize diffusion caching as a multi-objective optimization problem and apply a genetic algorithm to find an optimal speed quality trade-off to do inference with

@AnirudAgg @IamKyros69 heard xps is rlly good for efficient inference for diffusion models via caching

Why is everyone in tech using a ThinkPad?

@LogitechG No thanks my secondhand G502 is going strong 7 years later

Imagine pulling up to this house

@prodarhan can we get a comparison to Mr Pilu as a baseline??

Checkout Mateo’s work! He put a lot of effort into the videos and figures. Lotta fun to play around with the gradio too

@psandovalsegura Super interesting! Also seeing computational redundancy in DiTs. Some tokens cache well across timesteps (arXiv:2410.05317) and some entire blocks can be skipped (hasty plot). Maybe they could be pruned like your attention heads? Though uncertain of its practical application haha

Attention sinks in LLMs are weird. There’s ~20% of heads that don’t seem to do anything. Do these heads matter? Turns out that if we get rid of them, benchmark scores don’t change.

Super grateful to co-authors @abhi2610 & Matthew Gwilliam. Preprint, code, and more results: aniaggarwal.github.io/ecad. Feedback greatly appreciated! 👇

ECAD discovers more intricate caching schedules than heuristics can design. This PixArt-α schedule produced the previous images (using 20 diffusion steps)! Red = cached components, gray = recomputed.

🧵 Your DiT, faster Introducing ECAD: we reframe diffusion model caching as multi-objective optimization and evolve Pareto-optimal schedules via a genetic algorithm—achieving 4.47 FID gain at 2.58× speedup, with no retraining or tuning. 🔗 aniaggarwal.github.io/ecad #MachineLearning