Dong-Hwan Jang

@CVPR scheduling can be chaotic, so we made … a PLAN. 😄🗺️ ✨𝗣𝗟𝗔𝗡 𝗟𝗮𝗯 is heading to Denver with 7 papers at CVPR 2026, and we put everything in one place so you can browse the work, watch the videos, find the posters, and add sessions to your calendar.📌 Explore our #CVPR2026 hub and come meet us in person 😊 Click on the banner plan-lab.github.io Direct link plan-lab.github.io/cvpr2026 See you in Denver!🏔️ #CVPR2026 #CVPR26 #ComputerVision #MultimodalAI #GenerativeAI #EmbodiedAI

🚨 MOCHA is now released. ☕️️ A new benchmark for evaluating code LLM safety under multi-turn attacks. Can your model resist malware requests when the intent is hidden across seemingly harmless steps? 🧵⬇️

Thank you for your interest in our work. First, we want to clarify that our method is a weight merging technique for fine-tuned models trained with different random seeds. Merging two fine-tuned models trained under different settings can lead to unexpected behaviors, as these models are not guaranteed to lie on the same thin shell. Our approach also offers insights into merging FT models with different hyperparameters (please refer to the discussion section). Based on this, we are developing a new merging method that can accommodate FT models with diverse setups. Please stay tuned for more updates.

@dhjang10 @ryanmart3n We see how task_arithmetic continues to be the complete and dominant merge method despite new methods emerging and attempting to overtake it. Just look at these two coding examples from Ai using the Top 2 merge methods. task_arithmetic and model_stock

arxiv.org/abs/2403.19522… Model Stock: All we need is just a few fine-tuned models @JagersbergKnut @chidambara09 @KevinClarity @enilev @EstelaMandela @RLDI_Lamy @AchuthaSubhash

[LG] Model Stock: All we need is just a few fine-tuned models D Jang, S Yun, D Han [NAVER AI Lab] (2024) arxiv.org/abs/2403.19522 - Fine-tuned models with different random seeds lie on a thin shell in weight space layer-wise. - Proximity of weights to the center of this thin shell correlates with better in-distribution and out-of-distribution performance. - Averaging weights of multiple fine-tuned models approaches the center, improving performance. - A novel method called Model Stock approximates a center-close weight using only two fine-tuned models. - Model Stock leverages geometric properties of weight space and pre-trained model as anchor point. - Model Stock achieves strong performance using only two fine-tuned models, surpassing other model merging methods. - Periodic merging during training moves weights even closer to center for better performance.

NAVER AI Lab Introduces Model Stock: A Groundbreaking Fine-Tuning Method for Machine Learning Model Efficiency Quick read: marktechpost.com/2024/04/01/nav… Researchers at the NAVER AI Lab have introduced Model Stock, a fine-tuning methodology that diverges from conventional practices by requiring significantly fewer models to optimize final weights. What sets Model Stock apart is its utilization of geometric properties in the weight space, enabling the approximation of a center-close weight with only two fine-tuned models. This innovative approach simplifies the optimization process while maintaining or enhancing model accuracy and efficiency. In implementing Model Stock, the team conducted CLIP architecture experiments, focusing primarily on the ImageNet-1K dataset for in-distribution performance analysis. They extended their evaluation to out-of-distribution benchmarks to further assess the method’s robustness, specifically targeting ImageNet-V2, ImageNet-R, ImageNet-Sketch, ImageNet-A, and ObjectNet datasets. The choice of datasets and the minimalistic approach in model selection underscore the method’s practicality and effectiveness in optimizing pre-trained models for enhanced task-specific performance.

@maximelabonne @maximelabonne Thank you so much for your interest in our work! We're thrilled to see our merging method's success in LLMs. Your sharing of the results is greatly appreciated :) Please stay tuned for more of our research!

⚙️ Model Stock: new SOTA merge method This technique was implemented in MergeKit just four days after the paper was published. I used it to create a new type of merge with Zebrafish-7B. It worked so well I already added it to AutoMerger. 🤗 Model: huggingface.co/mlabonne/Zebra…

Devoured all papers related to model merging & rarely does anything shine like "Model Stock" arxiv.org/abs/2403.19522. Clear findings, wise insights and splendid figures. For anyone merging models or interested in finetuning, this is your must-read. TLDR; interpolate towards init.

@ramealexandre Thanks, @ramealexandre, for highlighting Model Stock! Honored by your support. We aim to impact the pretrain-finetune field significantly. Please stay tuned for the upcoming research too :)

For every 5 likes this gets, I will ask ChatGPT to make this ramen look more profitable.

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding project page: gweb-research-imagen.appspot.com sota FID(7.27 on COCO), without ever training on COCO, human raters find Imagen samples to be on par with the COCO data itself in image-text alignment

There's a new competitor to DALL-E out there: Google's Imagen. I don't have access to Imagen but I do have some of their example images. Let's do some side-by side comparisons. (left is Imagen, right is DALL-E) 1. "A blue jay standing on a large basket of rainbow macarons."

Asked #Dalle to generate “Photograph of a crowded Tokyo street full of business people wearing VR headsets going to work”

"Focusing is about saying no." - Steve Jobs

Are you a student from underrepresented groups and applying for graduate school in CS this year? Check out many wonderful (student-led) pre-application programs for supporting PhD applicants!

How to network in a virtual conference?🕸️ Excited to attend your first (virtual) conference? How do we meet new people and expand the professional network? Some ideas that I found useful 👇👇👇

@docmilanfar @CSProfKGD Just glad to see more works discussing these issues which I think to be overlooked in the vision community. Ideally, we want the network architectures can work in arbitrary image resolution during training and inference.

(1/n) How to start a deep learning project? We use a remarkably streamlined step-by-step process to set up deep learning projects. At the same time, people who are new to deep learning tend to always make the same (avoidable) mistakes. Check out the thread below! 🧵

How to make steady progress in my research? I worked so damn hard but "IT JUST DOESN'T WORK!"😤 How can I unblock myself quickly and make good progress toward the goals? Below I compiled a list of tips that I found useful. 👇