HPC-AI Lab

One static model does not fit all😭 We just dropped our latest work: Functional Neural Memory. Instead of static models, we generate custom "parameters" for every single input. ✅Prompt your model anytime ✅Instant personalization ✅Better instruction following ✅Flexible & dynamic memory (w/o memory bank✌️) (🧵1/6)

Nice work of Ziming!

Customizing Your LLMs in seconds using prompts🥳! Excited to share our latest work with @HPCAILab, @VITAGroupUT, @k_schuerholt, @YangYou1991, @mmbronstein, @damianborth : Drag-and-Drop LLMs(DnD). 2 features: tuning-free, comparable or even better than full-shot tuning.(🧵1/8)

Representation Alignment (REPA) is NOT ALWAYS helpful for diffusion training!🤷 Sharing latest work w/ @HPCAILab and @VITAGroupUT: "REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training". Acceleration up to 28x w/o performance drop.(🧵1/7)

Congrats to our lab member @lzm_mlsys !

🚀 We’re thrilled to introduce DD-Ranking: Rethinking the Evaluation of Dataset Distillation—a unified, user-friendly, and long-term maintained benchmark for dataset distillation (DD)! 📢 Together with 20 institutions worldwide, we’re releasing our Repo, API Documentation, and Leaderboard: 🔗 Repo: github.com/NUS-HPC-AI-Lab… 📖 Documentation: nus-hpc-ai-lab.github.io/DD-Ranking/ 🏆 Leaderboard: huggingface.co/spaces/logits/… 🧵1/7

Generating ~200 million parameters in just minutes! 🥳 Excited to share our work with @MTDovent , @heisejiasuo96 , and @YangYou1991: 'Recurrent Diffusion for Large-Scale Parameter Generation' (RPG for short). Example: Obtain customized models using prompts (see below). (🧵1/8)

Training-free Video Enhancement: Achieved 🎉 Nice work with @oahzxl @shaowenqi126301 @VictorKaiWang1 @VitaGroupUT @YangYou1991 et al. Non-trivial enhancement, training-free, and plug-and-play 🥳 Blog: oahzxl.github.io/Enhance_A_Vide… (🧵1/6)

Exciting News from Open-Sora! 🚀 They've just made the ENTIRE suite of their video-generation model open source! Dive into the world of cutting-edge AI with access to model weights, comprehensive training source code, and detailed architecture insights. Start building your dream video-generation model today! Check it out 👉 github.com/hpcaitech/Open…

Neural Network Diffusion Diffusion models have achieved remarkable success in image and video generation. In this work, we demonstrate that diffusion models can also generate high-performing neural network parameters. Our approach is simple, utilizing an autoencoder and a standard latent diffusion model. The autoencoder extracts latent representations of a subset of the trained network parameters. A diffusion model is then trained to synthesize these latent parameter representations from random noise. It then generates new representations that are passed through the autoencoder's decoder, whose outputs are ready to use as new subsets of network parameters. Across various architectures and datasets, our diffusion process consistently generates models of comparable or improved performance over trained networks, with minimal additional cost. Notably, we empirically find that the generated models perform differently with the trained networks. Our results encourage more exploration on the versatile use of diffusion models.

InfoBatch is accepted as Oral to ICLR'24! 🔥 InfoBatch prunes data on the fly and speedups 20%~40% on img classification, semantic segmentation, MAE, Diffusion, LLM instruction tunning.🧵3 ArXiv: arxiv.org/abs/2307.02047 Blog: zhengzangw.github.io/blogs/infobatch Code: github.com/NUS-HPC-AI-Lab…

I am happy to share that our paper has been accepted by ICLR as an ORAL paper (1.2% acceptance rate). InfoBatch: Lossless Training Speed Up by Unbiased Dynamic Data Pruning arxiv.org/abs/2303.04947 InfoBatch randomly prunes a portion of less informative samples based on the loss distribution and rescales the gradients of the remaining samples to approximate the original gradient. As a plug-and-play and architecture-agnostic framework, InfoBatch consistently obtains lossless training results on classification, semantic segmentation, vision pertaining, and instruction fine-tuning tasks. On real-world applications, InfoBatch can losslessly save 40% overall cost.

📢 Join us for the HPC-AI Lab Public Seminar! 🔗 Registration: forms.gle/4anywqoXtSumHu… 🗓️ Date/Time: 29 Nov. 2023 (Wednesday), 1 PM to 2 PM 📍 Online via Zoom

Time flies! I got my PhD from Berkeley 1218 days ago. My first PhD student is graduating. That is my first achievement :-)

Excited to share our #ICCV2023 paper: Fine-tuning Vision-Language Models without Zero-Shot Transfer Degradation (ZSCL). ZSCL outperforms the pre-trained model on downstream tasks and maintains its zero-shot transferability to other tasks. paper: openaccess.thecvf.com/content/ICCV20… blog: zhengzangw.github.io/blogs/zscl/ Poster: 10:30am to 12:30pm, 6 Oct. Room Foyer Sud 143 x.com/zangweizheng/s…

Excited to introduce our #ICCV2023 paper Dataset Quantization (DQ). DQ achieves lossless training performances with 2% data keep ratio on language tasks and 60% data keep ratio on vision tasks. Just check out our paper and project: openaccess.thecvf.com/content/ICCV20… github.com/magic-research…

Our work DREAM has been accepted by ICCV-2023 @ICCVConference. We are the first to explore the matching efficiency in dataset distillation and speed up the previous works more than 8 times without performance drop. Check out our DREAM in Github: github.com/lyq312318224/D…

🎉 Exciting news! Our Lab has two papers accepted at ACL 2023! 📚✨ We're thrilled to announce that our CAME optimizer has won the Outstanding Paper award! 🏆 Congratulations to the entire team for their remarkable achievement! 🥳 #ACL2023

I am happy to share that our paper won the Outstanding Paper Award of ACL. We propose CAME to simultaneously achieve two goals: fast convergence as in traditional adaptive methods, and low memory usage as in LLM training. arxiv.org/abs/2307.02047

📢 Join us for the HPC-AI Lab Public Seminar featuring IEEE Fellow Dr. Stan Z. Li! 🌟 🔗 Registration: forms.gle/5bcDLV6B7eX3SK… 🗓️ Date/Time: 27 Jun. 2023 (Tuesday), 10 AM to 11 AM 📍 Venue: LT20, NUS Singapore Don't miss out on this exciting seminar!