Liger Kernel
24 posts
@liger_kernel
Efficient Triton Kernels for LLM Training
🥳🥳 Introducing "CoT-ICL Lab" to #ACL2025 A highly flexible (synthetic) data generation framework to study Chain of Thought learning with "In-Context" examples. TL;DR: This work bridges the gap between theoretical setups and complex NLP settings to study how transformers learn abstract concepts and their underlying causal dependencies. 🧵 Joint work with: @mamhamed, Maziar Sanjabi
We are open sourcing bytecheckpoint and veomni! bytecheckpoint is the Bytedance's production checkpointing system for foundation model training, battle-tested with jobs with 10k+ GPUs. Blazing fast save/load, load-time checkpoint auto-resharding for different parallelism across training stages (pretrain/SFT/RL). veomni is a open source model training framework for llm and multi-modal training. UI-TARS (the SOTA GUI Agent model prior to OpenAI operator's release) is trained with veomni. Developed with modular design, integrated with sequence/expert/zero-optimizer parallelism, offloading optimizations, @liger_kernel. Trainer-free (let user control the training loop) and easy for researcher to hack! The go-to framework for text/multimodal llm pre-training and post-training, from research to production. Try them today, your feedback is welcome! Code: - github.com/ByteDance-Seed… - github.com/ByteDance-Seed… Paper: - NSDI paper: arxiv.org/abs/2407.20143
Join our online meetup on Oct. 16 for efficient LLM deployment and serving, co-hosted by SGLang, FlashInfer, and MLC LLM! 🥳 You are all welcome to join by filling out the Google form forms.gle/B3YeedLxmrrhL1… It will cover topics such as low CPU overhead scheduling, DeepSeek MLA optimizations, kernel generation for high-performance LLM serving, universal LLM deployment, Low-latency serving, and fast grammar-based decoding.
