Sparsity in LLMs Workshop at ICLR 2025

We are happy to announce that the Workshop on Sparsity in LLMs will take place @iclr_conf in Singapore! For details: sparsellm.org Organizers: @TianlongChen4, @utkuevci, @yanii, @BerivanISIK, @Shiwei_Liu66, @adnan_ahmad1306, @alexinowak

@AggieInCA giving the first Oral talk at #ICLR2025 @sparseLLMs workshop

Presenting in short! 👉🏼 Mol-MoE: leveraging model merging and RLHF for test-time steering of molecular properties. 📆 today, 11:15am to 12:15pm 📍 Poster session #1, GEM Bio Workshop @gembioworkshop @sparseLLMs #ICLR #ICLR2025

@black_samorez giving his oral talk at the #ICLR2025 @sparseLLMs workshop

We are presenting “Prefix and output length-aware scheduling for efficient online LLM inference” at the ICLR 2025 (@iclr_conf) Sparsity in LLMs workshop (@sparseLLMs). 🪫 Challenge: LLM inference in data centers benefits from data parallelism. How can we exploit patterns in requests – like shared prefixes and variable decode length – to optimally assign requests to GPU workers? 💡 Idea: both prefix and output length-aware scheduling! We build on Preble (ICML 2025, @vikranth22446, @yiying__zhang), which was the first distributed LLM serving system to exploit prompt sharing (see mlsys.wuklab.io/posts/preble). In our proof-of-concept work, we carefully benchmark Preble vs. prefix-unaware schedulers to identify opportunities for performance improvement. ⭐️ By adding output length-aware scheduling to Preble, we reduce latency by 14.31% at 64 RPS and 28.89% at 128 RPS. ⭐️ 📖 Full paper here: openreview.net/forum?id=DOZiC… Thank you to co-authors @InakiArango, @YepHuang, @rana_shahout, and @minlanyu at @hseas. Thank you also to the Preble authors for their groundbreaking work!

@tydsh giving his invited talk at the #ICLR2025 @sparseLLMs workshop!

our workshop on sparsity in LLMs is starting soon in Hall 4.7! we’re starting strong with an invited talk from @DAlistarh and an exciting oral on scaling laws for MoEs!

@DAlistarh giving his invited talk at the #ICLR2025 @sparseLLMs workshop

Our ICLR 2025 Workshop on Sparsity in LLMs (@sparseLLMs) kicks off with a talk by @DAlistarh on lossless (~1% perf drop) LLM compression using quantization across various benchmarks.

First poster session at the #ICLR2025 @sparseLLMs workshop

a PACKED hall for @tydsh‘s talk at our sparsity in LLMs workshop -not surprising! we have another oral right after this, and then we’ll have the first of 2 poster sessions before lunch! @iclr_conf

@DAlistarh giving his invited talk at the #ICLR2025 @sparseLLMs workshop now!

#ICLR2025 @sparseLLMs Workshop Panel with @PavloMolchanov @ayazdanb @DAlistarh @realDanFu @YangYou1991 and Olivia Hsu, moderated by @PandaAshwinee

Sparse LLM workshop will run on Sunday with two poster sessions, a mentoring session, 4 spotlight talks, 4 invited talks and a panel session. We'll host an amazing lineup of researchers: @DAlistarh @vithursant19 @tydsh @ayazdanb @gkdziugaite Olivia Hsu @PavloMolchanov Yang Yu

I will travelling to Singapore 🇸🇬 this week for the ICLR 2025 Workshop on Sparsity in LLMs (SLLM) that I'm co-organizing! We have an exciting lineup of invited speakers and panelists including @DAlistarh, @gkdziugaite, @PavloMolchanov, @vithursant19, @tydsh and @ayazdanb.

Check out this post that has information about research from Apple that will be presented at ICLR 2025 in 🇸🇬 this week. I will be at ICLR and will be presenting some of our work (led by @samira_abnar) at SLLM @sparseLLMs workshop. Happy to chat about JEPAs as well!

🔹 Mentor sign-up: fill out docs.google.com/forms/d/e/1FAI… (Extended deadline: April 11th, 2025) 🔹Mentee sign-up: fill out docs.google.com/forms/d/e/1FAI… (Deadline: April 11th, 2025)

This is a unique chance for young researchers to connect with senior mentors for guidance on research challenges, collaborations, publishing, and more. 📅 Dedicated mentorship session during the workshop + networking opportunities over coffee & lunch breaks!

🔊Exciting Mentorship Opportunity @iclr_conf! We’re thrilled to announce that mentee applications are now open for our mentorship program at the Sparse-LLM workshop at @iclr_conf! Details 👇

Our QuEST paper was selected for Oral Presentation at ICLR @sparseLLMs workshop! QuEST is the first algorithm with Pareto-optimal LLM training for 4bit weights/activations, and can even train accurate 1-bit LLMs. Paper: arxiv.org/abs/2502.05003 Code: github.com/IST-DASLab/QuE…

📅 Dedicated mentorship session during the workshop + networking during coffee & lunch breaks! Attending in person? Join us! 🔹Mentor sign-up: fill out this form by March 14th, 23:59 AoE: docs.google.com/forms/d/e/1FAI… 🔹Mentee sign-up: TBA — stay tuned! 👀

🚨 Exciting Mentorship Opportunity at ICLR '25 🚨 We’re organizing a mentorship program during our Sparse-LLM workshop @iclr_conf to connect young researchers with senior mentors! This is a great chance to seek guidance on research challenges, collaborations, publishing and more.