Cambridge ML Systems Lab

Do not miss a series of deep technical contributions from the @Cambridge_Uni ML Systems lab (@CaMLSys) @iclr_conf this week in Rio. 4 conference papers, 2 workshop papers, 1 talk. I'll be there from Friday through Sunday. I'm also looking to hire current and future superstars for our frontier AI team @flwrlabs (careers.flower.ai) and a new postdoc at CaMLSys. DM me to meet up. [paper] MT-DAO: Multi-Timescale Distributed Adaptive Optimizers - Pavilion 3 Poster Location: P3-#405 - Sat, Apr 25 2026; 4:15 PM – 6:45 PM - openreview.net/forum?id=5yPP2… [paper] DES-LOC: Desynced Low Communication Adaptive Optimizers for Foundation Models - Pavilion 3 Poster Location: P3-#406 - Sat, Apr 25 2026; 4:15 PM – 6:45 PM - openreview.net/forum?id=6N2qF… [paper] Cascadia: An Efficient Cascade Serving System for Large Language Models - Pavilion 3 Poster Location: P3-#1625 - Sat, Apr 25 2026; 4:15 PM – 6:45 PM - openreview.net/forum?id=wkrWb… [paper] Rethinking Data Curation in LLM Training: Online Reweighting Offers Better Generalization than Offline Methods - Pavilion 3 Poster Location: P3-#519 - Thu, Apr 23 2026; 11:30 AM – 2:00 PM - (also appearing at ICLR 2026 Workshop on Navigating and Addressing Data Problems for Foundation Models Sun, Apr 26, 2026) [workshop] Homophily as a Lossy Channel: Decomposing Information in Graphs and Graph Neural Networks - Workshop on Scientific Methods for Understanding Deep Learning - Sun, April 26 2026 in Room 101-B - openreview.net/forum?id=wFBRZ… [talk] I don't know the title yet - Sun, April 26 2026 10.15am at Laguna Barre Hotel - informal workshop on Protocol Learning: Decentralized Collaborative Learning at Scale by @Pluralis @Alex__Iacob, @itsmaddox_j, @mher_safaryan, @meghdadkurmanji, @lorenzosani97, @sam_hrvth, @williamfshen, @xinchiqiu, @Renee42581826, @yihong_thu, Yuzhi Tang, Wentao Ma, Shengchao Hu, @shelling343, Abhinav Mehrotra, @parisgiampouras, @P_Aleks24, Youhe Jiang, Fangcheng Fu, @steverab, Jintao Zhang, Binhang Yuan, Vivek Kothari, Nicholas D. Lane University of Cambridge + @Cambridge_CL

I’ve launched a personal website where I’ll write about my work as a researcher at @CaMLSys, focusing on the intersection of machine learning, optimization, and scalable systems. iacob-alexandru-andrei.github.io

New Preprint: LoRDO 🚨 How can we design high-performance low-rank optimizers for communication-efficient training? We introduce LoRDO: Distributed Low-Rank Optimization with Infrequent Communication 🚀

Great to be representing @flwrlabs @italiantechweek in Turin. So many discussion about federated and privacy in AI. So many Flower users. Change is her. Ping me to meet.

idk where the complaints about gpt5 come from, it's pretty good

Our next talk in the @CaMLSys Seminar Series is this Monday. Join us at 11am Aug 4th to hear from Salma Kharrat -- @KAUST, who will give a talk on ML under various forms of constraints -- including examples such as federatations and LLMs. "Learning Under Constraints: From Federated Collaboration to Black-Box LLMs" -- talks.cam.ac.uk/talk/index/233… Salma Kharrat (KAUST) Monday 04 August 2025, 11:00-12:00 Computer Lab, FW26. Abstract: In both federated learning (FL) and large language model (LLMs) optimization, a central challenge is effective learning under constraints, ranging from data heterogeneity and personalization to limited communication and black-box access. In this talk, I present three approaches that address these challenges across different settings. FilFL improves generalization in FL by filtering clients based on their joint contribution to global performance. DPFL tackles decentralized personalization by learning asymmetric collaboration graphs under strict resource budgets. Moving beyond FL, I will present ACING , a reinforcement learning method for optimizing instructions in black-box LLMs under strict query budgets, where weights and gradients are inaccessible. While these works tackle distinct problems, they are unified by a common goal: developing efficient learning mechanisms that perform reliably under real-world constraints.

Catch this podcast with @niclane7 of CaMLSys, especially watch out for the discussion of large-scale decentralized training of LLMs pioneered in our lab with the Photon system -- jointly developed with @flwrlabs. buzzsprout.com/2417792/episod…

Resource-Efficient Knowledge Editing for Mobile LLMs -- Zhenyan Lu, Dongqi Cai, Chen Peng, Zexi Li, Shanggua Wang, @niclane7 and Mengwei Xu (Beijing University of Posts and Telecommunications, @Cambridge_Uni)

The work, "Resource-Efficient Knowledge Editing for Mobile LLMs," tackles critical challenges in enabling adaptable and highly efficient large language models on mobile and edge devices. This paves the way for more personalized, sustainable, and energy-efficient AI applications.

🎉 Huge congratulations to the teams at CaMLSys and BUPT -- particularly Zhenyan Lu, Dongqi Cai, Zexi Li and all the collaborators -- for winning the Best Poster Award at MobiUK 2025 in Edinburgh!🏆 mobiuk.org/2025/abstract/…

This week, we are delighted to host Max Ryabinin (@m_ryabinin) of @togethercompute who will be speaking at the Cambridge ML Systems Seminar Series. His talk is at 3pm June 18th, LT1 @Cambridge_CL

A huge thank you @SPRIND for hosting the Composite Challenge mentor days recently. Looking forward big decentralized training runs in the coming months, and helping to expand participation in AI more broadly.

Tomorrow join us for the latest edition of the Cambridge ML Systems Seminar Series. We are delighted to be joined by @Rosco_Hunter_ who will present "Building Oranizational Resilience with AI Malfunction Drills". See you tomorrow, May 20th at 3.30pm at FW26 in the Computer Lab.

Flower Labs Social @MLSysConf 2025 Join us for food, drinks, games, swag, and a quick look at our new open-source tools — including Photon, our SOTA system for decentralized foundation model pretraining. 🍕🎮🌸 📍 5 mins drive from MLSys venue 🕢 Starts 7:30 PM | All are welcome! 🔗 Want to join? Just scan the QR code or follow the link in the thread to register.

Photon: A New SOTA for Decentralized LLM pre-training at @MLSysConf 2025. Poster today; talk on Thursday. @lorenzosani97 converting one person at a time to the merits of federated everywhere for everything. Paper: arxiv.org/abs/2411.02908

Thanks to everyone who joined online and in person for the fantastic talk by @Jamie_Shotton from @wayve_ai!

Join us for the next Cambridge ML Systems Seminar: @Jamie_Shotton, Chief Scientist at @wayve_ai who give his talk: "Frontiers in Embodied AI for Autonomous Driving" on Friday May 2nd at the @Cambridge_CL in LT1. See you all then!

Lots of interest in SparsyFed @iclr_conf. Big step forward in practical use of sparsity for federated training. High sparsity w/out heavy hyperparameter tuning arxiv.org/abs/2504.05153 Fantastic team: @AdrianoGuastel1, @lorenzosani97, @Alex__Iacob, Alessio Mora, Paolo Bellavista

🚀 DEPT: A New Breakthrough in LLM Embeddings for Decentralized AI We're thrilled to share that Decoupled Embeddings for Pre-training (DEPT) will be presented as an ICLR 2025 Oral in Singapore on Saturday, April 26! 💪🏼 DEPT introduces a new approach for embeddings used in LLM pre-training by decoupling token embeddings from the transformer body. We've used it extensively in decentralized training runs -- but DEPT also has more broadly changes the way embeddings should be used in conventional centralized training as well. DEPT enables: ⭐️ Effective training across highly diverse domains and languages ⭐️ Up to 714x reduction in communication costs ⭐️ 80% smaller embedding matrices ⭐️ Robust, vocabulary-agnostic federated training This has been a joint work with the @Cambridge_CL (specifically @CaMLSys lab) and Flower Labs. Congrats to all of the DEPT co-authors for pushing the boundaries of scalable multi-domain and language LLM training. 🔗 Read the full story in the link in the comments below. DEPT Team: @Alex__Iacob, @lorenzosani97, @meghdadkurmanji, @williamfshen, @xinchiqiu, @DongqiCai, @yangao381, @niclane7

Join us for the next Cambridge ML Systems Seminar: Alexander Hägele (@EPFL) on "Learning Rate Schedules, Scaling Laws, and Techniques for Pretraining LLMs" 📍 @Cambridge_CL, FW26 🕒 Tue 08 April, 14:00–15:00 Featuring new results from his recent NeurIPS spotlight.