Zichun Yu

RT @ruihanglai: Two moments every ML researcher knows. You get onto a new cluster, and week one goes to fitting the framework to your setup…

Agentic test-time scaling (TTS) is effective -- until you find its inherent limits. 💡We show that classic TTS methods offered limited practical gains due to two fundamental limitations: the context ceiling and the verification gap. 🧵 Check the website: general-agentbench.github.io

We're excited to announce the launch of MMU-RAG at #NeurIPS2025! A new competition from the LTI & Amazon AGI focused on real-world Retrieval-Augmented Generation (RAG) with live user feedback via RAG-Arena. Read more: lti.cmu.edu/news-and-event…

@liweiche77 @LTIatCMU @Apple Nice work! 🎉

Thrilled to share our #ICML2025 paper! We introduce a variational approach for speech language models, automating speech attribute learning to deliver more natural, human-like speech. Joint work b/w @LTIatCMU and @Apple Read it: arxiv.org/abs/2506.14767

Come and try out our open-source MoE research platform🤩

Check out our recent work on influential synthetic data generation by adapting teacher to student learning 🧑‍🏫

@ChaoyiPan Thanks Chaoyi 🫡

@Zichun_Yu congrats, zichun!

🧑‍🤝‍🧑 Introducing MATES: Model-Aware Data Selection for Efficient Pretraining with Data Influence Models 🚀 MATES significantly elevates the scaling curve by selecting the data based on the model's evolving needs. Paper: arxiv.org/pdf/2406.06046 Code: github.com/cxcscmu/MATES 🧵[1/n]

@MrCatid Our real validation set is actually the target task, which is represented as the reference task (D_r) in the formula.

@MrCatid Yes, the hold-out set can be regarded as a validation set, but not necessarily. The only constraint here is that it is sampled from the same distribution as the training data.

@MrCatid The approximation difference from RHO and MATES

@MrCatid 2. MATES selects the data only based on the current model state (no reference model needed). The data influence model is used to parameterize the data influence calculation for better efficiency.

🎉 Thanks to all who contributed to this work! @Zichun_Yu Spandan Das @XiongChenyan ❤️ We welcome feedback/contributions! 🧵[n/n]

Finally, limitations: * Assuming each data contributes independently is practical now, but it would be interesting to obtain and learn the combinational effect of data. * As exploratory work, we run the model size up to 1B. The next step is to scale up our method. 🧵[5/n]

@PandaAshwinee Oh, thanks for your suggestions, Ashwinee. I am a newbie to Twitter, lol.

@Zichun_Yu @Zichun_Yu very cool work! I recommend making all the posts be “replies” to each other in a thread so that people can see the order. Otherwise twitter will sort the replies however it sees fit and it can be hard to follow the ordering.