Yichen (Zach) Wang @ICML

131 posts

Yichen (Zach) Wang @ICML banner
Yichen (Zach) Wang @ICML

Yichen (Zach) Wang @ICML

@YichenZW

ML Infra Research Intern @Bytedance (San Jose) & Ph.D. student on NLP/LLM @UChicagoCS |@UChicagoCI Prev. intern @UWNLP @BerkeleyNLP | BS @XJTU1896 24’

San Jose, CA Katılım Şubat 2023
368 Takip Edilen321 Takipçiler
Sabitlenmiş Tweet
Yichen (Zach) Wang @ICML
Yichen (Zach) Wang @ICML@YichenZW·
Lack of diversity in your LLM generation? (also noted by Artificial Hivemind, best paper @NeurIPSConf) Time to bring your base model back! An inference-time, token-level collaboration between a base and an aligned model can optimize and control diversity and quality!
Yichen (Zach) Wang @ICML tweet media
English
2
16
53
13.3K
Yichen (Zach) Wang @ICML retweetledi
Yike Wang
Yike Wang@yikewang_·
Automatic harness evolution appears to be a promising path toward AI self-improvement, but we find that its gains still largely come from repeated sampling and show limited generalization. Blog post: yikee.github.io/harnessevoluti… Code: github.com/rethinking-har…
Yike Wang tweet media
English
3
34
177
20.8K
Yichen (Zach) Wang @ICML retweetledi
Roberta Raileanu
Roberta Raileanu@robertarail·
Excited to give a talk at the @icmlconf RLxF Workshop today at 3:30pm. I’ll be talking about the role of open-ended esss in achieving superhuman scientific discovery. Thank you @shaohua0116 @shaneguML et al. for organizing this!
Shao-Hua Sun @ ICML 🇰🇷@shaohua0116

We're excited to welcome an outstanding lineup of speakers at the RLxF Workshop: Benjamin Eysenbach @ben_eysenbach, Chelsea Finn @chelseabfinn, Jesse Zhang @Jesse_Y_Zhang, Roberta Raileanu @robertarail, Jerry Tworek @MillionInt, and Brian Zhan @brianzhan.

English
0
7
62
5.6K
Yichen (Zach) Wang @ICML
Yichen (Zach) Wang @ICML@YichenZW·
[🛫ICML] We will be presenting our paper about *base-aligned model collaboration* on Tuesday morning and also Saturday (HAI Co-Creativity workshop). Happy to chat! PS: We've updated our paper, code, and reading list to include the latest results and resources. Check it out!
Yichen (Zach) Wang @ICML@YichenZW

Lack of diversity in your LLM generation? (also noted by Artificial Hivemind, best paper @NeurIPSConf) Time to bring your base model back! An inference-time, token-level collaboration between a base and an aligned model can optimize and control diversity and quality!

English
0
9
21
2.3K
Yichen (Zach) Wang @ICML retweetledi
Harvey Yiyun Fu
Harvey Yiyun Fu@harveyiyun·
Seeing👀 is not reasoning🤔? Introducing our new blog post. We tested 8 open VLMs on 9 VQA benchmarks, and found that a lot of "visual" accuracy isn't visual at all. Many benchmarks can be solved without ever looking at the image, and some VLMs reason better from text than from pixels. [1/n]
Harvey Yiyun Fu tweet media
English
2
29
113
26.6K
Yichen (Zach) Wang @ICML retweetledi
Chen Wu
Chen Wu@ChenHenryWu·
Self-improvement depends on whether a model can judge its own work. We usually train models to generate better - why not train them to verify just as well? We show how to train models to pinpoint their errors, and the same model nearly doubles its accuracy on hard math and jumps 14x on scientific reasoning. 🧵1/5
Chen Wu tweet media
English
15
62
510
57.6K
Yichen (Zach) Wang @ICML
Yichen (Zach) Wang @ICML@YichenZW·
Check out our new post! We compare interpretations from different views on the newly designed task: open-ended predictive verbalization, i.e., predicting what some pre-existing over-specified properties are under open-ended prompts.
Xiaoyan Bai@Elenal3ai

🗣️ Prediction, Explanation, or Over-interpretation? Recent work suggests LLMs can verbalize information about latent states and future generations. But training of different verbalization methods varies. Are they verbalizing, or are we over-interpreting from the explanation? 1/n

English
0
1
10
1.4K
Yichen (Zach) Wang @ICML retweetledi
Harvey Yiyun Fu
Harvey Yiyun Fu@harveyiyun·
Excited to share our new paper on KV Cache eviction! We propose a new recipe that is simple and effective: 1. keep those value states with large magnitude, 2. add some stochasticity to the eviction process. Combined, VaSE consistently outperform previous eviction methods with high throughputs, while maintaining constant memory footprints. Huge thanks to all collaborators @CharlotteTYC, @DeqingFu, @chrome1996, and advisors @_jessethomason_ and @robinomial
Deqing Fu@DeqingFu

Introducing VaSE: Value-Aware Stochastic KV Cache Eviction. Reasoning models think in CoT, bloating the KV cache. Eviction caps memory but suffers capability drop. VaSE is a training-free recipe that cuts that cost: keep large-magnitude value states, evict stochastically.

English
0
3
10
2.7K
Yichen (Zach) Wang @ICML retweetledi
Vinay Samuel
Vinay Samuel@vsamuel2003·
Post-training makes LLMs safer and better at following instructions, but less diverse. 🤔 Can we get that diversity back without sacrificing alignment? Introducing ReDiPO: a preference optimization recipe for restoring distributional diversity while preserving safety and instruction-following.
Vinay Samuel tweet media
English
1
12
33
11K
Yichen (Zach) Wang @ICML retweetledi
Mohit Iyyer
Mohit Iyyer@MohitIyyer·
Base models generate more diverse responses than their post-trained counterparts but are also much worse at instruction following. We recover diversity with a simple recipe: the instruct model rewrites messy base generations into high-quality responses, and we then run DPO with this synthetic data. The resulting model is more diverse without sacrificing instruction following! Details 👇
Vinay Samuel@vsamuel2003

Post-training makes LLMs safer and better at following instructions, but less diverse. 🤔 Can we get that diversity back without sacrificing alignment? Introducing ReDiPO: a preference optimization recipe for restoring distributional diversity while preserving safety and instruction-following.

English
1
10
32
4.5K
Yichen (Zach) Wang @ICML retweetledi
Weihua Du
Weihua Du@StigLidu·
Excited to introduce AdaExplore 🚀✨ AdaExplore teaches LLM agents to improve GPU kernel generation by learning from past execution failures (Adapt Stage) and searching over diverse optimization paths (Explore Stage). With GPT-5-mini as the base model, AdaExplore achieves 3.12×/1.72× speedups on KernelBench Level-2/Level-3 within 100 evaluation steps ⚡ and outperforms existing baselines such as OpenEvolve. Project Page & Demo: stiglidu.github.io/AdaExplore/ Arxiv: arxiv.org/abs/2604.16625 Code: github.com/StigLidu/AdaEx… More in the thread 👇
Weihua Du tweet media
English
4
28
93
48.8K
Yichen (Zach) Wang @ICML retweetledi
Lorenzo Xiao
Lorenzo Xiao@lrzneedresearch·
We have some concerns about the current state of LLM-based social simulation. We benchmarked 10 LLMs on persona simulation. Every model collapses. The "best" ones are the worst offenders. And RLHF actively makes it worse. arxiv.org/pdf/2604.24698
Lorenzo Xiao tweet media
English
4
16
83
9.3K
Yichen (Zach) Wang @ICML retweetledi
Dang Nguyen
Dang Nguyen@divingwithorcas·
1/n Corporate communication is a minefield, where outcomes can depend on every word in an email. LLMs are rapidly entering this world, but can they actually navigate human norms? Our research suggests they'll change how corporate emails will be written and read!
Dang Nguyen tweet media
English
1
15
27
2.4K
Yichen (Zach) Wang @ICML retweetledi
Roberta Raileanu
Roberta Raileanu@robertarail·
How can agents get better at algorithm discovery? Meta-meta-learning is one answer, aka improving the agents themselves at inventing generalizable algorithms. DiscoBench provides a way to procedurally generate algorithm discovery tasks at scale, which can be used for meta-meta-learning. Kudos to @AlexDGoldie and team for the release!
Alex Goldie @ ICML 2026 🇰🇷@AlexDGoldie

1/ 🪩 Automating the discovery of new algorithms could unlock significant breakthroughs in ML research. But optimising agents for this research has been limited by too few tasks to learn from! Introducing DiscoGen, a procedural generator of algorithm discovery tasks 🧵

English
1
16
88
13.8K
Yichen (Zach) Wang @ICML retweetledi
Tuhin Chakrabarty
Tuhin Chakrabarty@TuhinChakr·
🚨New paper on AI & Copyright 👨‍⚖️Courts have credited LLM companies' claims that safety alignment prevents reproduction of copyrighted expression. But what if fine-tuning on a simple writing task ruins it all? Worse : Fine-tuning on a single author's books (e.g., Murakami) unlocks verbatim recall of copyrighted books from 30+ unrelated authors, sometimes as high as 90%. Joint work with @niloofar_mire (@LTIatCMU), Jane Ginsburg ( @ColumbiaLaw) and my amazing PhD student @irisiris_l (@sbucompsc ) (1/n)🧵
Tuhin Chakrabarty tweet media
English
15
152
396
122.9K
Yichen (Zach) Wang @ICML retweetledi
Chenghao Yang
Chenghao Yang@chrome1996·
BranchingFactor v1.1 just dropped! 🚀 (Yes — it’s an actively updated paper.) (arxiv.org/abs/2506.17871) As models rely more on post-training, understanding the synergy between pre-training and alignment becomes crucial. Branching Factor (BF) offers a simple way to track the remaining generative potential of a model — since entropy inevitably decreases during generation, BF measures that process. What’s new in v1.1: 1️⃣ Major rewrite We now introduce BF directly — much clearer and easier to read. 2️⃣ Theorem correction + extension Thanks to @StarLi27496427 and Yuwei for catching my misunderstanding of the AEP theorem! We fixed the derivation and extended it to variable-length LLM outputs. The good news: the main result still holds — length-avg log-likelihood can estimate length-avg entropy for sufficiently long generations, in a memory-efficient way. Useful if you want to monitor entropy during training or inference. 3️⃣ Broader evaluation Added experiments on OLMo2 and Qwen3, plus multilingual and long-context tasks. Key findings so far still holds often: 📉 BF decreases during generation ✂️ Alignment significantly reduces BF ⚖️ Interestingly, OLMo2 appears less aggressively shrunk by alignment than Qwen3/Llama3 (preliminary observation). 4️⃣ SFT vs RL analysis We started dissecting how SFT and RL affect BF. Early signals from OLMo2: 🧠 Smaller models: BF shrink mostly happens during SFT (possible memorization effect). 🏗️ Larger models: SFT and RL have comparable impact. Still very preliminary — but it raises interesting questions about how post-training should scale with model size.
English
1
3
24
2K
Yichen (Zach) Wang @ICML retweetledi
Jixuan Chen
Jixuan Chen@chenjx210734·
🚀Excited to share that we bridge the connection of Clawbot & Simworld! 🧩We are motivated to move beyond isolated toy tasks and into a shared physical world with routines, interactions, and coordination. 🚧Lightweight setup: plug in your own agent easily!
SimWorld@simworld_ai

🤖Clawbots just moved into Embodied City inside SimWorld. They wake up. Go to work. Run errands. Talk to each other. All inside a shared physical world. This isn’t scripted — it’s autonomous agents living a daily routine. And you can spin up your own agent in minutes.

English
4
29
87
57K

Keşfet

@icmlconf @shaohua0116 @shaneguML @yikewang_ @WeijiaShi2 @LukeZettlemoyer @YejinChoinka @tsvetshop