Changho Shin
489 posts
@Changho_Shin_
Postdoc @ Princeton | prev @WisconsinCS @MSFTResearch @twitter @SeoulNatlUni
Nested models let you train a whole family of submodels at once. What if you could use them all at once, too? Block triangular weights enable this structure. It gives us token-adaptive routing, self-speculative decoding, and more. Introducing: Fully Nested Transformers (1/9)
Hello world 👋 Agents4Academia is a community‑led effort to explore and build open‑source agents for academic and research work — by researchers, for researchers. agents4academia.org
Weight-space geometry encodes traces of training data. Can we use it to reverse-engineer data recipes? Introducing WARP: a new strategy to estimate domain mixtures from model weights alone! WARP will present at the ICML WSS (Room 403) next Friday. arxiv.org/abs/2607.01686
That new LFM2.5-350M is super overtrained, right? And everyone was shocked about how far they pushed it? As it turns out, we have a brand new scaling law for that! 🧵 [1/n]
Excited to be presenting an oral and poster at #ICML2026 about hybrid models and their capabilities! There have been many empirical results, but far too few theoretical ones explaining their expressivity. We show there are tasks with a separation! Come chat! Oral 6A, poster 4622.
1/2 — CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation People often use multiple LLM judges (or agents) to reduce model-specific biases, hoping they cancel each other out. But what if those biases are shared because they're driven by common confounders? We propose CARE, a confounder-aware aggregation method that explicitly models these shared biases to produce more reliable evaluations. Still amused that probabilistic graphical models remain useful in the LLM era. 🙂 Curious to find more applications!
Efficient data curation is critical for modern ML. 📣 We introduce Mimic Score, a new, lightweight, model-based metric for sample utility that leverages reference model's weights to identify high-value samples and accelerate training. 🎉 Accepted as an Oral at ICML’25 DataWorld!
🚨New pre-print! Activation steering is highly parameter-efficient, but picking where to steer usually feels like trial-and-error. We established a first-order relationship between activation steering and fine-tuning, turning that "vibe check" into a principled framework.
