Alex Yun

Nested models let you train a whole family of submodels at once. What if you could use them all at once, too? Block triangular weights enable this structure. It gives us token-adaptive routing, self-speculative decoding, and more. Introducing: Fully Nested Transformers (1/9)

🚨New pre-print! Activation steering is highly parameter-efficient, but picking where to steer usually feels like trial-and-error. We established a first-order relationship between activation steering and fine-tuning, turning that "vibe check" into a principled framework.

🚨New pre-print! Activation steering is highly parameter-efficient, but picking where to steer usually feels like trial-and-error. We established a first-order relationship between activation steering and fine-tuning, turning that "vibe check" into a principled framework.