His rate of experimentation is like no other! Check out @tokenbender
tokenbender@tokenbender
Ever wondered if you could extract capabilities and behaviors from neural networks and reuse/update/route it as needed? We introduce low-rank circuit conditioning, a novel approach that preserves the model's output behavior while reshaping how an existing capability is represented. In the base model, standard compact recovery stalls at 29%. After conditioning, the same extraction pipeline reaches 91.33% autoregressive full-answer recovery from 5.05% of MLP channels. The evidence points to a possibility of extracting and using isolated capabilities saving cost, latency and high adaptability. Read our work to understand more - tokenbender.com/posts/honey-i-…
English
