Piotr Mardziel

@satchelgrant @ChrisGPotts @ARTartaglini Did you look into less state-destructive interventions like on attentions? Intervention there in theory does not change hidden state but instead how hidden states from one layer to the next are mixed.

1/8 New preprint! We show many interp methods (patching, SAEs, DAS) can push models off their natural manifold. This can be harmless or can activate hidden circuits. We provide a mitigating solution making interventions less divergent. If you care about reliable interp, read on!

@caltrain How does one miss their train despite arriving a full 6 minutes ahead of its scheduled time? By having an opposong train sit at the station keeping the pedestrian crossings closed the entire time.