Anya Belz retweetledi

a preprint with @anyabelz : Does Explicitly Exercising the Induction Circuit Improve ICL Performance?
We investigate whether explicitly exercising induction circuit during pretraining improves in-context learning (ICL), or whether natural text alone is sufficient when compute is matched (iso-FLOPs).
English





