Alexander Tian

Paper: arxiv.org/abs/2607.09052

Hardware yearns for block sparse attention, yet it seems largely absent from open weight LLMs. DeepSeek developed NSA, and people speculated DeepSeek v4 would integrate it, yet it was never utilized. We have a hypothesis as to why. We found that replacing dense attention with NSA significantly degraded its ability on synthetic retrieval tasks, even when finetuned on them. On our 32k context benchmark, it scored 0.300 compared to dense attention’s 0.904. We found the reason, and how to fix most of it. Introducing COBS🌽(Cumulant Order Block Sparse Attention), with @AdiGhai18 @sanjitneelam @ZVasania @tensorpro: • Raises NSA’s 0.300 → 0.820, closing ~86% of the gap to dense • 15.15x less KV-cache read traffic than dense (just 1.21x the NSA baseline) • Lower position-wise NLL than dense in our comparison The key insight: block selection is the keystone to block sparse attention, and existing methods are mathematically stuck in storing a first-order approximation of a cumulant generating function. COBS caches a compressed second cumulant and escapes this ceiling. Details in the paper.