
Lukas
10.4K posts

Lukas
@lejooon
Mathematics. Modelling in finance/insurance. Investing. Minimizing over loss landscapes. Asking questions.







Börsen ger och tar, den gav väldigt mycket tid för repositionering den här gången. ♻️





Lite svårare med TACO denna gången.


Funderar på att smälla ihop en bok om finansbranschen, tankar om investeringar, lite blandade anekdoter, vad som händer bakom de blanka fasaderna och på handlargolven. Skall jag orka lägga tid på det?






Väldigt märkligt att Verve trots sin dominans inom mobil SSP inte får högre fart på tillväxten. (Äger inte.)



Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation. Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers. 🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth. 🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale. 🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead. 🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains. 🔗Full report: github.com/MoonshotAI/Att…


1 million context window: Now generally available for Claude Opus 4.6 and Claude Sonnet 4.6.







