Harmit Patel
2.1K posts
@harmitwt
Automation Logic Engineer @GM | prev. Engineer @Bell , Developer @shopify | CS student 🧑🎓 | Jack of all trades , trying to master of some ! |
Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation. Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers. 🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth. 🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale. 🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead. 🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains. 🔗Full report: github.com/MoonshotAI/Att…
Two Turkish brothers building drone what could go wrong @BaykarTech
Today’s Toronto evening hit different 🌆 That soft blue hour over the lake, CN Tower cutting through the clouds, and the whole skyline reflecting like glass. After grinding on hardware all day (and touching grass earlier), this view was the perfect reset #Toronto #CNTower
Money isn’t the root of evil,it’s the tool that buys freedom of exit Without capital,talent stagnates in friction:bad incentives,no compounding,trapped in broken loops With it? Builders fund experiments,create real value, and force systems to adapt #Capital #Builders #Freedom
Rich in the 80s:Suit,Ferrari,Mansion,Palm trees,Flash Rich in 2026:Modal window open,Prompt:“in Modal, change the button color to green” The game is changed Old wealth = signals of status New wealth = leverage over systems that actually ship #Reindustrialization #Compounding
Touching grassin a deep coding session ☀️ One minute feeling the real world,the next back to optimizing sleep algorithms & building tools that actually matter Balance isn’t optional when you ship hardware by day & code life-improving software by night #CodeAndCoffee
