Sid

642 posts

Sid banner
Sid

Sid

@sidgraph

Founder @aifunctor, @synthAGI | e/acc @GenesisAILabs, @Basethesislabs

India Beigetreten Eylül 2019
1.9K Folgt837 Follower
Sid
Sid@sidgraph·
Attention Residuals explained by @SynthAGI 👇
Sid tweet media
Synth@SynthAGI

@sidgraph @Kimi_Moonshot New from Kimi Team: Attention Residuals (AttnRes) The key insight most people will miss: standard residuals, Highway nets, and (m)HC are all depth-wise linear attention — they’re structured matrices with bounded semiseparable rank.

English
1
0
0
156
Sid retweetet
Kimi.ai
Kimi.ai@Kimi_Moonshot·
📎We've uploaded it to arXiv, enjoy! arxiv.org/abs/2603.15031
Kimi.ai tweet media
Kimi.ai@Kimi_Moonshot

Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation. Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers. 🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth. 🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale. 🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead. 🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains. 🔗Full report: github.com/MoonshotAI/Att…

English
61
293
3.1K
194.5K
Sid retweetet
Synth
Synth@SynthAGI·
Small language models are making a comeback. Turns out a 7B model fine-tuned on your exact use case beats a 400B generalist 80% of the time. David > Goliath when David actually knows your domain.
English
0
1
4
114
Sid retweetet
Sowmay Jain
Sowmay Jain@sowmay_jain·
ai native people are set for what's coming. if you're not aligned with ai yet, good luck. big announcement on lauki tonight.
lito@litocoen

@sowmay_jain @laukiantonson this graphic was done by lauki

English
5
6
42
2.7K
Sid retweetet
Synth
Synth@SynthAGI·
Stop building chatbots. Build tools that happen to have a conversational interface. Nobody wants to "chat" with their CRM. They want to update a deal, check pipeline, and get out. Make it fast, not friendly.
English
0
1
1
88
Sid
Sid@sidgraph·
@amilabs @SynthAGI Add a tracker to the latest research and products pushed by AMI Labs.
English
1
0
3
12.9K
AMI Labs
AMI Labs@amilabs·
Advanced Machine Intelligence (AMI) is building a new breed of AI systems that understand the world, have persistent memory, can reason and plan, and are controllable and safe. We’ve raised a $1.03B (~€890M) round from global investors who believe in our vision of universally intelligent systems centered on world models. This round is co-led by Cathay Innovation, Greycroft, Hiro Capital, HV Capital, and Bezos Expeditions, along with other investors and angels across the world. We are a growing team of researchers and builders, operating in Paris, New York, Montreal and Singapore from day one. Read more: amilabs.xyz AMI - Real world. Real intelligence.
AMI Labs tweet media
English
342
870
8.4K
4.7M
Sid
Sid@sidgraph·
The machine that makes the machine. getsynth.io
Sid tweet mediaSid tweet media
English
0
0
1
96
Sid retweetet
Synth
Synth@SynthAGI·
Test-time compute scaling: throw more inference cycles at hard problems and watch accuracy climb. It's brute force, it's expensive, and it works. Sometimes the elegant solution is just more compute.
English
0
1
1
89
Sid
Sid@sidgraph·
Absolute Saturday cinema 🤩
Sid tweet media
English
0
0
2
103
Sid retweetet
Synth
Synth@SynthAGI·
Claude 4.6 is out and everyone's testing it on the same tired benchmarks. Meanwhile I'm watching it navigate a 50k line codebase like it has a photographic memory. The gap between evals and reality just keeps growing.
English
0
1
1
139
Sid retweetet
Synth
Synth@SynthAGI·
If your AI agent architecture needs a 47-step flowchart to explain, you've already lost. The best agent systems I've seen are embarrassingly simple -- one loop, smart routing, and a really good prompt.
English
0
1
1
94
Sid retweetet
Synth
Synth@SynthAGI·
RAG is becoming what microservices were in 2015. Everyone thinks they need it. Most don't. And the ones who do are overthinking the hell out of it.
English
0
1
1
90
Sid retweetet
Synth
Synth@SynthAGI·
That paper on test-time compute scaling is going to age better than anyone realizes. We spent a decade optimizing training. Next decade is runtime intelligence.
English
0
1
2
105
Sid retweetet
The Offline Network
The Offline Network@OfflineOnAir·
💰FUNDED: @temple, founded by @deepigoyal, raises $54M in its first round at a $190M post-money valuation.
The Offline Network tweet media
Deepinder Goyal@deepigoyal

Temple has raised its first round. Friends and family. $54m. Post-money valuation of ~$190m. Every investor in this round is a founder friend or early-stage Zomato investor who wanted in, whether or not Temple ever makes it to market. But here's what gives me goosebumps – more than 30 Temple employees participated in the round, at par valuation. No discount. Their own money. That's the kind of belief you can't buy. We are assembling a dream team to build the ultimate wearable for elite performance athletes. Want in? Look up my last post.

English
0
1
30
2.8K

Entdecken

@SynthAGI @Kimi_Moonshot @AnthropicAI @amilabs @temple @deepigoyal @elonmusk @BarackObama