SID

92 posts

SID banner
SID

SID

@SID_AI

solving retrieval one model at a time | @ycombinator

San Francisco, CA Katılım Aralık 2022
0 Takip Edilen1.2K Takipçiler
Sabitlenmiş Tweet
SID
SID@SID_AI·
we just released our first model: SID-1 it's designed to be extremely good at only one task: retrieval. it has 1.8x better recall than embedding search alone (even with reranking) and beats "agentic" retrieval implemented using all frontier LLMs, including the really large and expensive ones (see chart). we trained SID-1 using multi-environment, multi-turn RL on Qwen. it was a lot of work (a lot of which is documented in our tech report -- see pinned tweet). our RL environments build on the idea that humans with search tools can find almost any information given sufficient iteration. like humans, SID-1 makes a first search, read the results, and adapts its strategy. and it can do this much faster *and* better than frontier LLMs: 24x faster than GPT-5.1, 27x faster than Gemini 3 Pro. the better part is critical! if a model is fast and wrong, it's just wrong. that's why we trained SID-1 until it was the most likely to deliver the correct results. bar none. we're partnering with a small number of companies today and have a waitlist for everyone else. (we don't have enough inference compute for everyone yet).
SID tweet mediaSID tweet mediaSID tweet mediaSID tweet media
English
18
41
392
153.9K
SID retweetledi
Max Rumpf
Max Rumpf@maxrumpf·
The Multi-Model Future is Here. That's why we're partnering with Baseten to bring agentic search to every dataset. People often misinterpret The Bitter Lesson, which is about using general methods, not about creating general models. One of Sutton's examples of a Bitter-Lesson model is AlphaGo, which is the opposite of general (it only plays Go) but was trained using general methods. That the most general method to pretrain a text transformer happened to create a very general model is less a law of nature than a property of the data: the biggest dataset (the internet) also happens to be the most general. But even the most general language models aren't very good at playing Go. There is value to general models, but this generality comes at the cost of outright performance. What you really want to maximize is leverage on computation: capability per FLOP, whether spent in training or at inference. A better training objective gives you more leverage. A way to use less human data gives you leverage (albeit indirectly). For Go, AlphaZero training has orders of magnitude more leverage than retrofitting a language model. At SID, we have found ways to get much more leverage for the task of search; incidentally, one of the two tasks Sutton believes scales arbitrarily (yes, he meant search as in MCTS, but we'll take it). We don't know the full extent of our leverage, but SID-1 gives a lower bound: it matches frontier models on retrieval accuracy at roughly 1/500th the inference compute, more than 2 OOMs. Unlike Go, search is very useful. We’re thrilled to partner with Baseten to bring this utility to everyone. Together we offer agentic search at 20x lower latency and 100x lower cost than a general frontier model, at equal or better accuracy. We expect many other tasks to go this way. The future will be multi-model not as a compromise to save money, but because specialized models are simply better: faster, cheaper, and more accurate at the thing you actually need.
Baseten@baseten

Today, we're announcing Baseten for Model Labs. We believe the AI landscape will be made up of a diverse ecosystem of specialized, closed-weight models running alongside open-weight ones. Baseten for Model Labs is the platform built to enable labs to monetize, distribute, and scale their closed models on Baseten infrastructure.

English
9
8
74
13.9K
SID
SID@SID_AI·
@baseten Great to be a partner!
English
1
0
6
660
SID retweetledi
Baseten
Baseten@baseten·
Today, we're announcing Baseten for Model Labs. We believe the AI landscape will be made up of a diverse ecosystem of specialized, closed-weight models running alongside open-weight ones. Baseten for Model Labs is the platform built to enable labs to monetize, distribute, and scale their closed models on Baseten infrastructure.
Baseten tweet media
English
11
25
215
40.4K
Sam Dauncey
Sam Dauncey@SJDauncey·
Tokenization has resisted the bitter lesson for 10 years. Bringing the tokenizer inside the model is the obvious path forward, but the key question is: how do we train this tokenizer?
Sam Dauncey tweet media
English
5
6
46
1.9K
SID
SID@SID_AI·
@maxrumpf Full details on our website!
English
1
0
1
55
Max Rumpf
Max Rumpf@maxrumpf·
specialized search subagents work extremely well outside of code, too! it's such an easy win: cost, latency, accuracy, context pollution. this model only had little RL (400 questions) and imo 4B is too small for many things. (maybe enough for code?) we've scaled this further at @SID_AI and it *really works*
Florian Brand@xeophon

good stuff from microsoft: 4B model just to explore code bases, cutting token costs by 10-50% (!!!) while the performance of the big model stays the same :)

English
1
0
7
1.3K
Max Rumpf
Max Rumpf@maxrumpf·
turbopuffer x SID An easy way to tell a good from a great AI researcher: how much do they think about infrastructure. Infra extends beyond what’s running on the GPUs: Slow environments will bottleneck your training steps. More parallel and powerful models make this problem worse. RL environment specifics are usually secret, but we shared some details in a recent post with our friends at @turbopuffer Training great models requires great infrastructure and we’re excited to be working with the best.
Max Rumpf tweet mediaMax Rumpf tweet mediaMax Rumpf tweet mediaMax Rumpf tweet media
turbopuffer@turbopuffer

SID-1 is an agentic search model by @SID_AI → 1.9x recall over RAG + rerank → 24x faster, 99% cheaper than GPT-5.1 trained using large-scale RL on turbopuffer at 1k+ QPS bursts over 10M+ document corpora across thousands of steps tpuf.link/sid-1

English
13
17
212
37.3K
SID retweetledi
turbopuffer
turbopuffer@turbopuffer·
SID-1 is an agentic search model by @SID_AI → 1.9x recall over RAG + rerank → 24x faster, 99% cheaper than GPT-5.1 trained using large-scale RL on turbopuffer at 1k+ QPS bursts over 10M+ document corpora across thousands of steps tpuf.link/sid-1
English
7
26
217
113.8K
SID
SID@SID_AI·
@maxrumpf all the best ones are.
English
0
0
2
49
SID
SID@SID_AI·
We now have the tools to solve search.
SID tweet media
English
2
0
5
458
SID retweetledi
SID
SID@SID_AI·
we just released our first model: SID-1 it's designed to be extremely good at only one task: retrieval. it has 1.8x better recall than embedding search alone (even with reranking) and beats "agentic" retrieval implemented using all frontier LLMs, including the really large and expensive ones (see chart). we trained SID-1 using multi-environment, multi-turn RL on Qwen. it was a lot of work (a lot of which is documented in our tech report -- see pinned tweet). our RL environments build on the idea that humans with search tools can find almost any information given sufficient iteration. like humans, SID-1 makes a first search, read the results, and adapts its strategy. and it can do this much faster *and* better than frontier LLMs: 24x faster than GPT-5.1, 27x faster than Gemini 3 Pro. the better part is critical! if a model is fast and wrong, it's just wrong. that's why we trained SID-1 until it was the most likely to deliver the correct results. bar none. we're partnering with a small number of companies today and have a waitlist for everyone else. (we don't have enough inference compute for everyone yet).
SID tweet mediaSID tweet mediaSID tweet mediaSID tweet media
English
18
41
392
153.9K
Max Rumpf
Max Rumpf@maxrumpf·
@trychroma We published our research in December and told Chroma's CEO Jeff. 4 months later, Chroma republished it without citing it. We think this sets a pretty bad precedent: x.com/maxrumpf/statu…
Max Rumpf@maxrumpf

Chroma's "new" model sure seems familiar. A story. Imitation is the sincerest form of flattery. But there is a point where it goes from "inspiration" to whatever Context-1 is: 6 months ago, Chroma's CEO @jeffreyhuber asked us about our research. 4 months ago, we proudly shared SID-1's tech report with him. An exchange I now understand very differently (see the emails). Today, they released a report heavily "inspired" by ours. Charts, datasets, methods, and the whole model itself. Down to the toggle for Figure 1 and our 4x RRF rollouts. They never reached out to benchmark our model. Their claims of "pareto-optimality" ring hollow. They provable knew there was another model. Unfortunately, we can't benchmark their model: While their weights are open, the harness they say one needs isn't yet. Their claims of "pareto-optimality" ring hollow. They knew there was another model. I know Jeff well and our offices neighbor. We shared a lot of insights in our tech report. Maybe more than prudent. But we believe in advancing human knowledge. (Making search better is our way of doing so). We applaud companies like @thinkymachines that are brave enough to share the ideas that make the work possible. But where do we go as a research community when we stop respecting each other's work? When we don't give credit where it's due? And trick "friends" into sharing more, just to steal it? While claiming moral high ground by calling this "open-source?" This completely destroys any incentive for us (and others) to go into as much depth as we did in our tech report. It’s sad to see the poor research practices that are sadly common in academia making their way into startups. Context-1 has some interesting ideas: Pruning is clever. I wish I were writing about them. Followers and copycats, even if they're bigger, don't scare us. I'm very proud of what we've built. And even more proud of who I'm building this with. We're also hiring original thinkers.

English
1
8
65
8.3K
Chroma
Chroma@trychroma·
Introducing Chroma Context-1, a 20B parameter search agent. > pushes the pareto frontier of agentic search > order of magnitude faster > order of magnitude cheaper > Apache 2.0, open-source
English
145
405
4.2K
1.1M
SID retweetledi
SID
SID@SID_AI·
SID-1: Tech Report It has way more detail than is prudent. sid.ai/research/sid-1…
English
1
3
14
3.2K
SID retweetledi
Max Rumpf
Max Rumpf@maxrumpf·
Chroma's "new" model sure seems familiar. A story. Imitation is the sincerest form of flattery. But there is a point where it goes from "inspiration" to whatever Context-1 is: 6 months ago, Chroma's CEO @jeffreyhuber asked us about our research. 4 months ago, we proudly shared SID-1's tech report with him. An exchange I now understand very differently (see the emails). Today, they released a report heavily "inspired" by ours. Charts, datasets, methods, and the whole model itself. Down to the toggle for Figure 1 and our 4x RRF rollouts. They never reached out to benchmark our model. Their claims of "pareto-optimality" ring hollow. They provable knew there was another model. Unfortunately, we can't benchmark their model: While their weights are open, the harness they say one needs isn't yet. Their claims of "pareto-optimality" ring hollow. They knew there was another model. I know Jeff well and our offices neighbor. We shared a lot of insights in our tech report. Maybe more than prudent. But we believe in advancing human knowledge. (Making search better is our way of doing so). We applaud companies like @thinkymachines that are brave enough to share the ideas that make the work possible. But where do we go as a research community when we stop respecting each other's work? When we don't give credit where it's due? And trick "friends" into sharing more, just to steal it? While claiming moral high ground by calling this "open-source?" This completely destroys any incentive for us (and others) to go into as much depth as we did in our tech report. It’s sad to see the poor research practices that are sadly common in academia making their way into startups. Context-1 has some interesting ideas: Pruning is clever. I wish I were writing about them. Followers and copycats, even if they're bigger, don't scare us. I'm very proud of what we've built. And even more proud of who I'm building this with. We're also hiring original thinkers.
Max Rumpf tweet mediaMax Rumpf tweet mediaMax Rumpf tweet mediaMax Rumpf tweet media
Chroma@trychroma

Introducing Chroma Context-1, a 20B parameter search agent. > pushes the pareto frontier of agentic search > order of magnitude faster > order of magnitude cheaper > Apache 2.0, open-source

English
27
34
514
72.5K
SID
SID@SID_AI·
@maxrumpf CUDA out of memory.
English
2
1
9
948
Max Rumpf
Max Rumpf@maxrumpf·
@SID_AI Infrastructure is research. Research is infrastructure.
English
1
0
8
460

Keşfet

@maxrumpf @baseten @SJDauncey @turbopuffer @trychroma @elonmusk @BarackObama @taylorswift13