Omar Khattab
13.1K posts
Omar Khattab
@lateinteraction
Asst professor @MIT CSAIL @nlp_mit. Research includes https://t.co/VgyLxl0oa1, https://t.co/ZZaSzaRaZ7 (@DSPyOSS), RLMs, and GEPA. Prev: CS PhD @StanfordNLP. Research @Databricks.
Cambridge, MA เข้าร่วม Aralık 2022
3.4K กำลังติดตาม31.3K ผู้ติดตาม
yes yes everyone should start saying this more, and probably tagging my boss
Brendan (can/do)@BrendanFoody
it's under appreciated how excellent OpenAI models are at web search
English
tied with
x.com/mixedbreadai/s…
Mixedbread@mixedbreadai
Introducing Mixedbread Wholembed v3, our new SOTA retrieval model across all modalities and 100+ languages. Wholembed v3 brings best-in-class search to text, audio, images, PDFs, videos... You can now get the best retrieval performance on your data, no matter its format.
English
biggest AI news in 2 weeks ICYMI
Antoine Chaffin@antoine_chaffin
BrowseComp-Plus, perhaps the hardest popular deep research task, is now solved at nearly 90%... ... and all it took was a 150M model ✨ Thrilled to announce that Reason-ModernColBERT did it again and outperform all models (including models 54× bigger) on all metrics
English
@nileshgupta2797 @LightOnIO @mixedbreadai Yup I don’t believe late interaction should fundamentally revolve around MaxSim. It’s just a lot better than single vector.
It is possible to design richer pruning-friendly set-level / multi-vector scoring functions I think!
English
@lateinteraction @LightOnIO @mixedbreadai I agree logical compositions is def where single vectors fail hard - eucledian space by design doesn't allow modeling logical relevance. Though I've been wondering what are the limits of maxsim, is it limited to AND like queries or goes beyond (e.g. (A or B) and (not C))?
English
late interaction model (150M) beats the 54x larger Qwen3-8B-Embedding by... hmm, looks like up to 34% relative increase :D
also really funny that the entire top section of the BC+ leaderboard, sorted by Recall, is just late interaction models by @LightOnIO and @mixedbreadai
Antoine Chaffin@antoine_chaffin
BrowseComp-Plus, perhaps the hardest popular deep research task, is now solved at nearly 90%... ... and all it took was a 150M model ✨ Thrilled to announce that Reason-ModernColBERT did it again and outperform all models (including models 54× bigger) on all metrics
English
@nileshgupta2797 @LightOnIO @mixedbreadai The difference is huge on every distribution not yet overfit by the single vector models. This was true back in the day on all of BEIR, all of LoTTE, and all the OpenQA datasets, etc.
Single vector is uncompositional. It advanced only by moving distributions in domain.
English
@lateinteraction @LightOnIO @mixedbreadai Do you think such a big diff is primarily because of the logical AND like multi-constraint nature of queries in BrowseComp for e.g. find X that satisfies A and B and C and D ..
English
Omar Khattab รีทวีตแล้ว
Mixedbread just made 115h of videos accessible to my agent.
With the new @mixedbreadai v3 release, you can upload any video to your Mixedbread store and make its content accessible to your agent.
English
Omar Khattab รีทวีตแล้ว
Omar Khattab รีทวีตแล้ว
@pamelafox @dbreunig The link to the slides from the Dropbox Dash team is at the bottom of our eng blog post dropbox.tech/machine-learni…
English
Omar Khattab รีทวีตแล้ว
Late to BrowseComp-Plus...but good interactions 🙂
Antoine Chaffin@antoine_chaffin
BrowseComp-Plus, perhaps the hardest popular deep research task, is now solved at nearly 90%... ... and all it took was a 150M model ✨ Thrilled to announce that Reason-ModernColBERT did it again and outperform all models (including models 54× bigger) on all metrics
English
Omar Khattab รีทวีตแล้ว
this guy just never stops winning!
Antoine Chaffin@antoine_chaffin
BrowseComp-Plus, perhaps the hardest popular deep research task, is now solved at nearly 90%... ... and all it took was a 150M model ✨ Thrilled to announce that Reason-ModernColBERT did it again and outperform all models (including models 54× bigger) on all metrics
English
Omar Khattab รีทวีตแล้ว
Omar Khattab รีทวีตแล้ว
currently re-embedding my entire machine, thank you very much!
LateOn-Code-edge for code search and Reason-ModernColBERT for prose/docs search.
Antoine Chaffin@antoine_chaffin
BrowseComp-Plus, perhaps the hardest popular deep research task, is now solved at nearly 90%... ... and all it took was a 150M model ✨ Thrilled to announce that Reason-ModernColBERT did it again and outperform all models (including models 54× bigger) on all metrics
English
what do people mean by ex-founder?
how did you edit history to un-founder yourself
English
someone should try having RLMs, optimized with GEPA, write REPL code primarily using DSPy to do autoresearch for ColBERTv3
alex zhang@a1zhang
someone should try having RLMs write REPL code primarily using DSPy
English
Omar Khattab รีทวีตแล้ว
someone should try having RLMs write REPL code primarily using DSPy
English
Omar Khattab รีทวีตแล้ว
Omar is shy about this late interaction approach changing everything. As an outsider and seeing @LightOnIO 's crew delivering exceptional results day after day, I prefer to frame this type of result as being one of those rare AlexNet moments.
There is simply no turning back.
Omar Khattab@lateinteraction
imo, these kinds of regular amazing results don't quite mean that late interaction is extremely strong per se as much as they mean that dense single-vector retrievers are a permanent bottleneck on your quality and generalization they're so bad!
English
@HansiZeng @raphaelsrty @LightOnIO @mixedbreadai I definitely agree it’s not trivial to implement it right.
But why does one need to implement it? It’s already implemented and optimized. We don’t usually implement our own attention kernels or HNSW index or anything like that.
Rerankers are also far more expensive. Need GPU.
English
@lateinteraction @raphaelsrty @LightOnIO @mixedbreadai But what if dense retriever plus reranker? I like ColBert(v2), but it is not trivial to implement them right… dense retrieval + reranker are much easier to implement.
English
Omar Khattab รีทวีตแล้ว
Omar Khattab รีทวีตแล้ว
@dbreunig The DSpy meetup tonight was fantastic - even though I've never used DSpy, the case studies were full of general insights. Will speakers be sharing their slides?
English