Rohan Jha

@LightOnIO 's Colbert Zero by @antoine_chaffin @l_arnaboldi and @AmelieTabatta presented in Seoul by the one and only @l_arnaboldi 🚀

@LoganMarkewich @antoine_chaffin He's saying in the quoted BP that you can compress down to 32 tokens ~losslessly.

@antoine_chaffin 32 tokens a doc is pretty small no? I would say 512-1024 is more standard (i.e one page of a PDF)

I did some maths If you are using int8 storage for your dense embeddings, it's actually more expensive than a ColBERT model! If you combine the asymmetric binary quantization results from Mixedbread (x.com/mixedbreadai/s…) and these pooling results, for MS MARCO (N_docs = 8.8M): ColBERT (32 tokens/doc, d_proj= 128, binary doc-side): storage ≈ N_docs × (T × d_proj × 1 / 8) bytes ≈ 4.5 GB Dense (one vector/doc, hidden_dim = 768): FP32: N_docs × (hidden_dim × 32 / 8) bytes ≈ 27 GB BF16: N_docs × (hidden_dim × 16 / 8) bytes ≈ 14 GB int8: N_docs × (hidden_dim × 8 / 8) bytes ≈ 7 GB So you almost need twice as much storage, even in int8! Yes, obviously, you probably can do binary quantization for dense without loosing much either but: 1. Is it so standard everyone saying ColBERT is expensive is using it? 2. Even though you could probably make dense even more lightweight, you probably would end up with worst in domain results at a similar storage regime (look DenseOn vs LateOn results). And this isn't even about nDCG in domain. It's about the generalization, the long context capabilities, the agent capabilities, the multilingual capabilities...

I am actively hiring postdocs and PhD students to join me at George Mason. If you are @aclmeeting and want to chat, please fill out these forms.

Hierarchical pooling is very strong to reduce the footprint of late interaction without degrading results We recently showed that training for MUVERA/SMVE improved performance retention What if we trained for hierarchical pooling? We get even stronger (5×) lossless compression!

@RuiTheBaker @aaxsh18 @joeldierkes And he just found the nitrous button today

tell you are german without telling you are german @joeldierkes created a gauge-like control panel to benchmark the QPS of silo (our search engine), so he can feel like speeding on a highway while stress test it.

We've made big advancements in reducing representation size / doc, and making indices handle much greater volumes of documents. We might think about exchanging some of those savings for greater index augmentation to better serve harder queries.

@mrdrozdov Any update on the tech report?

@mrdrozdov Awesome! I really like your extending the search parallelism to include (info-need x aspect) too. Look forward to reading more about it.

New Product Update: We trained a retrieval-specialized model for Knowledge Assistant. It matches Claude Sonnet 4.5 retrieval quality at substantially lower latency. Introducing Instructed-Retriever-1.

@din0s_ "For backwards compatibility with the previous version from a prompt ago" wtf

i hate this so much. and the underlying cause is the same thing that makes it try to support previous schemas/designs even when you're actively iterating on a feature. i do NOT need you to support the thing you introduced in the last turn, i asked you to change it for a reason.

No particular reason, but today feels like a good day to remind everyone that mirroring papers on your website is ~free. 😊 jbarrow.ai/field_notes/fa…

The cool part is that k-meANNs is completely generic. It can be used to accelerate other heavy pipelines as well. (Is anyone thinking about late interaction/multivector? 🤔)

@AmelieTabatta @kirtangajjar_ I can send you the latex source later if it’d help

@Robro612 @kirtangajjar_ Remove the might, it's confirmed!! (Though I have been obsessing with remaking all figures with the same style so I fear I will have to waste time doing the same for this one 😂)

The figure:

@kirtangajjar_ Originally I created it for a poster at the Late Interaction Workshop @ ECIR: cs.jhu.edu/~rjha5/files/l… But it’s also now in my slides from a talk at Adobe: cs.jhu.edu/~rjha5/files/m… And @AmelieTabatta might use it in her lecture for the AI Powered Search series

@Robro612 Was it part of a video/blog? If yes, can we get a link to it please :)

I'll be at #ICML2026 next week! I'm presenting our poster on massively multilingual encoders and would love to learn about coffee in Seoul Please reach out to talk about data, model building and the amazing things that happen when you look at the data

@Robro612 told us there are 3 axis to compress ColBERT Did you listen though

I mean, @l_arnaboldi will be there discussing ColBERT-Zero... obviously it blew up 🐕

I’m a man of many hats

Claude loves clear binary dichotomies "Not ____, but _____" but as soon as *I* suggest a schema / binary structure to *it*, Claude goes all Jacques Derrida on my ass

LIMIT, but make it RecSys - very nice motivating example too!

@mrdrozdov Strongly agree! Thinking in terms of database concepts has made me way more sensitive to various realistic workloads / concerns

Have been enjoying reading more about databases these last few years since joining Databricks. Many CS PhDs working on AI would benefit a lot by spending more time on this topic.