Stéphane Clinchant

New sparse retrieval model: introducing SPLARE, which extends SPLADE by replacing the vocabulary head with pretrained SAEs! paper: arxiv.org/abs/2603.13277 (ICLR'26) also how we won the WSDM'26 Cup on multilingual retrieval: arxiv.org/abs/2602.20986 (model weights coming soon!)

𝘏𝘶𝘮𝘢𝘯𝘴 𝘵𝘩𝘪𝘯𝘬 𝘧𝘭𝘶𝘪𝘥𝘭𝘺—𝘯𝘢𝘷𝘪𝘨𝘢𝘵𝘪𝘯𝘨 𝘢𝘣𝘴𝘵𝘳𝘢𝘤𝘵 𝘤𝘰𝘯𝘤𝘦𝘱𝘵𝘴 𝘦𝘧𝘧𝘰𝘳𝘵𝘭𝘦𝘴𝘴𝘭𝘺, 𝘧𝘳𝘦𝘦 𝘧𝘳𝘰𝘮 𝘳𝘪𝘨𝘪𝘥 𝘭𝘪𝘯𝘨𝘶𝘪𝘴𝘵𝘪𝘤 𝘣𝘰𝘶𝘯𝘥𝘢𝘳𝘪𝘦𝘴. But current reasoning models remain constrained by discrete tokens, limiting their full potential. Introducing 𝐒𝐨𝐟𝐭 𝐓𝐡𝐢𝐧𝐤𝐢𝐧𝐠: a training-free method that mimics human-like “soft” reasoning by generating continuous, abstract concept tokens. These tokens smoothly blend multiple meanings through probability-weighted mixtures of embeddings, enabling richer representations and seamless exploration of diverse reasoning paths. 𝐓𝐡𝐞 𝐢𝐦𝐩𝐚𝐜𝐭? ✅ Improved accuracy on math & code benchmarks by up to 2.48% (pass@1). ✅ Reduced token usage by up to 22.4%, making reasoning models both smarter and more efficient.

Arrived in Singapore for #ICLR2025 and will be presenting PROVENCE on Friday, Poster session 3 at 10am, poster #255! Blogpost: huggingface.co/blog/nadiinchi… Will be happy to meet & chat about #LLMs, #RAG, #InformationRetrieval and #MultilingualNLP :) #NLProc @naverlabseurope

AllenAI COOKED, Llama 3.1 Tulu 405B beats DeepSeek V3 - all whilst being 40% SMALLER! 🔥 Fully open model weights, data and training pipeline 🤗

We welcome contributors to add datasets, metrics, and other tasks to BERGEN. Join us! 🤝 #RAG #LLMs

Our recommendations are detailed in our first Arxiv paper, with additional findings on multilingual RAG in our second paper. arxiv.org/abs/2407.01102 arxiv.org/abs/2407.01463

What’s a good baseline for RAG? 🤔 The literature shows consistent differences in experimental setups, retrievers, datasets, and metrics. So, we built the BERGEN library github.com/naver/bergen to enhance reproducibility and identify strong baselines : 🧵 @naverlabseurope

😀We're looking for a talented researcher to join our team at Naver Labs Europe (@naverlabseurope) , working on LLMs and Retrieval!😃 Please apply here: europe.naverlabs.com/job/research-s… !

@jerryjliu0 @rpradeep42 If efficiency matters, a simpler solution is to actually use a state of the art reranker (cf our study comparing LLMs and cross-encoders) arxiv.org/pdf/2403.10407…

RankZephyr is a nice 7B model by @rpradeep42 et al. that is optimized for list-wise zero-shot reranking. Since it is much smaller than proprietary models, it is a big step towards practically using LLMs as part of production retrieval systems (as opposed to simply combining LLM with a retrieval system in a typical RAG setup). Excited to present an integration with @llama_index through the RankLLM library, which contains a collection of open-source models specifically for reranking. Check out our full @llama_index RankLLM guide: docs.llamaindex.ai/en/stable/exam… RankZephyr paper: arxiv.org/pdf/2312.02724…

... especially when reviewers said ‘dense retrieval on its own has shown to surpass sparse retrieval considerably ‘ and that our ‘approach is quite incremental’ 2/2

It feels good when someone from a big company shares that they saw ‘ pretty promising results in terms of quality and space savings [for SPLADE] compared to dense embedding models’ ... 1/2

@andysingal @thibault_formal @naverlabseurope github.com/naver/dcmm

@thibault_formal @sclincha @naverlabseurope Old paper, but is there any open source code for it... Ranking for images will be a good use case

What a great pleasure and honor to share this session about generative AI, ethics, bias, and politics with 3 passionate speakers @plimantour, Andrew Wyckoff, and Juha Heikkilä. Thanks @AI2S2Symposium for the invitation. See you in Geneva on Monday!

I will present our #ACL paper “Should you marginalize over possible tokenizations?” on Wednesday, Jul 12, 11:00-12:30, at Poster session 7! Come to chat about tokenization in LMs. w/ @germank @josrzn @MarcDymetman. arxiv.org/abs/2306.17757

Outstanding!

Excited to share our #ACL2023 paper "Memory-efficient NLLB-200: Language-specific Expert Pruning of a Massively Multilingual Machine Translation Model". arxiv:arxiv.org/abs/2212.09811 Joint work w/ Alexandre Berard, Vassilina Nikoulina during my internship @naverlabseurope 1/6

📢 REMINDER for TREC-AToMiC participants! News: Test topics are out! 🎉 Check them here: trec-atomic.github.io/annoucements/t…. We've carefully selected 200 sections from vital Wikipedia articles. Get ready for some fascinating exploration! Happy searching! 🚀

@srush_nlp On the question whether we need attention, I would add : Synthesizer: Rethinking Self-Attention in Transformer Models arxiv.org/abs/2005.00743 which shows that random attention works well too

Would love critiques/missed cites. Had to drop a couple of sections on diagonal S4 just because it is hard to explain. However it is critical in the development of these models.

Do we need Attention? (v0 github.com/srush/do-we-ne…): Slides for a survey talk summarizing recent Linear RNN models with a focus on NLP. Tries to cover a lot of different S4-related models (as well as RWKV/MEGA) in a digestible way.

Before signing off for the weekend sign up for next week's 📢 Open virtual seminar w @alireza_mshi Ph.D.@EPFL and @idiap_ch-@uzh_en ! Reference-Free Metric for Evaluating Question Generation by Answering the Question 📅Tue 16th May 9.30am CEST Register: tinyurl.com/uxzvaddr