Lola Le Breton

@ZainHasan6 @antoine_chaffin @bclavie @jeremyphoward @tomaarsen @apsarathchandar Thanks a lot for the interest! We evaluated all models with a standardized compute efficient finetuning and could not run a large grid search. LRs could probably be tuned for each of them. It'd definitely be interesting to see all models evaluated with more complex finetuning!

@antoine_chaffin @bclavie @jeremyphoward @tomaarsen Interesting discussion Maybe @apsarathchandar can shed some light here and comment on the observed underperformance of ModernBERT in their paper.

NeoBERT - Another successor to BERT! >> SotA performance on MTEB benchmark >> 250M parameters >> outperforms BERTlarge, RoBERTalarge, NomicBERT, and ModernBERT under identical fine-tuning conditions >> OSS - release all code, data, checkpoints, and training scripts

@zephyr277 @apsarathchandar @qfournier2 Thank you! NeoBERT is not integrated into the transformers library yet, so our code is hosted in the HF repo with the model (huggingface.co/chandar-lab/Ne…… needs to be run locally). But we would definitely be happy to work with a HF team to integrate it if people are interested!

@apsarathchandar @lo_LB_La @qfournier2 Congrats! Maybe a dumb question but why does it need trust remote code?

2025 BERT is NeoBERT! We have fully pre-trained a next-generation encoder for 2.1T tokens with the latest advances in data, training, and architecture. This is a heroic effort from my PhD student @lo_LB_La in collaboration with @qfournier2 and Mariam El Mezouar (1/n)

@IKEA_France ou l’incompétence à son maximum. Leur service client c’est un message copié-coller proposant de « chatter » (ce qui n’est en fait jamais disponible).

@IKEA_France Nous portons plainte de manière groupée contre Ikea. Il ne s’agit plus d’un problème de service client, mais d’un problème judiciaire. Associez-vous à nos témoignages.