Uri Berger

Ever used a top-ranked LLM that just... felt wrong for you? You’re not alone. Instead of leaderboards, many of us turn to "vibe-testing" - manually comparing models to our own needs. But can we turn these feelings into a structured evaluation? New paper: "From Feelings to Metrics" 🧵

What if you could automatically turn a large collection of documents into structured databases, tailored for your own research needs?📄 We introduce ScheMatiQ! From question ➝ schema ➝ structured data 🔍 @ShaharLevy19 @MintzReshef @RKeydar @BarakRaveh @GabiStanovsky 🧵 👇1/5

Today at #EACL2026 we present our work exploring PRINTED newspapers as a data source for summarization in low-resource languages. Join the virtual session at 18:00! 🧵👇

4. A spicy take 🌶️ In current multimodal models, the text component does most of the work. Vision components are comparatively weak- meaning there's still a lot of room for real progress there.

3. On multimodality: mixed opinions on whether "classic" image+text is saturated. But near-universal agreement that video and 3D remain wide open and underexplored.

4 takeaways from conversations with some very sharp minds during my recent visit to Cambridge & University of Edinburgh 🇬🇧 A short thread👇 1. Strong consensus: classic LLM tasks in standard setups (QA, summarization, etc.) are seen as sovled and are not interesting anymore.

New preprint 💥 Can a general-purpose model achieve results comparable to medically pre-trained models? 🤔 We show that lightweight fine-tuning of a general-purpose LVLM and an LVLM-aware retriever can. 🚀 🔗 GitHub: github.com/Nirmaz/CLARE 📄 Paper: arxiv.org/pdf/2508.17394

I'll be visiting the UK at the end of January (26–29 London, 29–30 Edinburgh) and giving a talk in @frank_e_keller 's group on the 30th. If you’re into multimodality, interactive learning, or language games and will be around- ping me!

Great time @iscol_meeting! We presented twp works: • arxiv.org/abs/2409.16646: Do different languages mention different objects in the same image? (w/ @PontiEdoardo ) • arxiv.org/abs/2506.05309: Can models decide when to speak and play Mafia? (w/ @niveckhaus @GabiStanovsky )

Honored to have been part of the ISCOL 2025 panel with such great professors! @yoavgo @melhadad It was an interesting discussion on AI’s role in academia and research, with diverse opinions and challenging perspectives. Thanks @iscol_meeting and @ella_rabinovich for having me!

3/🛠️ Improvement: Finally, we used these insights to build better systems: • Improving models using reformulation feedback: arxiv.org/abs/2501.04513 • Making evaluation metrics actually reflect human preferences: arxiv.org/abs/2408.04909

2/ 🗣️ Human Insights: Next, we looked at how humans actually generate language: • How visual inputs affect our syntax: arxiv.org/abs/2302.04811 • How our cultural background shapes what we choose to describe: arxiv.org/abs/2409.16646

I’m thrilled to announce that I have successfully defended my PhD! 🎓🥂 Huge thanks to my advisors Gabi Stanovsky, Omri Abend, and Lea Frermann for their mentorship My goal: Making Vision-Language Models (VLMs) more human-like. 🧵👇

Our team will present multiple works at #EMNLP2025. Check out the papers and get connected if interested.

Excited to present our paper "ReliableEval: A Recipe for Stochastic LLM Evaluation via Method of Moments" at #EMNLP2025 🎉 Everyone knows LLMs are prompt-sensitive, yet we still report single-prompt scores. Our work suggests a method to make evaluation statistically reliable!

Looking forward to presenting our TACL paper on enhancing LLM creativity at #EMNLP2025 tomorrow (Wed, Nov 5)! 📍 Room A108 🕝 14:30–16:00 (Linguistic Theories, Cognitive Modeling & Psycholinguistics) Details below ⬇️ #NLP #LLMs #Creativity

I can't make it to #EMNLP2025, but @EliyaHabba and @GiliLior will present our PromptSuite🌈(demo!): a framework tackling prompt sensitivity by generating benchmark variations for any task. Try it with just a few lines of code or the web interface! eliyahabba.github.io/PromptSuite/

Heading to #EMNLP2025! 🎉 Two of our papers will be there — come say hi 👋 🖼️ Image Captioning Evaluation — Nov 5, 17:45 📄 arxiv.org/abs/2408.04909 🕵️ Deceptive LLM Agents (Mafia Game) — Nov 5, 13:00 📄 arxiv.org/abs/2506.05309