Ori Ernst

🚨 New paper out! 📄 What happens when LLMs & RLMs face conflicting answers to a question? 🤔 They often ignore disagreement and confidently pick one “correct” answer. 🤯 📄 arxiv.org/pdf/2508.12355 #AI #LLM #NLP #MachineLearning

📢 Call for Papers: NewSumm 2025 - The 5th New Frontiers in Summarization Workshop at EMNLP 2025 The summarization research community is invited to submit to NewSumm 2025, co-located with EMNLP 2025! As LLMs continue to transform our field, we're expanding beyond traditional summarization to tackle the most pressing challenges in text generation and summarization. 🎙️ Invited Talks We have a exceptional lineup of speakers: @mohitban47, @armancohan, @gregd_nlp, @alexfabbri4, Mirella Lapata, Jey Han Lau, @stefan_fee, Yulia Tsvetkov! 🎯 Key Focus Areas - Trustworthiness & hallucination reduction in summarization - Multi-modal and long-form summarization - Query-focused and retrieval-augmented approaches - Fairness, bias, and privacy-preserving methods - Efficiency in large model inference - Novel evaluation metrics and methods 📝 What We're Looking For - Long papers (up to 8 pages) and short papers (up to 4 pages) - Survey papers and position papers welcome - Both archival and non-archival tracks available - Dual submission allowed with ARR fast-track option ⏰ Important Dates - Submission deadline: August 15, 2025 - ARR commitment deadline: August 22, 2025 - Workshop date: November 9, 2025 We're particularly excited about work that pushes the boundaries of summarization research, addresses real-world deployment challenges, and explores the intersection of summarization with other NLP tasks. Ready to contribute? Submit via our OpenReview portal: openreview.net/group?id=EMNLP… Find more details about NewSumm at newsumm.github.io/2025/, co-organized by @YueDongCS, @wendyxiao06091, Haopeng Zhang, @ruizhang_nlp, @oriern1, @LuWang__, @feiliu_nlp #NLP #Summarization #EMNLP2025

A blizzard is raging in Montreal when your friend says “Wow, the weather is amazing!” Humans easily interpret irony, while LLMs struggle at it. We propose a 𝘳𝘩𝘦𝘵𝘰𝘳𝘪𝘤𝘢𝘭-𝘴𝘵𝘳𝘢𝘵𝘦𝘨𝘺-𝘢𝘸𝘢𝘳𝘦 probabilistic framework as a solution. arxiv.org/abs/2506.09301 @ #acl2025

🚨 RAG is a popular approach but what happens when the retrieved sources provide conflicting information?🤔 We're excited to introduce our paper: “DRAGged into CONFLICTS: Detecting and Addressing Conflicting Sources in Search-Augmented LLMs”🚀 A thread 🧵👇

We propose task-agnostic metrics that work across various tasks using only model output probabilities. Check out our #ACL2025 work at arxiv.org/abs/2506.00637! Thank you to @oriern1 + Prof Jackie Cheung!

❗️ Confidence in text generation is tricky, as models can be confident in many, valid answers. 👀 Can we account for this without extra tuning or heuristics?

🛠️ PreSumm helps us move upstream: Instead of always fixing bad generations, we can predict and prevent them by detecting the source. 📂 Paper + model: github.com/oriern/PreSumm #AI #NLP #Summarization #ACL2025

📉 Practical win: We used PreSumm to filter low-quality docs and had humans summarize only those. The resulting summaries were significantly better than those filtered by baseline methods. (6/n)

🧵 New paper at Findings #ACL2025 @aclmeeting! Not all documents are processed equally well. Some consistently yield poor results across many models. But why? And can we predict that in advance? Work with Steven Koniaev and Jackie Cheung @Mila_Quebec @McGill_NLP #NLProc (1/n)

Do LLMs hallucinate randomly? Not quite. Our #ACL2025 (Main) paper shows that hallucinations under irrelevant contexts follow a systematic failure mode — revealing how LLMs generalize using abstract classes + context cues, albeit unreliably. 📎 Paper: arxiv.org/abs/2505.22630 1/n

🚨 Introducing LAQuer, accepted to #ACL2025 (main conf)! LAQuer provides more granular attribution for LLM generations: users can just highlight any output fact (top), and get attribution for that input snippet (bottom). This reduces the amount of text the user has to read by 2 orders of magnitude compared to standard self-citation methods. We invite researchers to use this experimental framework to advance localized attribution research!

RefVNLI Towards Scalable Evaluation of Subject-driven Text-to-image Generation

Models like DeepSeek-R1 🐋 mark a fundamental shift in how LLMs approach complex problems. In our preprint on R1 Thoughtology, we study R1’s reasoning chains across a variety of tasks; investigating its capabilities, limitations, and behaviour. 🔗: mcgill-nlp.github.io/thoughtology/

Attending 🌴#EMNLP2024 or interested in what people are working on these days? We organized it all for you with Knowledge Navigator! Explore all @emnlpmeeting accepted papers mapped by themes and subtopics—giving you a bird’s-eye view of the conference knowledge-navigators.github.io/emnlp_presenta…

@oaimli Thank you @oaimli !

Cool work! Alignment between the source and the summary is what we really want. This will push the community to dive more into understanding the internal process of multi-document summarisation!

Overall, this work showcases that alignments from an MDS dataset empower a rich collection of multi-document related tasks. These tasks are appealing on their own, and are additionally advantageous as sub-components of MDS solutions. Data: github.com/oriern/SPARK (7/7)

We automatically derive and release train and test datasets from the alignment data for six tasks: (1) salience detection, (2) proposition coreference clustering, (3) evidence detection, (4) text planning, (5) sentence fusion, and (6) in-context passage fusion. (6/n)

🚨 Happy to announce our latest research “The Power of Summary-Source Alignments", accepted in #ACL2024nlp findings! Work with @obspp18, @lovodkin93, Sharon Adar, @mohitban47, @JacobGoldberge1, Ran Levy, Ido Dagan @AmazonScience @uncnlp @biunlp arxiv.org/abs/2406.00842 (1/n)