Jesse Vig
458 posts
How does positional bias influence faithfulness in long-form summarization (LFSumm)? 🤔 In our latest study, we analyze the effect of positional bias on faithfulness metrics and generated summaries, and methods to mitigate this bias. 📄Paper: arxiv.org/abs/2410.23609 Thead🧵👇
Excited to share a new preprint on the 🩴FlipFlop Effect. We prompt LLMs with a classification task, and challenge the model by following up with “Are you sure?”. The model can confirm or flip its answer. The results? More flips than a gymnastics competition! 🤸♂️ 1/N
Very excited to present SWiPE in person at ACL in a few weeks. In short, we collaborated with Wikipedia editors to understand the process of document simplification and take a (small) step towards improving document accessibility by releasing a large dataset!
Finding a document too dense to decipher? 🤔Content a bit convoluted? Essay too esoteric? Check how we simplify and improve document readability using SWiPE. Join us in making knowledge accessible to all! 🌐 🔗Paper: arxiv.org/abs/2305.19204 🔗Github: github.com/salesforce/sim…
🚨🆕📄🚨 How gold is your human evaluation? We seek the answer, and its implications in the GPT3 era, in our preprint “Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation” Paper: arxiv.org/abs/2212.07981 Equal contribution @YixinLiu17
📰 How can we make it easier for news readers to access nuanced and diverse coverage from multiple sources? In our #CHI2023 paper, we propose to highlight news coverage diversity through generated *discord questions* shown to the readers. Link: bit.ly/41yviRP
🚨🆕📄🚨 How gold is your human evaluation? We seek the answer, and its implications in the GPT3 era, in our preprint “Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation” Paper: arxiv.org/abs/2212.07981 Equal contribution @YixinLiu17
