Stefan Bejgu

Four of our industrial #PhD students, @SBejgu, @PereLluisHC, @alescire94 and @SimoneTedeschi_, were awarded their #PhD in #AI last Friday with the best grades (and two cum laude)! Congrats all! 👏 🎉 With @RNavigli, their advisor and Babelscape's scientific director, in the photo

@rohanpaul_ai Thank you for highlighting our work! 🙌 You can explore the training and evaluation datasets on Hugging Face here: huggingface.co/collections/Ba….

Want to know if an AI is lying? LLM-OASIS helps detect factual accuracy in AI outputs with 81k training examples. LLM-OASIS introduces the largest dataset for training factuality evaluators, created by extracting and falsifying information from Wikipedia articles. This enables end-to-end verification of AI-generated text accuracy. ----- 🤔 Original Problem: LLMs still produce hallucinations in their outputs. Existing factuality evaluation resources are limited by being task-specific, small in size, or focused only on simple claim verification. ----- 🔧 Solution in this Paper: → LLM-OASIS extracts claims from Wikipedia passages using an LLM-based pipeline. → The system falsifies selected claims by introducing subtle but critical factual errors. → It generates pairs of factual and unfactual texts based on the original and modified claims. → The dataset covers 81k Wikipedia pages with 681k claims for training factuality evaluators. ----- 💡 Key Insights: → Task-agnostic factuality evaluation is possible with a large-scale synthetic dataset → Wikipedia provides reliable source material for generating factual/unfactual pairs → Human validation confirms high quality of automated data generation (90%+ accuracy) ----- 📊 Results: → GPT-4 achieves 60% accuracy on end-to-end factuality evaluation → 68% accuracy with Retrieval Augmented Generation → Human validation shows 96.78% accuracy for claim extraction → Dataset creation pipeline maintains 89-98% accuracy across all steps

✨ Meet #ResiDual, a novel perspective on the alignment of multimodal latent spaces! Think of it as a spectral "panning for gold" along the residual stream. It improves text-image alignment by simply amplifying task-related directions! 🌌🔍 arxiv.org/abs/2411.00246 [1/6]

🚀 Today marks the start of the @MakerFaireRome 🎉 We’re super excited to be part of it and introduce #Vera, our new LLM-powered fact-checking tool! 🤖🧠 Here’s a sneak peek of what you can expect at our booth! 👀✨ #MakerFaireRome #FactChecking #LLM #ArtificialIntelligence

✨Tired of verifying #AI-generated info?😵 🔎Meet Vera, our #LLM-based fact-checker using trusted sources from the Web or your knowledge base. 💥Check out the live demo at Rome #MakerFaire2024 (Oct 25-27)! More info 👉: babelscape.com/article/babels… #FactChecking #Misinformation

🔵🔴When do distinct learning processes learn similar representations? Detecting patterns and conditions for this to happen is an open direction: a thread🧵 Working on this topic? Submit at: openreview.net/group?id=NeurI… DEADLINE: 20 Sept See you at @NeurIPSConf! 🔵🔴 [1/N]

Post #ACL2024nlp dinner in #Bangkok with most of the presenting/attending band from our group + @Babelscape. Left to right: @SBejgu @LorenzoProiet13 @giumartinelli_ @19Stefano97 @RiccardoRicOrl @FMTucci @RNavigli @KarimAsh14 & Celebrating our outstanding paper award! #NLProc

Come to chat with us at the poster session C in #EACL2024, starting now in room Radisson! #NLProc

Exciting strides in text summarization with LLMs 🚀but verifying their factual accuracy is still an open challenge 🤔 We introduce FENICE, a factuality-oriented metric for summarization with a strong focus on interpretability🔍arxiv.org/abs/2403.02270 #NLProc #LLMs #Factuality

📢Happy to share that "Neuralign: A Context-Aware, Cross-Lingual and Fully-Neural Sentence Alignment System for Long Texts" has been accepted to #EACL2024 (main) 🫂Huge thanks to my co-authors @SBejgu @SimoneTedeschi_ @ConiaSimone @RNavigli 📃More details coming soon! #NLProc

How to Mitigate Hallucinations in Large Language Models (#LLMs)?🤔 In this new @Medium article, I review the most recent research on mitigating hallucinations, and explain the main methods that are used to address this issue. 📑 generativeai.pub/how-to-mitigat… #AI #NLP #GPT4 #LLM

Tomorow at 5pm @SBejgu will present our research work on word alignment in 14 language pairs! @CLiC_it_conf #CliCit2023, joint with @SapienzaNLP and many other partners! #NLProc #LLMs

Excited about #ChatGPT for your business? Check out #Emotionary! The revolutionary #multilingual AI system that understands #emotions: #analyze customer reviews, #track feelings in #news, #socialmedia & #chatbot conversations! babelscape.com/emotionary

📢 It looks like relative representations are here to stay! I'm beyond thrilled to announce that our work has been selected as one of the notable top 5% (oral) papers at #iclr23 ! 🥳 twitter.com/moschella_luca… [1/5]

The Rome Workshop on 10 Years of #BabelNet & Multilingual Neurosymbolic Natural Language Understanding was a great success, with productive in-person discussions, amazing talks & >100 online participants! Thanks! @ERC_Research @Babelscape @SapienzaNLP @SapienzaRoma @WikiResearch

Open & commercial Neural Machine Translation models heavily suffer from disambiguation biases! We present DiBiMT, our novel benchmark for lexical-semantic bias in MT at #ACL2022! By @Valahaar @FedeMartelli25 @FrancescoSaina @RNavigli @ELEXIS_EU #NLProc 📝:researchgate.net/publication/35…

Empower your natural language applications with WordAtlas! #WordAtlas is the next-generation multilingual knowledge graph. What makes it special is its linkage between words and concepts in hundreds of languages. babelscape.com/wordatlas

Classy is a @PyTorch-based library for the fast prototyping and sharing of deep neural network models. It wraps the best libraries like PyTorch Lightning, Transformers, @streamlit and offers them to users with a simple CLI interface. Try it here: github.com/sunglasses-ai/…