Yuval Reif

Is dataset debiasing the right path to robust models? In our work, “Fighting Bias with Bias”, we argue that in order to promote model robustness, we should in fact amplify biases in training sets. w/ @royschwartzNLP In #ACL2023NLP Findings Paper: arxiv.org/abs/2305.18917 🧵👇

Ever used a top-ranked LLM that just... felt wrong for you? You’re not alone. Instead of leaderboards, many of us turn to "vibe-testing" - manually comparing models to our own needs. But can we turn these feelings into a structured evaluation? New paper: "From Feelings to Metrics" 🧵

New paper: LLMs encode harmful content generation in a distinct, unified mechanism Using weight pruning, we find that harmful generation depends on a tiny subset of the weights that are shared across harm types and separate from benign capabilities. 🧵

🚨New paper alert🚨 🧠 Instruction-tuned LLMs show amplified cognitive biases — but are these new behaviors, or pretraining ghosts resurfacing? Excited to share our new paper, accepted to CoLM 2025🎉! See thread below 👇 #BiasInAI #LLMs #MachineLearning #NLProc

🚨 New paper! We present CHIMERA — a KB of 28K+ scientific idea recombinations 💡 It captures how researchers blend concepts or take inspiration across fields, enabling: 1. Meta-science 2. Training models to predict new combos noy-sternlicht.github.io/CHIMERA-Web 👇 Findings & data:

The longer reasoning LLM thinks - the more likely to be correct, right? Apparently not. Presenting our paper: “Don’t Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning”. Link: arxiv.org/abs/2505.17813 1/n

Heading to @iclr_conf ✈️🧩 ‘Tokens→Words’ shows how LLMs build full‑word representations from sub‑word tokens and offers a tool for vocab expansion. 🚀 See our #ICLR2025 poster ‑ 26.4, 15:00‑17:30. 📄 arxiv.org/abs/2410.05864 🔗 guykap12.github.io/FromTokens2Wor… 👇

[📄] Are LLMs mindless token-shifters, or do they build meaningful representations of language? We study how LLMs copy text in-context, and physically separate out two types of induction heads: token heads, which copy literal tokens, and concept heads, which copy word meanings.

✨ Ever tried generating an image from a prompt but ended up with unexpected outputs? Check out our new paper #FollowTheFlow - tackling T2I issues like bias, failed binding, and leakage from the textual encoding side! 💼🔍 arxiv.org/pdf/2504.01137 guykap12.github.io/guykap12.githu… 🧵[1/7]

אני רוצה לרכז שמות של עמותות שנשמח כמשפחה כשיתרמו להן לזכרו של נועם אחי. ספרו לי על עמותות ויוזמות ישראליות שעושות עבודה טובה בתחום בריאות הנפש

Hallucinations are a subject of much interest, but how much do we know about them? In our new paper, we found that the internals of LLMs contain far more information about truthfulness than we knew! 🧵 Project page >> llms-know.github.io Arxiv >> arxiv.org/abs/2410.02707

📢Paper release📢 : 🔍 Ever wondered how LLMs understand words when all they see are tokens? 🧠 Our latest study uncovers how LLMs reconstruct full words from sub-word tokens, even when misspelled or previously unseen. arxiv.org/pdf/2410.05864 (preprint) 👀 👇 [1/7]

In which layers does information flow from previous tokens to the current token? Presenting our new @BlackboxNLP paper: “Attend First, Consolidate Later: On the Importance of Attention in Different LLM Layers” arxiv.org/abs/2409.03621 1/n

Ever skimmed an article, pinpointing key info, and wished for a tailor-made summary without crafting it yourself?🤔 Introducing SummHelper: your go-to for personalized summarization. 📜✏️ w/ Niv Nachum @pyshmulik @obspp18 Ido Dagan 1/n

📢 New paper alert! 📢 Thrilled to announce `Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias'. Do instruction tuning and RLHF amplify biases in LMs? 🧵 Check it out arxiv.org/abs/2308.00225 W @boknilev @GabiStanovsky and N. Rosenfeld.

The data used to train an AI model is vital to understanding its capabilities and risks. But how can we tell whether a model W actually resulted from a dataset D? In a new paper, we show how to verify models' training-data, incl the data of open-source LMs!arxiv.org/abs/2307.00682

Read🧐, Look 👀 or Listen🎧? What’s needed to solve a multimodal dataset. 📣Excited to share our two-step method that maps each instance in a multimodal dataset to the modalities required for processing it. w. @YonatanBitton @royschwartzNLP Paper📄: arxiv.org/abs/2307.04532 1/5

We hope our framework could ease creating robustness tests for new datasets and guide the development of robust methods. Meet us at #Repl4NLP poster session July 13th at #ACL2023 🙌 Code: github.com/schwartz-lab-N… Bias-amplified dataset splits at 🤗: huggingface.co/bias-amplified… 8/8

We also compare automatically-extracted anti-biased test splits to manually curated challenge sets (HANS and PAWS). Anti-biased sets are as hard as challenge sets, but capture a more diverse set of biases (occurring in all task labels, vs. mostly in one label). 7/

Is dataset debiasing the right path to robust models? In our work, “Fighting Bias with Bias”, we argue that in order to promote model robustness, we should in fact amplify biases in training sets. w/ @royschwartzNLP In #ACL2023NLP Findings Paper: arxiv.org/abs/2305.18917 🧵👇