Nir Ratner

LLMs can attend to way more text than their original context window -- Accepted to ACL 2023 main conference 🥳🥳 "Parallel Context Windows for Large Language Models" Paper: arxiv.org/abs/2212.10947 Code: github.com/AI21Labs/Paral… #ACL2023 #ACL2023NLP #NLProc

Alright, ready for this? Here’s a quick thread with highlights from The 7 October Parliamentary Commission Report—also known as The Roberts Report. It’s the most comprehensive document published to date on the events of October 7, 2023, detailing thousands of incidents from that day. The report was compiled by a commission led by Lord Andrew Roberts of Belgravia—a historian and member of the UK House of Lords. Its members included Lords, Barons, and MPs from across the British political spectrum. “We added nothing that wasn’t proven. The horror needs no exaggeration.” - Lord Roberts, Chair of the Commission In this thread, I’ll share key excerpts from the report, divided into a few categories: 🔹 The numbers behind the barbaric attack 🔹 Selected quotes from Hamas terrorists during the assault—taken from Hamas’s own materials: GoPro footage, audio recordings, live streams, and social media posts 🔹 Some of the most extreme atrocities committed that day 🔹 Powerful testimony from survivors No drama. No exaggeration. Just the facts. Let’s start with the numbers: 👇

Today we launched Jamba 1.6, the best open model for private enterprise deployment. AI21’s Jamba outperforms Cohere, Mistral and Llama on key benchmarks, including Arena Hard, and rivals leading closed models while maintaining unmatched speed and quality.  Now available on AI21’s Studio and @Hugging Face.  Learn more: ai21.com/jamba/

📄Jamba-1.5 whitepaper is out! The whitepaper details the architecture, training schemes, novelties and in-depth evaluations of our new long context hybrid SSM-Transformer models - Jamba-1.5-Large and Jamba-1.5-Mini. Arxiv: arxiv.org/abs/2408.12570 Here are some highlights and insights from the paper 👇1/7

We released the #Jamba 1.5 open model family: - 256K #contextwindow - Up to 2.5X faster on #longcontext in its size class - Native support for structured JSON output, function calling, digesting doc objects & generating citations twtr.to/giIEE #AI #LLM #AI21Jamba

Introducing Jamba, our groundbreaking SSM-Transformer open model! As the first production-grade model based on Mamba architecture, Jamba achieves an unprecedented 3X throughput and fits 140K context on a single GPU. 🥂Meet Jamba ai21.com/jamba 🔨Build on @huggingface

#NLProc I am happy to share I will be presenting our paper “Generating Benchmarks for Factuality Evaluation of Language Models” at #EACL2024! Check out our updated version on arxiv, introducing a new benchmark: Expert-FACTOR (based on ExpertQA) 🚀 Paper, Datasets & Code: ⬇️⬇️

Hamas raped and mutilated women on 7 October bbc.com/news/world-mid…

🧵 I just witnessed ~45 minutes of footage of the October 7th terrorist attack at the @AtlanticCouncil courtesy of @IsraelinUSA along with colleagues from think tanks across the ideological spectrum. What I saw was worse than I’ve ever seen. Pure evil. ***Trigger Warning***

@zehavoc @broseph_stalin That's a blunt lie: timesofisrael.com/liveblog_entry…

@broseph_stalin What ?

Israel quietly reduced the Oct 7 civilian deaths to 900, equivalent to just 2 days of Palestinian civilian deaths in Israel’s 40+ of bombing. The full story hasnt come out, but as the Haaretz investigation revealed many Oct 7 civilian deaths were from Israel shooting on their own

@zehavoc @snarwani Nobody, he is part of channel 14, sort of the "newmax" super right wing channel in Israel.

@snarwani Who is that guy ?

Israeli TV host threatens to kill all Muslims and Arabs, and says 'we are prepared to fight the United States and the whole world too.' Israeli hate and bloodlust is next level.

@jastorj No, it will be quadratic in the number of task token plus the number of tokens in a single window, but not quadratic in the sum of tokens in all windows.

@NirRatner In the paper it is stated that attention masks allows to keep the computational cost in linear in the number of parallel contexts B. My doubt is that task tokens are still attending to every token in each context, wouldn't it still keep the cost to quadratic in window length?

#NLProc Is the context window of your LLM too small for you? Do you want to add in-context examples but can’t? Parallel Context Windows increase any LLM’s context *without further training*! 🚨 Paper from @AI21Labs "Parallel Context Windows Improve In-Context Learning" 🧵

@boknilev @janundnik Ha, I am not aware of anyone doing that. Kudos @janundnik, nice idea.

@NirRatner @janundnik Yeah I’m asking specifically about the trick to efficiently evaluate by just running a forward pass on many examples and collecting probabilities after every example.

Interested in few-shot in-context learning (ICL) in LLMs? You might like to hear about this neat trick. Many papers just report performance at a single fixed number N of in-context examples. Well it turns out, you can get nice ICL training curves like these at *no extra cost*.

@boknilev @janundnik @boknilev Many reported results for multiple choices of N (arxiv.org/abs/2102.09690 for example), but I can't recall any paper specifically focusing on those plots. I suspect that Min, S did this one of those plots in one of her papers but I can't recall which one.

@janundnik I thought this is common but can’t recall a particular paper. @NirRatner

Generating Benchmarks for Factuality Evaluation of Language Models paper page: huggingface.co/papers/2307.06… Before deploying a language model (LM) within a given domain, it is important to measure its tendency to generate factually incorrect information in that domain. Existing factual generation evaluation methods focus on facts sampled from the LM itself, and thus do not control the set of evaluated facts and might under-represent rare and unlikely facts. We propose FACTOR: Factual Assessment via Corpus TransfORmation, a scalable approach for evaluating LM factuality. FACTOR automatically transforms a factual corpus of interest into a benchmark evaluating an LM's propensity to generate true facts from the corpus vs. similar but incorrect statements. We use our framework to create two benchmarks: Wiki-FACTOR and News-FACTOR. We show that: (i) our benchmark scores increase with model size and improve when the LM is augmented with retrieval; (ii) benchmark score correlates with perplexity, but the two metrics do not always agree on model ranking; and (iii) when perplexity and benchmark score disagree, the latter better reflects factuality in open-ended generation, as measured by human annotators.

#NLProc New paper! “Generating Benchmarks for Factuality Evaluation of Language Models” From @AI21Labs Evaluate an LM’s tendency to generate true facts from your knowledge-intensive corpus! Paper: arxiv.org/abs/2307.06908 Code & Data (soon): github.com/AI21Labs/factor 🧵⬇️

Will be presenting a poster today at 1100 in #ACL2023NLP Come and say hello! 👾

Do you want to process long texts with LLaMA models, but can't due to its context length? This one is for you! We have implemented PCW for LLaMA, enabling larger contexts!! Link: github.com/AI21Labs/Paral…

A nice TL;DR we did for the original preprint: twitter.com/NirRatner/stat…

For those of you who read the original preprint - there’s more! We have fresh results of using PCW for question answering and multi-hop reasoning!

LLMs can attend to way more text than their original context window -- Accepted to ACL 2023 main conference 🥳🥳 "Parallel Context Windows for Large Language Models" Paper: arxiv.org/abs/2212.10947 Code: github.com/AI21Labs/Paral… #ACL2023 #ACL2023NLP #NLProc