Alan Akbik

🎓This semester, you can join us online in my "Deep Learning and NLP" course at HU Berlin! In the course, we go all the way from zero to "NLP hero": from classical NLP to transformers and LLMs, plus in-depth into to PyTorch🔥 How to enrol: hu-berlin.de/en/studies/cou…

@pieterdelobelle Also check out the nice logo :)

Announcing 🐤 Büble-LM 🐤, our new state-of-the-art 2B language model (LM) for German! Trained by @pieterdelobelle with a novel trans-tokenization approach, it outperforms other German LMs like Sauerkraut and LLäMmlein on most benchmarks. Try it out! huggingface.co/flair/bueble-l…

Problem: There are thousands of language models on 🤗 @HuggingFace - but which one is the best for *your* NLP task? Solution: ⚖️ TransformerRanker ⚖️! Our newest library directly connects to🤗 HF and quickly/efficiently ranks LMs for your task! github.com/flairNLP/trans…

2 papers accepted to #EMNLP2024 main conference! See you in Miami in November! ☀️🏖️🍹

Meet us this week at #ACL2024 in Bangkok, Thailand! 🛕🏝️ My students present two papers tomorrow (13.08.): • Fundus, our simple-to-use news crawler: arxiv.org/pdf/2403.15279 (at 10:30) • Automatic "best transformer" selection: alanakbik.github.io/papers/2298_Ch… (at 12:15)

Flair now supports🔥biomedical text analysis🔥: • Detect biomedical named entities (🧬 genes, 🦠 diseases, ⚛ chemicals, ..)! • Link them to entries in a knowledge base! See the tutorial (flairnlp.github.io/docs/category/…) and🔥Flair 0.14.0 release notes (github.com/flairNLP/flair…)!

@krasul Most newspapers have topics (such as financial, sports, etc.). Those are automatically parsed and assigned to the respective crawled article!

@alan_akbik are the articles also tagged by type? e.g. financial, sports etc?

Crawl 1 million news articles in 7 hours*! Announcing the release of 🗞️ Fundus v0.4 🗞️! github.com/flairNLP/fundus *crawling speed depends on your internet connection. I launched it 1.5 days ago and already gathered over 5 million news articles across 41 languages 🔥🔥🔥!

More details: - The position is funded through the HEIBRiDS programme: heibrids.berlin - PhD project is on NLP for scientific literature (material science) and co-supervised by Dr. Thomas Unold from the Helmholtz Center Berlin - Apply to project 60029!

Want do a PhD in NLP? We have a new PhD position available, fully funded for 4 years! - ⏰ Application deadline: August 23rd - 🗓️ Start of PhD project: January 2025 - 🔥 Requires strong PyTorch/Python skills, and knowledge of NLP/ML - ✍️ Apply here: heibrids.berlin/admission/open…

Pieter joins as a visiting researcher from KU Leuven until the end of the year. Hope you enjoy your stay at HU and in Berlin!

Excited to have Dr. Pieter Delobelle join our lab! Pieter is an LLM-expert, well known for creating Dutch foundation LLMs 🇳🇱🇧🇪. @pieterdelobelle: Welcome to Berlin! We look forward to working together on Dutch/German/English LLMs! 💪 pieter.ai

@rasbt @rasbt any idea why they are using NLL over the token probabilities in their teacher-student setup? Could they not also use MSE over the final layer hidden state?

3) Knowledge distillation (as in MiniLLM): The general idea is to transfer knowledge from a larger model (the teacher) to a smaller model (the student). Here, they trained a 27B (teacher) model from scratch and then trained the smaller 2B and 9B (student) models on the outputs of the larger teacher model. The 27B model doesn't use knowledge distillation but was trained from scratch to serve as a "teacher" for the smaller models.

What's noteworthy in the newly released Gemma 2 LLMs? The main theme is that they explore techniques w/o necessarily increasing the dataset sizes but rather focus on developing relatively small & efficient LLMs. The are 3 main design choices to create the 2B & 9B models:

Tacos and NLP! 🌮🌵 Meet us this week at NAACL 2024 in Mexico City! My group is presenting two papers on Tuesday: - OpinionGPT, our very biased GPT model: arxiv.org/abs/2309.03876 - LM Pub Quiz, our LLM evaluation library: arxiv.org/abs/2404.04113 #NLProc #NAACL2024

Announcing the 🧠LM Pub Quiz🧠- the ultimate test of your LLMs factual knowledge! Probe any LM (masked/causal) on Huggingface with our new library. We mitigate biases (answer distribution, domain, etc.) to give the most exact reading possible. Try it! lm-pub-quiz.github.io

Our paper "🗞️Fundus🗞️: A Simple-to-Use News Scraper Optimized for High Quality Extractions" accepted to ACL 2024 Demos! Fundus allows you to easily build a high quality corpus of news data for your NLP project. Try it out :) github.com/flairNLP/fundus #NLProc #ACL2024NLP

@GuillaumeLample Awesome! Have you tried benchmarking on PECC, our problem extraction + coding benchmark? It's extremely challenging for all models we tried, and it would be cool to see how well Codestral-22B fares! hallerpatrick.github.io/pecc/

Today we are releasing Codestral-22B, our first code model! Codestral is trained on more than 80 programming languages and outperforms the performance of previous code models, including the largest ones. It is available on our API platform, through instruct and fill-in-the-middle endpoints, and can be easily integrated into VScode plugins. You can also use it for free on Le Chat: chat.mistral.ai

PECC Leaderboard is here: huggingface.co/spaces/Patrick…

Announcing 📊PECC📊, our extremely challenging LLM benchmark for coding and math problems! Even very strong LLMs get less than 50% of coding questions correct. And less than 10% of math! - Presented 9:00 tomorrow (Friday) at #lreccoling2024 - Paper: arxiv.org/abs/2404.18766