BigScience Large Model Training

The BLOOM model is now officially released! Read more here: bigscience.huggingface.co/blog/bloom Find the model here: huggingface.co/bigscience/blo…

The Bloom paper is out. Looks like it's doing worse than current GPT3 API in zero-shot generation tasks in English but better than other open-source LLMs & better than all in zs multi-lingual (which was the main goal). Proud of the work from the community! arxiv.org/abs/2211.05100

Crosslingual Generalization through Multitask Finetuning 🌸 Demo: huggingface.co/bigscience/blo… 📜 arxiv.org/abs/2211.01786 💻github.com/bigscience-wor… We present BLOOMZ & mT0, a family of models w/ up to 176B params that follow human instructions in >100 languages zero-shot. 1/7

The super-fast inference solutions are finally here for all to use:

What do @StabilityAI @EMostaque #stablediffusion & @BigscienceW Bloom - aka the coolest new models ;) - have in common? They both use a new gen of ML licenses aimed at making ML more open & inclusive while keeping it harder to do harm with them. So cool! huggingface.co/blog/open_rail

@bparlan You can read more about the language selection in this tweet thread: twitter.com/YJernite/statu…

Dear @BigScienceLLM, Not sure whether it is too late and my eyes need rest, or you do not include the Turkish language in databases at all? #languages" target="_blank" rel="nofollow noopener">bigscience.huggingface.co/blog/building-…

The Technology Behind BLOOM Training🌸 Discover how @BigscienceW used @MSFTResearch DeepSpeed + @nvidia Megatron-LM technologies to train the World's Largest Open Multilingual Language Model (BLOOM): huggingface.co/blog/bloom-meg…

BLOOM is here. The largest open-access multilingual language model ever. Read more about it or get it at bigscience.huggingface.co/blog/bloom hf.co/bigscience/blo…

🌸@BigscienceW BLOOM's intermediate checkpoints have already shown some very cool capabilities! What's great about BLOOM is that you can ask it to generate the rest of a text - and this even if it is not yet fully trained yet! 👶 🧵 A thread with some examples

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 102%

For 111 days, we've enjoyed world-class hardware stability and throughput thanks to the hard work of our friends at @Genci_fr, @INS2I_CNRS, Megatron & DeepSpeed. Having reached our objective earlier than expected, we'll keep training for a few more days. Stay tuned, more soon ;)

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 101%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 100%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 99%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 98%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓ 97%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ 96%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ 95%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ 94%

▓▓▓▓▓▓▓▓▓▓▓▓▓▓░ 92%