Mostofa Patwary

Nemotron-H model family with 4X throughput gains without compromising accuracy! Blog: developer.nvidia.com/blog/nemotron-… Weights: huggingface.co/collections/nv…

Base model Nemotron-H weights have been released under a research license: huggingface.co/nvidia/Nemotro…

Nemotron-H base models (8B/47B/56B): A family of Hybrid Mamba-Transformer LLMs are now available on HuggingFace: huggingface.co/nvidia/Nemotro… huggingface.co/nvidia/Nemotro… huggingface.co/nvidia/Nemotro… Technical Report: arxiv.org/abs/2504.03624 Blog: research.nvidia.com/labs/adlr/nemo…

Excited to share some models we've been training over the last couple of months!

I'm glad to see Nemotron-4-340B doing well on the LMSYS Arena leaderboard! Lots more work to do to improve the Nemotron family. Most importantly, though, I hope this model supports the development of AI across the community.

NVIDIA joins the open-source LLM race offering a 15B-parameter model that demonstrates strong capabilities in English, multi-lingual, and coding tasks arxiv.org/abs/2402.16819

Nvidia announces Nemotron-4 15B introduce Nemotron-4 15B, a 15-billion-parameter large multilingual language model trained on 8 trillion text tokens. Nemotron-4 15B demonstrates strong performance when assessed on English, multilingual, and coding tasks: it outperforms all existing similarly-sized open models on 4 out of 7 downstream evaluation areas and achieves competitive performance to the leading open models in the remaining ones. Specifically, Nemotron-4 15B exhibits the best multilingual capabilities of all similarly-sized models, even outperforming models over four times larger and those explicitly specialized for multilingual tasks.

@GregoryDiamos Thanks Greg :-)

Congrats @MostofaPatwary and team on your new Megatron language model with 530B parameters. I once wondered what a language model trained using all of the energy from the sun would be like. You plenty of orders of magnitude to keep going.

Language models do just keep scaling! Today we’re announcing Megatron-Turing NLG: a 530B parameter language model. Joint work from @NVIDIA and @Microsoft. Trained using Megatron and DeepSpeed on DGX SuperPod. developer.nvidia.com/blog/using-dee…

Excited to share that our work on scaling up language model training with Megatron will appear at SuperComputing 2021! We achieve 502 petaFLOP/s on 3072 GPUs (per-GPU throughput of 52% of theoretical peak) on a model with 1 trillion parameters. Paper: arxiv.org/abs/2104.04473

Excited to share our #acl2020 work on Large Scale Multi-Actor Generative Dialog Modeling, done with amazing coauthors Alex Boyd, @MostofaPatwary, Mohammad Shoeybi, @ctnzr! Join our QA session 5-6 UTC tonight and 21-22pm UTC tomorrow. arxiv.org/abs/2005.06114

Experimental Evaluation of Mixed Precision Training for End-to-End Language Modeling. On NVIDIA Tesla V100 GPU, we achieved a 4.1X performance improvement for the whole application with minimal changes. #MachineLearning #AI bit.ly/2Ixo9cw