NVIDIA AI Japan

NVIDIA Nemotron で構築する日本のソブリン AI | Nemotron Labs x.com/i/broadcasts/1…

NVIDIA Nemotron で構築する日本のソブリン AI | Nemotron Labs x.com/i/broadcasts/1…

ファナック、２本の腕でＴシャツ畳むロボット　模倣学習で実現 nikkei.com/article/DGXZQO…

📣 いよいよ明日！5 月 13 日 (水) 10 時配信 スタート！ 信頼性が高く、本番環境で利用可能な AI の実現においてオープンソース AI が果たす役割について議論📚 AI 研究者 今井翔太氏 @ImAI_Eruel と東京エレクトロン @TokyoElectron_J 西口賢弥氏が登壇します。お楽しみに！ nvda.ws/4u6jG6M nvda.ws/4tzqREC

How do we make LLMs faster and lighter? Don’t force the GPU to adapt to sparsity. Reshape the sparsity to fit the GPU! ⚡️ Excited to share our new #ICML2026 paper in collaboration with @NVIDIA: "Sparser, Faster, Lighter Transformer Language Models". This work introduces new open-source GPU kernels and data formats for faster inference and training of sparse transformer language models: Paper: arxiv.org/abs/2603.23198 Blog: pub.sakana.ai/sparser-faster… Code: github.com/SakanaAI/spars… While LLMs are undoubtedly powerful, they are increasingly expensive to train and deploy, with a large part of this cost coming from their feedforward layers. Yet, an interesting phenomenon occurs inside these layers: For any given token, only a small fraction of the hidden activations actually matter. The rest approximate zero, wasting computation. With ReLU and very mild L1 regularization, this sparsity can exceed 95% with little to no impact on downstream performance. So, can we leverage this sparsity to make LLMs faster? The challenge is hardware. Modern GPUs are optimized for dense matrix multiplications. Traditional sparse formats introduce irregular memory access and overheads that cancel out their theoretical savings for GEMM operations. Our contribution is twofold: 1/ We introduce TwELL (Tile-wise ELLPACK), a new sparse packing format designed to integrate directly in the same optimized tiled matmul kernels without disrupting execution. 2/ We develop custom CUDA kernels that fuse multiple sparse matmuls to maximize throughput and compress TwELL to a hybrid representation that minimizes activation sizes. We used our kernels to train and benchmark sparse LLMs at billion-parameter scales, demonstrating >20% speedups and even higher savings in peak memory and energy. This work will be presented at #ICML2026. Please check out our blog and technical paper for a deep dive!