Thomas Chaton

How can we use small LLMs to shift more AI workloads onto our laptops and phones? In our paper and open-source code, we pair on-device LLMs (@ollama) with frontier LLMs in the cloud (@openai, @together), to solve token-intensive workloads on your 💻 at 17.5% of the cloud cost while maintaining 97.9% of the accuracy. See Gru and the Minions in action below, 🔉on please (h/t @cartesia)!

Introducing DeepSeek-R1 optimizations for Blackwell, delivering 25x more revenue at 20x lower cost per token, compared with NVIDIA H100 just four weeks ago. Fueled by TensorRT DeepSeek optimizations for our Blackwell architecture, including FP4 performance with state-of-the-art production accuracy, it scored 99.8% of FP8 on MMLU general intelligence benchmark. FP4-optimized DeepSeek checkpoint now available on @huggingface: huggingface.co/nvidia/DeepSee…

Here I show you how to finetune and deploy DeepSeek R1 (8B) for < $1.00 in 8 minutes using the AI Hub from @LightningAI ⚡️⚡️

🚀 Introducing NSA: A Hardware-Aligned and Natively Trainable Sparse Attention mechanism for ultra-fast long-context training & inference! Core components of NSA: • Dynamic hierarchical sparse strategy • Coarse-grained token compression • Fine-grained token selection 💡 With optimized design for modern hardware, NSA speeds up inference while reducing pre-training costs—without compromising performance. It matches or outperforms Full Attention models on general benchmarks, long-context tasks, and instruction-based reasoning. 📖 For more details, check out our paper here: arxiv.org/abs/2502.11089

@ThomasScialom It would be fantastic if the data and pre/post training code was open sourced too.

The team worked really hard to make history, voila finally the Llama-3.1 herd of models...have fun with it! * open 405B, insane 70B * 128K context length, improved reasoning & coding capabilities * detailed paper ai.meta.com/research/publi…

@Thom_Wolf It would be fantastic if the data and pre/post training code was open sourced too.

Among the most impressive aspect of the Llama 3.1 release is the accompanying research paper! Close to 100 pages of deep knowledge-sharing on LLMs like we havn't seen very often recently What a treat! It covers everything, pretrainining data, filtering, annealing, synthetic data, scaling laws, infrastructures, parallelism, training recipees, post-training adaptation, tool-use, benchmarking, inference strategies, quantization, vision, speech, videos... Mind-blown! Maybe the single paper you can read today to join the field of LLM from zero right to the frontier Read it here and feel the open-science ai.meta.com/research/publi…

@bhimrazyadav @LightningAI Hey @bhimrazyadav, thanks a lot for the shot-out. Really appreciated. Lucky to have you as a user !

Use LitData with MinIO —a high-performance, S3-compatible object store designed for large-scale AI/ML, data lakes, and databases It's a great library from the @LightningAI team.

@tomcocobrico @LightningAI That's great @tomcocobrico. Feel free to join your Discord if you need help on anything and you can publish to the Templates Gallery too lightning.ai/lightning-ai/s… ;)

When you get 2000$ in cloud credits for the fine tuning course but the first website you sign up for is actually @LightningAI ‘s new studio. I have to say it looks really neat. 24/7 free cpu with persistent storage, easy switch to gpus, reasonable auto sleep

@hertzfelt_io @LightningAI Looks cool !

I finished @LightningAI quest 3 - Run a hyperparameter sweep! lightning.ai/lightning-ai/s…

A while ago I complained here about persistent storage in Google Colab. Have been using @LightningAI Studios for a while now for: - Full VSCode (incl. GH Copilot) - Persisted files shared across notebooks - Multi-GPU/node (!!) It's been great. Feels like a remote ML workstation

@DataChaz @GoogleColab @LightningAI @code @pycharm So true. Colab is so 2015.

@bhimrazyadav @LightningAI Hey @bhimrazyadav. That's great ! BTW, do you know about LitData: github.com/Lightning-AI/l…. This is the library we built to make data processing on @LightningAI fast and scalable.

I was able to process almost 100 GB of image data using the concurrent_task_executor function in less than 5 minutes, @LightningAI Studios. Feel free to drop any suggestions or questions.

🚀 Boost your Python code's speed with `concurrent_task_executor`! 🏎️💨 No more waiting for slow processing. Just throw your tasks at this function, sit back, and watch your code go "Brrrrr" through your data! 💥✨ #Python #Coding #Efficiency #GoBrrrrr 🐍💻

@karpathy @karpathy Give it a try to Lightning Studio. You won't use your local computer ever again !

Setting up my shiny new fully maxed out Space Black MacBook Pro M3 Max 128GB 16-inch (upgrading from an M1 Air). I always like to set up the new one with a clean slate, from scratch - this time I will not allow my dev configuration to get out of hand. Then we'll talk to it.

@elitepax @LightningAI That's great to hear @elitepax. You can have a look at to our published Studios: lightning.ai/studios. There is a ton to learn from there and 1-click away to get everything ready.

Just got invited to Studio. It blows my mind how much value is packed in it. I've been riding the AI wave for the past year, learned a ton in the process, launched some production apps, but my stuff is scattered around because I'm moving quick and I always have to deal with some kind of friction when I start prototyping new ideas. With Lightning Studio I was able to rapidly pick up best practices, fine tuned a model with a custom dataset, served it and now I'm chatting with it, all under 1h. Hats down, true product & engineering! 💪🏻

Introducing Lightning AI Studios - A persistent GPU cloud environment. Setup once. Ready any time. Code online. Code from your local IDE. Prototype. Train. Serve. Multi-node. All from the same place. No credit card. 6 Free GPU hours/month. lightning.ai

@AnindyadeepS @LightningAI @williamfalcon @lantiga Hey @AnindyadeepS, thanks ! Great timing ! I am working on the docs right now. They should be available in the coming weeks ! Would you mind joining our Slack, you can reach out to me directly. My username is tchaton.

Hey @LightningAI, lightning .data looks great but I am not able to find any documentation on how to use it with S3 and DDP. Can anyone help? cc: @williamfalcon @chaton_thomas @lantiga

@AuroraNemoia Hey @AuroraNemoia Check this out: lightning.ai/lightning-ai/s…. With @LightningAI, you can easily prepare your dataset to train any models.

You can duplicate the Studio, you will get everything. The dependencies, the data, the code, etc... Finally, a benchmark you can reproduce yourself with a click!

We just finished benchmarking cloud data-loading libraries over Imagenet 1.2M: - Lightning AI Streaming Dataset - Webdataset - MosaicML Streaming Conclusion: Lightning AI is the fastest (up to 80%) 🚀 lightning.ai/lightning-ai/s…

Prepare a 1 trillion token dataset to train LLMs from scratch in under 4 hours instead of days with @LightningAI Studio! Everything is included, the final datasets, the code, dependencies, etc... Get started in seconds as no setup is needed. lightning.ai/lightning-ai/s…