FairMind

Programming is changing so fast... I'm trying VS Code Cursor + Sonnet 3.5 instead of GitHub Copilot again and I think it's now a net win. Just empirically, over the last few days most of my "programming" is now writing English (prompting and then reviewing and editing the generated diffs), and doing a bit of "half-coding" where you write the first chunk of the code you'd like, maybe comment it a bit so the LLM knows what the plan is, and then tab tab tab through completions. Sometimes you get a 100-line diff to your code that nails it, which could have taken 10+ minutes before. I still don't think I got sufficiently used to all the features. It's a bit like learning to code all over again but I basically can't imagine going back to "unassisted" coding at this point, which was the only possibility just ~3 years ago.

If you are looking for something to read this weekend, I am happy to share that Chapter 7 on instruction finetuning LLMs is now finally live on the Manning website: manning.com/books/build-a-… This is the longest chapter in the book and takes a from-scratch approach to implementing the instruction finetuning pipeline. This includes everything from input formatting to batching with a custom collate function, masking padding tokens, the training loop itself, and scoring the response quality of the finetuned LLM on a custom test set. (The exercises include changing prompt styles, instruction masking, and adding LoRA.) As a side note, it's also the last chapter, and the publisher is currently preparing the layouts for the print version. PS: After moving, traveling, and returning from the awesome SciPy conference, I am now also super eager to finally type up my notes from recent research papers on the instruction finetuning front. I will share them soon, in the next few days, as a follow-up!

Si chiama Nessi è un chatbot basato sull’intelligenza artificiale generativa: a sperimentarlo la Camera di Commercio di Milano per rendere i testi dei bandi più comprensibili. Su @wireditalia l'intervista ad @alexio di @FairMindAI 👇 wired.it/article/chatbo…

💙 Exciting Milestone at @FairMindAI! 💚 In just two days following the release of the @SapienzaNLP team's Minerva base models, we at #FairMind are thrilled to unveil our latest innovation: 𝗠𝗶𝗻𝗲𝗿𝘃𝗮-𝟯𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁-𝘃𝟭.𝟬. This new iteration enhances instruction-following capabilities, marking a significant step forward in AI technology. 𝗤𝘂𝗶𝗰𝗸 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀: 🚀 𝗕𝗮𝘀𝗲 𝗠𝗼𝗱𝗲𝗹: Minerva-3B-base-v1.0 (Transformer-based, bilingual in Italian and English) 📚 𝗗𝗮𝘁𝗮 𝗨𝘁𝗶𝗹𝗶𝘇𝗲𝗱: Italian translation of the Stanford Alpaca-Cleaned dataset, broadening the model's applicability across domains. 🎯 𝗢𝗯𝗷𝗲𝗰𝘁𝗶𝘃𝗲: Sharpen the model’s proficiency in processing and executing complex instructions in Italian. 📈 𝗣𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲: * Hellaswag_it: 0.5197 * Arc_it: 0.3157 * M_mmlu_it 5-shot: 0.2631 * Average: 0.366 🌐 Discover more on our model page: 𝗠𝗶𝗻𝗲𝗿𝘃𝗮-𝟯𝗕-𝗜𝗻𝘀𝘁𝗿𝘂𝗰𝘁-𝘃𝟭.𝟬 on Hugging Face: huggingface.co/FairMind/Miner… #AI #MachineLearning #TechInnovation #StartupLife #ModelRelease

Hello World 💙💚