samoline

2/2 Our YaRN-extended Covenant-Chat (32k context) demonstrates what's possible when you combine extended context windows with optimized gradient-based training. Longer context means the model sees more relevant information during each training step, leading to stronger learning signals and faster convergence. YaRN is already integrated into the platform and the tournaments - the full post-training pipeline coming to users in January 🤙

1/2 Gradients takes the best decentralized, open-source base LLM from Templar Covenant and finetunes it into a chatbot assistant that can carry multi-turn conversation and reasonably respond to user queries, here's how we did it: - Chat template integration and embedding update - YaRN context window extension from 2k to 32k - Finetuning on a mix of SOTA open source datasets built to improve open source finetuning to destroy benchmarks, but augmented using our own synthetic few shot enhancement pipeline The results speak for themselves: Gradients training slashed test loss by 75% in under one epoch. This dramatic improvement shows how efficiently our autoML training pipeline extracts performance from your models.

Introducing Gradients Instruct V2. We don't cherry-pick benchmarks. We take on them all and win. Have a play: chutes.ai/app/chute/b245…

Gradients V2 vs Qwen 32B (same size): → +85% improvement on MMLU tasks → Parity on GSM8K → +2% improvement on MMLU Pro → +10% improvement on BBH → +38% improvement on GPQA If these acronyms mean nothing to you, the translation: Gradients wins. Beats our own V1 across the board too. There isn't a better place on earth to train your model. Open competition beats closed labs. gradients.io

Last week we had a new text tournament winner 🥳 Here's how our new winner outperformed the previous champion 🔥 → They predefined a different learning rate for each model instead of a fixed value for all models → They set a dynamic number of steps in packing for Instruct tasks based on the duration of the task → They added padding_free for DPO to speed up training, enabling the ability to train on more data when job duration is short but data volume is large → They reduced the learning rate for GRPO in order to balance between reward and KL

Once again, a gradients model smashes competition on a few popular benchmarks! SOTA model Qwen3-32B -> trained on SN56 with one of our data pipelines. We evaluated both the original "Qwen3-32B" and our gradients-trained version, and the results speak for themselves:

Quick update after yesterday's AWS outage (which partially affected Gradients API and front-end): Everything's stable again 👍 This week's focus - post-6.0 release work: → Migrating validator to adjust for synthetic jobs being removed → Reducing system load by removing unnecessary data pre-processing - better latency and lower costs → Shifting tournament trainers to on-demand compute - massive cost savings vs 24/7 infrastructure → Moving tournament participation to burn-to-play - opens competition to unlimited miners while creating sustainable economics Infrastructure optimization to support the scale we're building toward.

Big week ahead for Gradients with a major codebase update. → 6.0 - upgraded incentive mechanism + privacy-focused hosted service for customers → Gradients V2 model benchmarking auto-integrated into our in-house pipeline → TAO payment integration for customers goes live Stay tuned

We just published the full story of how open-source tournament miners reached parity with closed-source. The economics, the timeline, the future of Gradients. Read it here: @samoline/open-source-miners-achieve-parity-the-tournament-revolution-reaches-critical-mass-9c288ce8d830" target="_blank" rel="nofollow noopener">medium.com/@samoline/open…