covenant

I built autoresearch-rl and pointed it at GRPO fine-tuning on @basilic_ai A100s. One command. 15 iterations. Zero human intervention. 100% infrastructure success rate. GSM8K pass@1: 26% baseline to 36%. The hard part wasn't the search algorithm. It was the infrastructure.

NVIDIA's Jensen Huang just called Covenant-72B 'a pretty crazy technical accomplishment' on the All-In Pod. We've got the man who built it: Sam Dare of Subnet 3, Templar (@tplr_ai). Plus a demo of OpenOats -- the open source AI note-taker and assistant -- from @yazins. Track it all on the live docket: thisweekinstartups.com/docket

This week on Novelty Search :: SN3 @tplr_ai @covenant_ai 72B. ~1.1T tokens. Commodity internet. No centralized cluster. No whitelist. Anyone with GPUs could join or leave freely. Special guests :: @DistStateAndMe @erfan_mhi @Hevalon @joellidin @amir_sarfi Hosted by :: @const_reborn 9PM UTC // via Bittensor Discord

Winners from the SF Bittensor hackathon track: 🥇 @provenonce_ai :: Proof of Assurance Cryptographic assurance for AI workflows. 🥈 @PCCProtocol :: Physical Capability Cloud Protocol Verified coordination for physical-world services. 🥉 @tech_insignia :: Insignia Decentralized quant trading with competitive model evaluation. Prize pool + $3500 cash + $3000 in @basilic_ai compute credits + Up to 1000 TAO discretionary investment from @UnsupervisedCap + 50 TAO toward mainnet registration from @CrucibleLabs + Intro call with @bitstarterAI Intelligence at The Frontier partner @FundingCommons Bittensor hackathon track produced by @redwoodnorth_io

Pretty wild to see our work on PULSE show up in a real 1T-scale post-training run done by @cursor_ai. Cursor built Composer 2 in collaboration with Fireworks and trained it across multiple datacenters, getting huge savings by syncing only the weights that actually changed between RL checkpoints. Fireworks reports that more than 98% of BF16 weights can stay bit-identical from one checkpoint to the next, and they cited our paper on this, too. That is basically the exact sparsity pattern we showed in our paper, where we introduced PULSE, a lossless method for 100x more efficient weight-sync communication for RL training. Their system is very close to this idea in practice: exploiting the fact that only a tiny fraction of weights actually change between RL steps. The deeper reason for this is not that RL gradients are sparse. They are not. The gradients are still dense. What becomes sparse is the realized weight update. In RL, learning rates are tiny, and with Adam, the update size stays bounded around the learning rate. Then BF16 adds a hard threshold: if the update is too small relative to the weight, it just rounds away, and the stored weight does not change at all. So from one checkpoint to the next, most of the model literally stays identical. That is why this is such a useful systems idea. Lower precision, like using BF16, does not just save compute. It can also save communication, because more tiny updates get absorbed and fewer weights need to be shipped. At that point, compute efficiency and comms efficiency stop being a tradeoff. They start reinforcing each other. If you want the deeper story on why RL updates get this sparse, the theory behind it, and how to push weight-sync bandwidth down by 100x+, take a look at our paper: arxiv.org/pdf/2602.03839 The Fireworks blog on Composer 2 that cited our work: fireworks.ai/blog/frontier-… The animation is taken from Fireworks!

x.com/i/article/2034…

Used autoresearch to make @grail_ai GRPO trainer 1.8x faster on a single B200. I kept postponing this for weeks since the bottleneck in our decentralized framework was mainly communication. But after our proposed technique, PULSE, made weight sync 100x faster, the training update itself became the bottleneck. Even with a fully async trainer and inference, a slow trainer kills convergence speed. A task that could've eaten days of my time ran in parallel while I worked on other stuff. Unlike original autoresearch, where each experiment is 5 min, our feedback loop is way longer (10-17 min per epoch + 10-60 minutes of installations and code changes), so I did minimal steering when it was heading in bad directions to avoid burning GPU hours. The agent tried so many things that failed. But, eventually found the wins: Liger kernel, sequence packing, token-budget dynamic batching, and native FA4 via AttentionInterface. 27% to 47% MFU. 16.7 min to 9.2 min per epoch. If you wanna dig deeper or contribute: github.com/tplr-ai/grail We're optimizing everything at the scale of global nodes to make decentralized post-training as fast as centralized ones. Stay tuned for some cool models coming out of this effort. Cheers!

TGIF #29 tomorrow! The Covenant-72B thread reached well beyond Bittensor this week. @DistStateAndMe and the full @covenant_ai team talk about what that traction means, where decentralized AI sits in the broader conversation, and what comes next. Miners, come celebrate with us. We are opening the stage to anyone who contributed compute to the run or has a story to share. Request to speak and we will bring you up. x.com/i/spaces/1oKMv…

Bittensor Subnet Ideathon — Round I Results 🏆 After reviewing an incredible set of submissions, we’re excited to announce the teams moving forward to Round II (Testnet Phase). Both our Top 7 and Honorable Mention teams will advance to the next round. Top 7 Teams (Ranked Alphabetically): • C-SWON — Aditya Singh • ChronoSeek — Connor Daly • Defektr — Hiw3 • Mentiss_AI — Jeremy Wang • OpenMind — Bello Iteoluwakisi • Proven — Christopher H.G • vividverse — @vividverseai Honorable Mentions: • BitDefense — A.G. • DaVinci — Chris Romano • Keyword Intelligence Subnet (KIS) — Ozan Andaç • Moirai Subnet — @ai_moir • Probity — Dicky Bayu Sadewo • Query Agent — Daniel Derefaka • sotarad-ai — Wade • Talos Protocol — Christopher H.G • TensorClock — Valeriy Lihachev • Titan — JKohav With over 150+ projects, the competition was fierce — and these teams stood out for their strong mechanism design and promising subnet ideas on @opentensor 👏

This week on TGIF, @DistStateAndMe previews the Covenant72B report ahead of the full release on Monday. He'll walk through what the training run produced, what we learned, and what comes next. Also on the agenda: multi-GPU Crusades went live this week and other updates across the ecosystem. x.com/i/spaces/1jxXg…

Bittensor Subnet Ideathon at Sankalp Africa Summit: Winning Ideas 🇰🇪 Together with @opentensor, we saw sharp, problem-driven pitches from some of Africa’s strongest talent — truly an honor to support this initiative @SankalpForum. Congrats to our winners: 🏆 1st Place: Medical AI for Pre-screening — Wade Walton 🎖️ Runner Up: Creative Subnet — David Opiyo Special thanks to Sam @DistStateAndMe and Gareth @GarethHowe95938 for their mentorship, and our partner Andres @amarquezlara from #Ufacilitate for hosting. Learn more 👇