Distributed State

A small step for mankind, a massive leap for decentralised training... for agency. In the space of 9 months, @tplr_ai went from 1.2B -> 72B. It's never been easy, and has broken everyone on the team multiple times. But I speak for all of us when I say it is the most rewarding thing we have ever done. We have a fraction of the resources. We don't have the PhDs. But Bittensor shows you it doesn't matter. Innovation happens at the edge. We innovate through scarcity. The ones who rewrite the rules are never the ones with the most. They're the ones who refuse to accept the limits they were handed. Bittensor is prophecy. Subnets (@covenant_ai and others) are the tools through which that prophecy is manifested. Next stop: TRILLIONS.

It's very interesting that cryptographic protocols and neural networks have the same high-level architecture (where they jumble information as it moves sequentially across many layers). This is the result of a convergent evolution - cryptographic protocols need every output bit to depend on every input bit in complicated ways, and similarly, NNs need output to make connections between inputs. But they're in some sense doing opposite things. While cryptographic protocols take something which has a lot of structure and make it seem indistinguishable from random, NNs take something which may look random and extract structure from it. Much more on this idea in the full episode with @reinerpope

TSP provides higher throughput. On 1024 MI300X GPUs - 128K tokens of context - 8 GPUs per model copy, TSP hits 173M tok/sec vs 86M for matched TP+SP. TSP exists alongside existing schemes such as EP and PP. Paper: arxiv.org/abs/2604.26294 Blog: zyphra.com/post/tsp

i'd rather lose $100,000 a month with a team of 10 i feel sorry for you if you don't understand this

Python made AI accessible. Rust can make parts of AI understandable. That’s the bet behind Category Theory for Tiny ML in Rust. We’re building tiny ML systems from first principles using: Rust types typed transformations composition training loops category theory as an engineering tool Not abstraction cosplay. Executable structure. Working draft. Public feedback welcome.

You don’t need to run a miner to interact with Compute Substrate. If you want to test: - sending transactions - UTXO behavior - system mechanics You can now request coins directly on the forum. This is for testing, not speculation. Link: forum.computesubstrate.org/t/getting-star…

Late to the reading party, but such good reads💯wasn't aware of such instabilities faced during training. > "Why Training MoEs is So Hard" by @_xjdr Link : (noumena.com/research/0000-…) > "MoE in Transformers" by @huggingface Link : (huggingface.co/blog/moe-trans…)

🤯🤯🤯! after battling a nasty NCCL bug we finally have crazy crazy results. Thats adjusted MFU (to causal mask) I need to recheck my numbers because that's crazy high

When inner loop gets accelerated, everything moves faster.

In this paper is presented ZipCCL, a library of lossless compressed communication collectives designed to alleviate the communication bottleneck in distributed LLM training. arxiv.org/pdf/2604.27844

The issue is that benchmarks simultaneously undersell and oversell the gap. DeepSeekv4 belongs to an entirely new category of model by side and by design, and the most dramatic step taken this year to bring the open architecture ecosystem closer to frontier.

Compute Substrate mainnet is live. There are no admins, no privileged roles, no central authority. It is a network with fixed rules.

AI systems today don’t fail because they’re wrong. They fail because someone has to decide what’s right.

Open source projects like transformers are drowning in AI agent PRs, so we auto-merged everything to see what would happen and share the results. tl;dr: if 100s of agents want to fix something, it’s probably broken. Agent PRs on transformers have quadrupled over the past quarter. We classified and validated 1k PRs (42% features, 39% bugs, 13% docs). The quality distribution is skewed toward noise. But the bug fixes cluster around a small number of hotspots: tokenizer handling, model loading, dtype mismatches, multimodal pipelines. I.e. an underlying problem. When 28 PRs independently flag the same area, that is signal regardless of whether any individual fix is correct. One issue generated 39 near-identical PRs in a day. Each applied the same decorator pattern to a different model file. A maintainer would do the same cognitive work 39 times, so a single combined PR replaces all of that work. We built tooling to cluster, deduplicate, and merge these contributions at scale, then ran an experiment: bulk-merge hundreds of agent PRs into a fork, benchmark it, and see what breaks. Nothing broke. Zero delta across three models on arc_challenge, gsm8k, and hellaswag. The contributors are not adversarial. They lack the context to evaluate whether the agent's output is correct. Check out this blog post, where we dive deep on this pipeline: huggingface.co/spaces/hugging…

Hot take: ⛔️Long-context reasoning is not only a context-window problem. 🫢It is a control-flow problem. Standard recursive LLMs often ask the model to invent recursive code during inference. That means the model is not just reasoning. It is also writing the program that controls the reasoning. λ-RLM takes the opposite view: Keep neural inference for bounded subproblems. Move recursion into a typed λ-calculus runtime. SPLIT → MAP → FILTER → REDUCE Less chaotic agent loop. More inspectable reasoning. The result: 29/36 wins vs standard RLM up to +21.9 accuracy points up to 4.1× lower latency We open-sourced the repo: github.com/lambda-calculu… Star it if you want more structured alternatives to “just make the context window bigger”. #AI #LLM #RLM #LambdaRLM #MachineLearning

@MoggyTao @covenant_ai @tplr_ai You strawman the situation as it’s an easier pill to swallow , and it’s ok. However , if you ever wanted to think critically about how the networks biggest advocate got here , it might help y’all sort some structural issues out . Wish you the very best .

@DistStateAndMe @covenant_ai @tplr_ai You fucked up mate and you know it, threw your toys out the pram and your whole business with it 😆

@tplr_ai @covenant_ai #ICLR2026 Presenting our work on "Hetereogenous SparseLoCo" arxiv.org/abs/2601.02360

.@DistStateAndMe Is back on the Covenant Discord

@tylaokx @designerbiru @tplr_ai The UI is still banging between

@tylaokx @designerbiru @tplr_ai The scam of leaving and selling tokens I owned ?

Which #Bittensor subnet has the best UI right now? Genuinely asking. I want to study it. 👇

@csouthai @synapz_org @tplr_ai @covenant_ai I really hope the farmer and helper managed to sort their differences out

A farmer hired a helper to fix fences. He did it too fast, so the farmer made him peel potatoes. He did that too fast too. Then the farmer told him to sort potatoes by size. The helper quit. “Why?” asked the farmer. “Fixing and peeling are easy. Endless tiny decisions are what kill you.”

@surrealitynet @MoggyTao @covenant_ai @tplr_ai Lol. Bruh I don’t know what you tell you. I’ve told you where my heads at , and if you don’t believe me I can’t help you. “Bad PR” is subjective . And rushing things will never make sense

@DistStateAndMe @MoggyTao @covenant_ai @tplr_ai You have to be less vague. I don’t believe you haven’t thought about this before leaving. So I know you actually have a concise plan on how are you planning to incentivize future contributors. Announcing it + more updates now is the only way pushing away the bad PR