
The evolution of decentralized training:
2022 — Together GPT-JT (6B): proving multi-machine collab is possible
2023 — SWARM Intelligence (~1B): proposed a heterogeneous-node collaborative training framework
2024 — INTELLECT-1 (10B): decentralized training across whitelisted peers
2026 — @covenant_ai-72B / SN3 @tplr_ai : the first 72B model trained decentrally to outperform centralized training on mainstream benchmarks
This article is worth translating to english.
When Bittensor was created, no-one knew decentralized training was possible, the models of the day were full of hallucinations and no-one felt the threat of job loss. @DistStateAndMe well done
0xai@0xai_dev
English













