Sabitlenmiş Tweet
Great article from @chinazhidx on how Meituan trained its 1.6T-parameter LongCat-2.0 entirely on 50,000 domestic AI accelerators. Here's the TL;DR:
- LongCat-2.0 is China’s first trillion-parameter model to complete the full training and inference pipeline entirely on domestic AI compute, using a peak cluster of 50,000+ domestic accelerator cards, the largest training run ever on a Chinese AI computing platform.
- The project began in July 2023, with the team adapting its training stack to domestic hardware and working closely with the chip vendor through weekly technical meetings.
- As they scaled, they rewrote key pieces of the software stack, including high-performance deterministic implementations of FlashAttention and Scatter operators, enabling deterministic training at massive scale while keeping the performance penalty to roughly 5%.
- The team also found that numerical error on some domestic chips was actually lower than on mainstream chips, arguing that as long as computational correctness is maintained, there is no fundamental technical barrier preventing domestic hardware from training frontier models.
- They acknowledge that individual domestic chips still lag the world’s best. But their view is that system-level engineering, co-designing algorithms, infrastructure, and hardware, can compensate for weaker individual chips.
- Their conclusion is that the next phase of AI competition won’t be won by better chips or better models alone, but by full-stack systems engineering.
Rui Ma@ruima
English
