Haonan Li retweetledi
Today, we are releasing a new version of K2 (K2-V2), a 360-open LLM built from scratch as a superior base for reasoning adaptation, while still excelling at core LLM capabilities like conversation, knowledge retrieval, and long-context understanding.
K2 fills a major gap: highly capable models with no transparency. Instead of releasing only weights, we’re sharing the full training story — dataset recipes, mid-training checkpoints, logs, code, and evaluation tools. That’s 360-open.
What’s inside:
• 70B dense transformer engineered as a reasoning-enhanced base model
• Native 512K context (extendable via RoPE scaling)
• Mid-training reasoning phase
• Strong tool-use scaffolding
What we’re open-sourcing:
• 250M+ reasoning traces (math, planning, multi-step logic)
• Full pre- & mid-training data compositions
• All mid-training checkpoints
• Training logs, code, Eval360
Performance:
• GPQA-Diamond: 55.1% mid-training → 69.3% after SFT (strongest fully open 70B model)
• KK-8 Logic Puzzles: 83% — competitive with DeepSeek-R1 & OpenAI o3-mini-high
• ArenaHard V2: 62.1% — close to Qwen3 235B
• Outperforms Qwen2.5-72B and approaches Qwen3-235B despite being smaller and fully transparent.
🔗 The Model:
bit.ly/3KIYwuo
🔗Technical Report:
bit.ly/49V8h2U
🔗Blog:
bit.ly/49V7gb6
English
