Song
205 posts
@songdng
Post-training @liquidai | Prev. PhD @CriteoAILab & @mlia_isir
🤔 Ever wondered how to summarize an entire book 𝑎𝑙𝑙 𝑎𝑡 𝑜𝑛𝑐𝑒? We delve into this question in our recent paper “𝐋𝐎𝐂𝐎𝐒𝐓: 𝐒𝐭𝐚𝐭𝐞-𝐒𝐩𝐚𝐜𝐞 𝐌𝐨𝐝𝐞𝐥𝐬 𝐟𝐨𝐫 𝐋𝐨𝐧𝐠 𝐃𝐨𝐜𝐮𝐦𝐞𝐧𝐭 𝐀𝐛𝐬𝐭𝐫𝐚𝐜𝐭𝐢𝐯𝐞 𝐒𝐮𝐦𝐦𝐚𝐫𝐢𝐳𝐚𝐭𝐢𝐨𝐧” (#EACL2024). 🧵 (1/8)
Today we release Antidoom, an open-source method that removes a common failure mode in reasoning models: the doom loop. Doom-loop rates before and after, with eval scores up across the board: > Early LFM2.5-2.6B checkpoint: 10.2% → 1.4% > Qwen3.5-4B: 22.9% → 1% (greedy sampling) 🧵
Today, we're releasing LFM2.5-8B-A1B, a device-optimized model designed to power real-life applications on phones, laptops, PCs, robots, and fast & lightweight server-side use-cases. > 8B MoE, 1.5B active > Expanded 128K context > LFM2.5 flagship hybrid MoE architecture > Trained on 38T tokens + large-scale RL > fast, reliable tool calling, punching above its weight, comparable to models with up to 4x its size > customizable on a single GPU for any specialized task > LFM2 open-weight license 🧵
