No_name
696 posts




TokenSpeed scheduler originally used a Radix Tree–based memory pool. As new attention patterns became more complex, we deprecated the Radix Tree–based design and implementation, and rebuilt the memory manager with a flat, block-based KV cache architecture. TokenSpeed now uses a single flat paged pool with heterogeneous views, making it easier to support different attention mechanisms. Similar approaches have also been explored in the community, including @vllm_project's Jenga and LMDeploy’s TurboMind. This redesign happened around the release of TML’s Inkling, so we were able to support Inkling from day one with the new architecture. We are excited to keep building with the open-source community. 👇





GLM 5.2 @ 19.8 tok/sec on 2 DGX Sparks ⚡️ I switched to a dspark draft model with K=2 (model from @RedHat_AI) Acceptance is ~68% because the drafter was trained with the full FP8 model... Next step is to fine tune the drafter so acceptance on this super quant goes up



Dla inwestorów JSW⚒️ mocny wzrost cen 🟢 wszystkich produktów w stosunku do pierwszego kwartału. biznes.pap.pl/wiadomosci/fir…








hello world✨👋 finally sending my first post. (my previous X got hacked be careful of scammers)













