Sabitlenmiş Tweet

Ready to take control?
Check out the repo, spin it up in minutes, and let us know what you think !
🌟 Star it on GitHub: github.com/celestial-rose…
cc @CloudflareDev @yusukebe @threepointone
English
Celestial Rose
2.3K posts

@Rose__Celestial
busy building my dream tech https://t.co/WyGkUxIagR · https://t.co/d9XcSgJliN







But the KV cache is created for each transformer layer. By sending each layer’s KV cache after it’s computed, we overlap communication with computation. We stream the KV cache and hide the network delay. We achieve a 4x speedup in prefill & 3x in decode, with 0 network delay.













72+ TPS on Qwen 3.6 27B on a Macbook pro M5 max. MTPLX V2 out now! The fastest way to run models on MLX.
