Gul Saeed Khattak

10/10 The tiny test has already paid for itself: it found a real blocker before I spent hours downloading 1.56TB. Ill keep the full K3 download paused until compatibility is fixed and two clean test runs pass.The lesson: test small, find problems early, then scale with confidence

9/10 Instead of guessing—or downloading 1.56TB—I am testing multiple compressed-tensors versions against the official K3 configuration on an empty meta model. This checks quantization setup across more than 82,000 expert modules without downloading weights.

1/14 I wanted to test AirLLM with Kimi K3 on my 128GB NVIDIA DGX Spark. But K3’s official model files are 1.56TB. Downloading them could take many hours..only to discover at the end that one small part of the software does not work. So I chose a smarter first step. 🧵

4/10 Then the rehearsal found the real blocker: the “compressed-tensors” software already in my NVIDIA container cannot understand the official K3 MXFP4 weight instructions. It rejects fields describing how K3’s weight scales and zero points are stored.

3/10 In simple terms, the warehouse system worked. AirLLM found the requested boxes on the SSD, moved only those boxes to the GPU, used them and removed them afterward. It did not accidentally move the entire warehouse into memory.