Mark

@axiomofmind @talemi @BrianRoemmele The only limitation is your imagination. I can deploy the same workflow on a DGX spark, and run a 3 bit quant on SSD streaming with much higher cache rate and tk/s. Probably still slow but good enough to let the agent work overnight and check the results in the morning.

@talemi @BrianRoemmele So you'd rather run a simple task at 0.32 tk/s over a long time instead of just using something faster to achieve the same results in 100x less time?

BREAKTHROUGH: A full, unmodified 2.78-trillion-parameter Kimi K3 on a consumer laptop by streaming only the activated experts from NVMe. YOU CAN’T RUN KIMI K3 “ON THAT” THEY DECLARED. There are many paths to do it. This is one: Marco Bambini Just Gave Us the Full Kimi K3 on a Laptop Meet Marco Bambini he did something that felt impossible only a day ago. He built WASTE Weight-Aware Streaming Tensor Engine a clean, dependency-free C inference engine that runs the complete, unmodified 2.78-trillion-parameter Kimi K3 model by streaming only the activated experts straight from NVMe. No distillation. No pruning. No cloud. The full open-weight model. We have it running in the lab right now. What Marco Actually Built Kimi K3 is a sparse Mixture-of-Experts system. Only about 4 % of its weights fire on any given token. Marco’s insight was simple and ruthless: the idle experts do not need to live in RAM. They only need to be reachable in time. WASTE keeps the model’s “trunk” (attention, shared components, embeddings) resident in memory — roughly 27 GB on the converted container. The 82,000+ routed experts stay on disk as tightly packed residual vector-quantized records. When the router selects its 16 experts per layer, the engine issues direct, cache-bypassing reads from the internal NVMe and feeds them into a bounded expert cache. The rest of the machine’s RAM becomes working space for that cache. On a 64 GB MacBook Pro with the container on the internal SSD, we are measuring 0.32–0.34 tokens per second at a comfortable memory budget. Prefill sits a little higher. The vision tower works. Logits match the reference implementation to within a few parts in a million. It is the real model. The container itself is 982 GiB after conversion from the original 1.42 TB MXFP4 weights. Minimum RAM floor is just over 29 GB for short context. Push the budget higher and the expert cache hit rate climbs; push too high and you start paging and the speed collapses. The sweet spot on current consumer hardware is clear and measurable. How We Are Testing It We converted the official weights, verified the container, and began systematic runs the same day the engine stabilized. First we confirmed numerical fidelity against the PyTorch reference on short prompts. Then we moved to longer generation, vision inputs, and multi-turn chat using Kimi’s native XTML format. We are measuring wall-clock decode, expert I/O versus compute split, cache hit rates at different RAM budgets, and thermal behavior under sustained load. We are also exercising the OpenAI-compatible server that sits on top of the same C library so we can drop the model into existing agent loops without rewriting anything. Early observations: •Expert I/O dominates the timeline, as expected. On a fast internal NVMe the engine is already near the practical ceiling of the storage subsystem. •The architecture’s sparsity is the entire enabler. A dense model of this size would be dead on arrival for local use. •Context length is currently limited by RAM more than by the model itself. Practical working contexts sit comfortably in the tens of thousands of tokens on 64 GB hardware; the full million-token window will need more memory or smarter KV management. •Thinking tokens are expensive at this speed. Long internal monologues turn into multi-hour runs. For agent work we are already experimenting with tighter control over when full reasoning is requested. We are treating this as a research instrument, not a finished product. Every run teaches us something about expert locality, prefetch opportunities, and how far pure software streaming can push trillion-scale inference on ordinary machines. 1 of 2

@JDGrandstaff @mindyisser Islamic Republic of Iran would be perfect for you

@mindyisser This is great, actually. This scumbag was likely giving her favoritism in the workplace that should have gone to hardworking young men. Not to mention, cheating on his wife. He should be imprisoned.

our society is so pathetic. two adult coworkers consensually making out in public had their business blasted on tiktok for millions of people to see, and now one of them is placed on a leave from work? absolutely insane.

Do you have 9 acres? I have a plan you will love. One scenario is 1,220 units on floors 2-6. 122 studios, 305 1BDs, 488 2BDs, 244 3BDs, and 61 4BDs. Plus about 177,000 sf of ground floor for whatever.

@deorcmona @SombraMapache And everyone else has the right to ignore their opinions.

@SombraMapache People who lost their families/ loved ones in this attack have the right to be racist.

Call me woke or whatever, but I don't think it's even vaguely reasonable to expect the literal mayor of NYC to be prohibited from attending a 9/11 memorial just because a handful of folks affected by it grew up to be malignant racists.

Message to patriots who think this is done & dusted. It's not. Imagine 40% turnout. That's what we had for the standalone equalization referendum in 2022. The 10% or so of the electorate who are hard core separatists will crawl over broken glass to vote. Not hard to imagine another 10% who are frustrated federalists showing up to cast a "send a message" vote. Then you've suddenly got 20% of the electorate winning, and pushing us into wood chipper for the next year. Don't let it happen. Get involved in @VotetoStayAB or another group that supports a strong Alberta within Canada. And don't take for granted the overwhelming support for national unity. It only matters if the silent super majority show up to vote.

@lifemaxed @sudoingX Here's the neat part, with open weight you don't need to trust anyone and nobody can gatekeep anything. My local funded community library can easily run a decent enough server farm and anyone with a digital library card can use it. The only limitation is your imagination.

@MarrrkLoki @sudoingX Thats a cool idea but having to trust liberal government Dems gatekeeping and tracking that local inference is less appetizing, maybe more of a co-working space would be better.

dear open labs, and i mean this with love. you keep dropping these trillion parameter giants and they're genuinely beautiful and i can't run a single one of them, none of us can, they land on huggingface to a standing ovation and then they just sit there, because the only machines that can load them are the same datacenters we were all trying to get free of. i watched local ai turn into something real this year, people running 27b models on one gaming card, an 8gb board doing work that needed a server a year ago, whole little communities forming around whatever fits on the hardware already sitting on the desk, and that's the part that actually reaches people, the revolution nobody's putting on a keynote slide. so here's the fair ask, give us the 40b dense, give us the 120b moe, the sizes a person with one good card or a 128gb box at home can actually hold and run and learn from and build on, because not everything has to be shaped for an enterprise with a rack to spare. the frontier is thrilling and i'll keep cheering every open weight you ship, i mean that, but the ground is where most of us live and the ground is quietly running out of models it can lift. build for the hardware we already own. that's where the next wave comes from, it always was. with love, from someone running yours on a laptop at midnight.

@jamiljivani Maple MAGA is triggered

If Carney wants to be the leader of the Democratic Party, he should move there, and add a US passport to his collection. Canadians deserve a Prime Minister who will put Canada first. Sign here if you agree: forms.gle/kkxSADCbEmQYdr…

@aye_pete @jacobeverly You can't cheat to create a front tier AI. Distillation doesn't work that way. That's why only the most gullible people believe the lies from anthropic and the Trump admin, and all the researchers in the field push back against this.

@jacobeverly Open weights are good, by the way. But you can't host them on normal infra. Cheating just kills the incentive to put in the work to drive innovation as fast as it is going right now.

The obsession with distillation is cope. It may accelerate post-training, but it does not copy weights or eliminate the work of building and training a frontier model. Open-weight models are coming, and more competition is normally good. If the US wants to lead, it cannot regulate its way out of competing. It needs to build: more power generation, more grid capacity, more compute, and better models faster than ever before. Unleash the American Dynamism 🇺🇸

@aye_pete @jacobeverly "why do they expend so much effort and resource to do it" You automatically assume and believe the premises where there's no evidence presented to back it up. His argument on the other hand is consensus within the AI field and researchers. At least look up how distillation works

Bro, your argument is dumb. If it's so negligible, why do they expend so much effort and resource to do it? Why not just train their own Chinese models and compete directly instead of distilling US Frontier models? Oh, right. They were left far behind and the only way to catch up was to cheat.

@earn35tguy @sudoingX @mkratsios47 Distillation doesn't work that way. Model can't gain real capability from learning the style of how other models repond to the problems. Exhibit 1: Every Qwen community fine-tune variants with high quality Opus distillation always regress in benchmark comparing to the base.

@sudoingX @mkratsios47 You have a point but missed the point here. The point was, K3 took a shortcut, and was not a first class model trained from scratch. Orders of magnitude less work

We have information that Moonshot AI distilled Anthropic’s Fable for the development of its K3 model. To do this they developed a sophisticated internal platform to conduct large scale distillation against U.S. models, allowing them to quickly switch between multiple methods of access to avoid detection. Moonshot AI has also acquired GB300-equipped servers and has accessed GB300s in Thailand, likely to train its AI models.   The United States strongly supports the free and fair development of AI, including a thriving competitive ecosystem that spans frontier models, specialized systems, open-source frameworks, and open-weight models. Legitimate AI distillation used to create smaller, more efficient models plays a vital role in this open innovation ecosystem. However, large-scale, covert industrial distillation aimed at stealing proprietary U.S. technology and undermining American research is unacceptable.

@LonghornTXFan89 @Teknium @mkratsios47 And even so, there are tons of evidence that anthropic distilled Qwen and you can still reproduce that today by asking Claude what model is it in Chinese, and it'd say it's Qwen. So the entire distillation argument they use is dishonest and shameless to the core.

@LonghornTXFan89 @Teknium @mkratsios47 In common sense, IP is in the algorithm, architecture, training methods, etc. No serious person in AI field would consider the output of an AI model as IP.

@mkratsios47 stop this bro lol

Most goated nerd

@SarahKHeck @mkratsios47 You guys gonna lose so bad and that's a good thing for humanity

Thanks @mkratsios47 for speaking out on this important issue. Illicit, adversarial distillation is IP theft and industrial espionage that supports adversary military and intelligence capabilities. It is a national challenge that creates serious national security risks for the United States and democratic allies. We'll continue to crack down on it and work with the Administration and Congress to maintain American AI leadership. 🇺🇸

@pmddomingos @Dan_Jeffries1 Classic motte and bailey

@Dan_Jeffries1 No one is saying Kimi is 100% distillation, or that fast effective distillation is trivial, even when building on distillation of previous models. The point is Kimi wouldn't be where it is without it, and the whole narrative that it's honestly surpassed Fable etc. is BS.

@Dan_Jeffries1 It was available for much longer than that, and they had plenty of time to distill Opus before. Also, stick to what’s relevant.

Now you see how he bankrupted casinos.

The Trump administration is considering an executive order, and other means, to ban Chinese open-source models within in the United States. Kimi K3 has reignited this debate. Reporting this morning by Axios. Commerce is also considering adding Chinese AI labs to the Entity List.

> be Kimi Founder > Zhilin Yang. nobody in the West knows your name. > build K1. K2. K2.5. K2.6. K2.7. > each one sets the open-source record > everyone says China can't reach the frontier > raise $2B. hit $20B valuation. > keep your head down. keep shipping. > July 16. drop K3. > 2.8 trillion parameters. > the biggest open model ever released. > 1 million token context. native multimodal. > Kimi Delta Attention — 6.3x faster decoding. > Artificial Analysis ranks it #4 overall. > right behind GPT-5.6 Sol and Fable 5. > open weights July 27. free for anyone. > the closed labs charge for the frontier. > you gave it away. > different game.