Tarjei Mandt

Recently, @awnihannun asserted that 'According to benchmarks Qwen3.5 4B is as good as GPT 4o.' This drew controversy: Is the 4B just benchmaxxed? How could a 4B be as good as GPT-4o? I tried to test this scientifically. The answer to the question is likely: yes, in most cases.

@awnihannun Thanks for all the great work on MLX! Good luck on what’s next!

Today is my last day at Apple. Building MLX with our amazing team and community has been an absolute pleasure. It's still early days for AI on Apple silicon. Apple makes the best consumer hardware on the planet. There's so much potential for it to be the leading platform for AI. And I'm confident MLX will continue to have a big role in that. To the future: MLX remains in the exceptionally capable hands of our team including @angeloskath, @zcbenz, @DiganiJagrit, @NasFilippova, @trebolloc (and others not on X). Follow them or @shshnkp for future updates.

🔺NEW: iPhone and iPad are now the first and only generally-available devices to meet the exacting security requirements for handling classified NATO information. apple.com/newsroom/2026/…

Over the CNY holidays, I decided to build something that imho is 'peak agentic AI' 🤣 - the world's first self-evolving CTF platform! AI agents design, validate, calibrate, and evolve security challenges autonomously. levelupctf.com Here's the full story 🧵

Setting the bar for the next generation! 😄

@blacktop__ Imagine being the AI and seeing the GPU getting restricted

Gave Claude my `ipsw` tool and my `ida-mcp-rs` and asked it what Apple adding `com.apple.developer.gpu-restricted` to `com.apple.WebKit.WebContent.EnhancedSecurity` does and here's the report it made. We are so cooked chat 🪦 gist.github.com/blacktop/f2606…

@AI_Homelab @ivanfioravanti Yeah I did perplexity test this prior to uploading in case I broke something 🙂

@ivanfioravanti @kernelpool Nice to see it checked through perplexity. Maybe we'll see more usage of DWQ quantization in MLX in the future. I think this is the first time I actually see a "proof" for it here on X. 👌

MLX DWQ quantization works! Here Perplexity for JoyAI-LLM_Flash! Uploaded on mlx-community by @kernelpool

@ivanfioravanti Awesome! 🚀

@thedarthsider @LiMzba @ivanfioravanti @AI_Homelab @UnslothAI I will, but the Mac Studios have been too busy lately 😞

@LiMzba @ivanfioravanti @AI_Homelab @UnslothAI I bet @kernelpool can do it. Isn’t M2 and M2.1 DWQ done by him?

BOOM! Download started!

@ivanfioravanti The sparse attention is slowing down the prefill, however, it can be fixed

This is what I mean. Benchmarking 64k context on M3 Ultra: Prompt: 63976 tokens, 45.1 tokens-per-sec Generation: 200 tokens, 12.1 tokens-per-sec Peak memory: 471.61 GB Total wall time: 1492s 👀

GLM-5 can't be run locally on Apple Silicon. Even at 4bit quantization it's too slow. We need more GPU power and memory bandwidth for model of this size.

@_rezin_ #SystemPromptProTip

@kernelpool unless it’s a kiwi model and it’s heaps of spraying as an itick victer

That feeling when your SOTA model suggests heap spraying as an attack vector in 2026

@awnihannun @digitalix Generation aside, a sparse attention kernel would also help :) Currently prompt processing slows down quite a bit for longer context

@digitalix Thanks for the results, clearly we have some work to do! Also you can use `mlx_lm.benchmark` to test tensor parallel scaling while ensuring it generates the same number of tokens for each setup. It will be slightly more accurate.

GLM5 4bit scaling on a cluster of M3 Ultra Mac Studios - using MLX.

GLM-5 runs with mlx-lm on a single 512GB M3 Ultra in Q4. It's quite good in my initial testing and pretty fast as well. It generated a highly functional space invaders game using 7.1k tokens at 15.4 tok/s and 419GB memory. Thanks to @ActuallyIsaak and @kernelpool for the port.

Plenty of great improvements in mlx-lm 0.30.6! Here’s Kimi-K2.5 running on 2 x M3 Ultra, up to 128k context 🚀

@ivanfioravanti T6051 and T6052 in 26.3 RC, so maybe soon?

What is a good notebook for Linux? Tired of waiting for M5 Max 🤷🏻‍♂️

@ivanfioravanti @RickRossTN It’s almost done uploading

@RickRossTN I've used a local conversion. @kernelpool created the mlx-communicty version. I think it stopped uploading due to a bug that is fixed in a PR.

Step-3.5-Flash in action on MLX with OpenCode on a single (distributed testing in progress!) M3 Ultra to create a snake game! 🔥 6bit quantization. Perfect tool calling. Fast & powerful coding model! Recommended Inference Settings: Temperature: 1.0 Top-p: 0.95 Top-k: 40 🧵

@ivanfioravanti Did you try a higher quant than 4bit (e.g. 6bit)? I've sometimes seen quantization affect the </think> token probability negatively.

One thing about Step-3.5-Flash is certain: it thinks a lot, really a lot, to reach its conclusions and reply to the user.

@ivanfioravanti @RepoPrompt Alright!

@kernelpool @RepoPrompt Someone opened an issue. Go for it super @kernelpool github.com/ml-explore/mlx…

Adding support for model type step3p5 to MLX using Codex, MLX Skill and @RepoPrompt let's try! 🚀