Espen JD

Repo for Hybrid NVFP4 on SM120 is up on GitHub now (with more graphs and data as well) github.com/net-snix/vllm-…

😕

To my friends in the local AI space, I have some good news. After advocating on here for better hardware support for sm120 (i.e. RTX 6000 Pro Blackwell cards) and sm121 (DGX Spark), @nvidiaai reached out to setup a call. That call happened yesterday. I spoke with a large group of people from various teams about the problems we've experienced with new model and inference engine support. How much that degrades our UX and what can be done to improve it. The company seems very receptive and eager to improve support. We spent about 45 minutes talking about the next steps and I walked away feeling more confident than ever that @NVIDIA will be giving us first class support moving forward. Obviously there will be some time from this being an initiative to becoming a reality, but it is now in the works. I want to thank the team at Nvidia for their time and attention. Never in my wildest dreams did I expect a few social media posts to galvanize this kind of attention and interest. I also want to thank all the people in the community that I reached out to for feedback and those that supported my posts on this matter. You were heard. We are making progress. All of this to say, this is what happens when we come together as a community. We can make great things happen. I appreciate you all! I love the local AI community and all that we're building together. Onto the next!

@Blackwellboy Wish you guys all the best

Two tough days, Package turns up empty with no spark, police reports etc. today, my son jumped over the couch, smacked his head on the tiles and knocked himself out, now in hospital for the past several hours, pray he will be ok.

@LukaTheFounder Half of the time? I don’t get notifications at all

Does anyone else think X messages are the most unreliable feature on this app. Half the time i don't get notifications Anyone else experience this?

@TechMDAI Is the cat essential to the setup?

I’m loving this DGX Spark set up. I need a switch from MikroTik

@thsottiaux Can we just keep it this way, 5h limit is stupid

Morning. The last 48 hours of Codex and ChatGPT Work have been intense! Three important updates: - Temporarily removing the 5 hour usage limit restriction for all Plus, Business and Pro plans - Rolling out changes that will make GPT 5.6 Sol more efficient across the board and that will be reflected in less usage being used so that it can take you further. Exact impact to be quantified and shared - We hit 6M active users, and are landing a usage reset in the next hour Go do things

We're extending Claude Fable 5 access on all paid plans, as well as keeping Claude Code’s weekly rate limits 50% higher, through July 19.

Noooo, they removed Ultra from the slider I loved the effect when it was selected

@TechMDAI 🔥🔥

@Snixtp Ya getting new fans and aio

My main rig is down due to current upgrades and maintenance. Currently very hot and need to improve my temps on the rig with heavy workloads. It may take a while before it gets back online. So only sparks for a while. I feel fine.

@DJLougen 😭

Would anyone be interested in a smart bidet that is connected to chatgpt and it talks to you about your poop

@AdamHoltererer nice setup bro

Desk upgrade! I got a standing desk, so I'm legally obligated to tell you that I have a standing desk.

@DJLougen ollama run qwen-3-6-garbage

@Snixtp Whats the cli command for that? I need to copy paste it

FABLE + SOL at home officially is what the people are saying

@DJLougen @luksamuk @0xSero ill fight with you

@luksamuk @0xSero @Snixtp

@0xSero yeah there is a reason they arent big dog

We’re releasing new Qwen3.6 quants that run 2.5× faster on your GPU. Qwen3.6-27B NVFP4 runs on 24GB VRAM. 35B-A3B can hit 17,561 tok/s (B200). We also improved accuracy, tool calling, agent use, and looping. Guide: #nvfp4" target="_blank" rel="nofollow noopener">unsloth.ai/docs/models/qw… Qwen3.6 NVFP4: huggingface.co/collections/un…

@TechMDAI I feel dumber lol :( AI keeps doing more and more

Fire week for AI. We all got smarter as a result.

@peterk x.com/Snixtp/status/…

@Snixtp nice find, did you post a vllm issue or PR for this? im curious to try it

The RTX Pro 6000 has two matmul engines, and vLLM was only using one of them when I ran Qwen3.6 27B NVFP4 Marlin is nice for decoding, but CUTLASS is 3.5x faster processing large token prompts. For some reason, vLLM ran Marlin for both, because NVIDIA's checkpoint files mislabel what the model can actually do. Fable 5 found out about all this and wrote a vLLM plugin that looks at the batch size and picks the right engine per call. Results is almost 2x increase in prefill at all context lengths:

@HHHVIHHH @chribjel @slopshipper we can thank the EEA

@chribjel @slopshipper You will have to buy Starlink and VPN soon anyway at the rate EU fuck things up in Europe AND Norway, even though we're not a member.

Please make Grok 4.5 available in Norway. We are not part of the EU 🙏🙏