Azeez

@sudoingX We JUST got our compute as @AtlasInference. Just give us a teeny bit more time, we'll have some great numbers for you :)

actually, let's not just wait. if you are running local models on AMD right now, R9700, Strix Halo, a 7900 XTX, any RDNA card, on a ROCm or Vulkan build, drop your numbers in the replies. model, quant, the card, tok/s. one line is enough. i'll pull the best into a proper thread and amplify the builders who contribute. consider it the AMD half of the list, written by the people actually running it. real numbers from real cards. let's build the AMD picture together while my hardware ships.

@MFloresFive @Tnimbus @spark_arena We have a dedicated #openclaw channel for our discord, with many users having success. Join and ask away, we'll get to the bottom of it!

I'm trying to combine this with either openclaw or hermes. when i do that i get a lot of tool call errors. It tries to use bash instead of exec, atlas says tool call rejected: error: unknown 'path', same for 'pattern' etc. any ideas on how to fix this? I am using this model: Qwen3.5 Qwen3.5-35B-A3B Sehyo/Qwen3.5-35B-A3B-NVFP4 35B / 3B GDN + attention + MoE, MTP

DGX Spark just benched 200+ tok/s for Qwen3.6-35B with @AtlasInference on @spark_arena 🔥 How's that possible? Providers like Codex and Claude get ~60. Other major engines don't come close 🦥 We haven't seen speeds like this on GB10. NO ONE HAS. Atlas is shattering records 🚀

We're way faster than that. MTP-enabled with Dflash rolling out, check out the discord thread for more details!

@AtlasInference @AMD I would really like to know what the t/s is for Qwen 3.6 27B Q8 with @AtlasInference Im running it know on llama.cpp and only getting 11 t/s. Same for vLLM. I have a Asus Ascent GX10 so basically the same as the DGX Spark.

🚀 Huge thanks to @AMD for sending @AtlasInference a Strix Halo laptop! Excited to squeeze every last drop of compute out of it. Our goal is staying community-first with the simplest stack possible, ROCm here we come 🔥 Join our Discord for early access and help shape what we build next. What should we tackle first? 👇

@LottoLabs Beautiful, let us know if we can help help contribute to the benchmarks in any way too. Will make a separate post on this soon :)

Localmaxxing now supports infra benchmarks from @AtlasInference Looks like an interesting rust + cuda engine Use today with DGX Spark

@SW_Quigley @spark_arena This is without dflash too... stay tuned!

@AtlasInference @spark_arena Best I've had is 100 TPS out of vLLM. Will give this a try!

@lucatac0 @spark_arena You'll be one bite into your lunch and it'll be ready to serve. We love to brag about our cold start times <2 mins for models like Qwen3.6-35B-A3B!

@AtlasInference @spark_arena bookmarked to try out during my lunch break!

@JJJOOOHN @spark_arena Coming to SparkArena soon. MTP just confirmed working, with DFlash it's going to be lightspeed ⚡️

@AtlasInference @spark_arena How about 27B? How many token per second at decoding

@bridgemindai Try Atlas Inference and you will know why. With current local model capability and the speeds we're able to achieve, you can absolutely fly through your work.

I have two NVIDIA DGX Sparks stacked in my office. They've been sitting there for a month. Here's my honest take. Open source AI is never going to compare to frontier models. Running quantized Kimi K2.6 and GLM 5.1 locally is cool. But practical? No. Not even close. I run all my Hermes agents on GPT 5.5 through my ChatGPT Pro subscription. Practically free. GPT 5.5 is the intelligent model in the world. Why would I route serious tasks to a watered down local model? If you need fast and accurate, you're not using local inference. You're using GPT 5.5 or Claude Opus 4.7. I'm not saying this to rage bait. I genuinely want to know. Why would anyone serious about vibe coding and AI agents use a local model when frontier is this far ahead?

We, Atlas Inference and our absurdly talented community, are just getting started @NVIDIAAI😉 These numbers represent the floor, not the ceiling. The DGX Spark has much more in the tank, we're on a mission to keeping pushing GB10 🔨 New optimizations grounded in academic literature that aren't shipped yet. New tricks nobody's tried 📚 If you've got a Spark and you want in, JOIN US. We're open source. Builders welcome. Skeptics welcome. We'll convert you 🗣️ Records are made to be broken🔥

@AtlasInference @spark_arena 🔥

@RisingSayak Done! Added more context to the issue thread as well. github.com/huggingface/ke… Feel free to reach out if you need further details on our end :)

@AtlasInference Yes, please. Open an issue thread and we can take it from there.

The kernels project at Hugging Face has been growing! We want it to be the go-to place for kernel devs and kernel users. We're looking to work w/ folks who're interested in doing agentic kernel dev, providing real optim value to real models. Reach out if interested :)

@Tnimbus @spark_arena Should work well, we have a dedicated thread with Atlas Openclaw users on our discord as well :)

@AtlasInference @spark_arena Will it work on openclaw? I will try!

@loktar00 Believe it or not, it's what a DGX Spark can do too. Atlas inference makes it possible😏

Crazy that 200tps is slowly becoming the floor for local inference... a year ago that was top tier API speed, now its what a single 5090 or dual 3090 with MTP can get you for "free" ™

Right. That recipe should work, if you don't have sparkrun here's the docker command for nvfp4 docker pull avarok/atlas-gb10:latest sudo docker run -d --name atlas \ --network host --gpus all --ipc=host \ -v ~/.cache/huggingface:/root/.cache/huggingface \ avarok/atlas-gb10:latest \ serve RedHatAI/Qwen3.6-35B-A3B-NVFP4 \ --port 8888 \ --max-seq-len 131072 \ --kv-cache-dtype fp8 \ --gpu-memory-utilization 0.88 \ --scheduling-policy slai \ --tool-call-parser qwen3_coder \ --enable-prefix-caching \ --speculative

@AtlasInference @spark_arena I saw them, but the benchmark was with nvfp4

Docker commands are on the website atlasinference.io for FP8. Some light modifications to make it NVFP4 via the recipe in github.com/Avarok-Cyberse… docker pull avarok/atlas-gb10:latest sudo docker run -d --name atlas \ --network host --gpus all --ipc=host \ -v ~/.cache/huggingface:/root/.cache/huggingface \ avarok/atlas-gb10:latest \ serve Qwen/Qwen3.6-35B-A3B-FP8 \ --port 8888 \ --max-seq-len 65536 \ --kv-cache-dtype fp8 \ --kv-high-precision-layers auto \ --gpu-memory-utilization 0.90 \ --scheduling-policy slai \ --tool-call-parser qwen3_coder \ --enable-prefix-caching \ --speculative

@AtlasInference @spark_arena wow. What is the serve syntax for this speed?

@iotcoi @spark_arena Right, should have provided the run command. This exact recipe cold starts in <2mins. Not only is it the fastest, but the startup times are also unprecedented :) sparkrun run @atlas/qwen3.6-35b-a3b-nvfp4-atlas

@spark_arena Check out our Github: github.com/Avarok-Cyberse… Join our discord too! discord.com/invite/DwF3brB… We're bringing the best speeds FOR the community. AMD support is coming 🔜

@TeksEdge @RayFernando1337 I'm having trouble understanding this. Point to point communication should be slower right? All-reduce is slow for TP sure, but not all of the inter-node data transfers... is it because the intra-node speeds aren't that fast itself?

@RayFernando1337 Did you see this? You get better than 2x on 2 sparks

I might have to get a second DGX Spark!