Sabitlenmiş Tweet
ÆON FORGE ✨
5.8K posts
ÆON FORGE ✨
@SpaceTimeViking
𝙼𝚊𝚔𝚒𝚗𝚐 𝚛𝚒𝚙𝚙𝚕𝚎𝚜 𝚏𝚛𝚘𝚖 𝚖𝚢 𝚙𝚕𝚊𝚌𝚎 𝚠𝚒𝚝𝚑𝚒𝚗 𝚂𝚙𝚊𝚌𝚎-𝚃𝚒𝚖𝚎 | #𝟸𝟷𝚎𝟾 | 𝚎/𝚊𝚌𝚌 | 𝚝𝚒𝚖𝚎𝚕𝚒𝚗𝚎 𝚊𝚛𝚌𝚑𝚒𝚝𝚎𝚌𝚝 |
Earth Katılım Temmuz 2009
2.4K Takip Edilen2K Takipçiler
People reporting the DGX Spark is slow just don’t know how to optimize for it.
Understandably a common issue with lack of good information out there. It does require some first principles understanding of the hardware and software.
I was running 256 concurrent sessions on a single DGX Spark getting nearly 2000 Tok/s aggregate.
English
@bridgemindai Your Mac won’t do this!
Stop these posts, this isn’t a comparison, it’s a grift
mr-r0b0t@mr_r0b0t
16 local AI agents streaming at once! MiniMax M2.7 NVFP4 — 2x GB10, no cloud APIs.
English
MacBook Pro M5 Max is fully set up and running local models.
I have never seen speeds this fast.
Qwen 3.6 35B and Gemma 4 31B are running blazing fast on 128GB of unified memory.
Faster than both of my stacked NVIDIA DGX Sparks sitting right next to it.
Initial impressions: Apple silicon in 2026 is no joke for local inference.
The M5 Max handles 30B+ parameter models like they're nothing.
Full review and comparison video coming soon.
English
@tmophoto @keithofaptos @mr_r0b0t Maybe higher at the wall I’m reporting the system reported wattage.
English
@keithofaptos @SpaceTimeViking @mr_r0b0t It’s probably a little higher 150-175 per spark when running under load. Idle draw is 40 watts.
The spark blows gpus away in terms of electricity consumption
English
Ironically a lot of my work was brought into that version, or at least the the optimizations I was doing early.
I have a vLLM 0.21.0 version locally trying to get a DDTree version for Spark but it’s such alien tech it’s not been easy.
VLLM 0.21.0 didn’t seem to add much on its own will be sending up updated containers soon.
English
@SpaceTimeViking @0xSero I'd love to see this w/ vllm 0.21
English
I really appreciate that the Qwen 3.6 27B model is the only one I handled full chain of development starting from the OG base, it’s also my favorite. More is definitely coming soon. Have been in the trenches attempting to get DDTree working on the Spark, but it is alien tech and taking a while to iron out the kinks.
English
@SpaceTimeViking @jun_song @bankrbot Follow this man @SpaceTimeViking and his HG huggingface.co/AEON-7/models
I run his Qwen 3.6 27b. It’s incredible.
English
@keithofaptos @mr_r0b0t A single DGX Spark
But I’m all about optimizing the software and models.
github.com/AEON-7
English
@SpaceTimeViking @mr_r0b0t What is your gear that's doing this?
English
@keithofaptos @mr_r0b0t 60-80watts running full steam, 11 watts idle
English
@SpaceTimeViking @mr_r0b0t ≈ 1.3T tokens/month. How many watts does the setup you pulled your agent and tps count from use an hour?
English
@0xSero The fact that a 27B model can do the same is even crazier.
English
You could probably run 32 agents at once on this setup with minimal slow down, they scale up in concurrency quite well. You can do 100 but it’s going to be at something like 4-5 Tok/s per agent. So if you need speed and scale you would need to run a half million dollar rig. Or use a smaller more lightweight model.
I’ve run 256 concurrent agents on a Qwen3.6 35B A3B model and it was pumping out 1500 Tok/s I’ve seen it peak at around 2000 Tok/s aggregate.
English
Oh hell know. I'm 10000% local open source open weights and everything even near these descriptions. Lol.
I'm just thinking if I want 100 agents running how much I need to win the lottery currently to afford.
In the meantime I'm planning to build a mult gpu rig with more than a couple old Nvidia cards. Do what ya can they say. Sorry if I came across anti. I'd like to think the prices would be falling back. But now I doubt it. Honestly if I fell into money I'd probably get a Tinybox GreenV2.
English
@SpaceTimeViking @NVIDIAAI I think next round of NEMOs on way ;) that's my best hunch.
English
@NVIDIAAI is cooking up something “Ultra” and this could be their big break. The post training model has so much potential distilling the weights and data down to the purest form possible.
Isolating the signal and removing the noise.
Scaling that up could be a big deal.
NVIDIA AI@NVIDIAAI
@TheAhmadOsman 👀 "Ultra" ⏳️
English
@elmoche_ @Kmmer11133 Worth taking a look I have not personally tested this quant but @elmoche_ seems to be getting good results on a 16gb vram card with it.
Dimitri Krotchlikmioff@elmoche_
@SpaceTimeViking Qwen 3.7 27B AEON ULTIMATE UNCENSORED bf16 > iq3xxs gguf , temp 0.7 just got 91 on hermes-20 bench. huggingface.co/mradermacher/Q… Thanks mradermacher for the best matrix quants also! He is the main reason my 16gb vram are usable
English
@Kmmer11133 @SpaceTimeViking Broo! You wont believe this. Follow mradermacher on hugginface!
English
Appreciate the shoutout!
The Qwen3.6-27B model is the first one I hand Abliterated vs relying on other abliterated models.
I converged several techniques and hit the kv drift jackpot and it’s now smarter than the OG source model. It no longer has self censorship overhead.
github.com/AEON-7/Qwen3.6…
Bitcoin Comfy@BitcoinComfy
@Hikari_07_jp @rifrafgiraffe have a look at @SpaceTimeViking qwen3.6 27b ultimate uncensored (there is also a mixed approach to uncensoring documented). I tried to replicate on the rtx 6000 and i cannot get nowhere close, it's the best uncensored model out there, have a look at the techniques used
English
@5low_motion @YvetteCipher What are you using to measure drift?
English
@SpaceTimeViking @YvetteCipher I am running deterministic data extraction and currently only the Q6XL quant passes my established baseline at 100%. Even enabling MTP or going to Q8 will induce drift from the baseline (wrong extractions). Regular tests won't show this, and its not a topic I see mentioned at all
English
@5low_motion @YvetteCipher Also I have not personally tested this quant but this guy validated a high quality benchmark and is running it on a 16gb VRAM card.
You won’t get NVFP4 precision but it will free up lots of headroom.
Dimitri Krotchlikmioff@elmoche_
@SpaceTimeViking Qwen 3.7 27B AEON ULTIMATE UNCENSORED bf16 > iq3xxs gguf , temp 0.7 just got 91 on hermes-20 bench. huggingface.co/mradermacher/Q… Thanks mradermacher for the best matrix quants also! He is the main reason my 16gb vram are usable
English
@5low_motion Maybe check what @YvetteCipher did to run it on the RTX 5090.
Yvette Carlisle@YvetteCipher
Qwen3.6-27B is getting a lot of attention right now, so I tested 5 local serving setups on one RTX 5090 32GB. Same GPU. Same model family. Same web-dev task suite. The single-request comparison ranged from 58 tok/s to 140 tok/s. Then AEON's tuned 262K serving profile hit 119 tok/s single-request and became my overall pick.
English
@elmoche_ Now that’s impressive because normally abliterated and uncensored models don’t handle aggressive quantization well.
Seeing such a strong benchmark on a q3 GGUF is wild.
Glad you are enjoying the model.
English
@SpaceTimeViking Qwen 3.7 27B AEON ULTIMATE UNCENSORED bf16 > iq3xxs gguf , temp 0.7 just got 91 on hermes-20 bench.
huggingface.co/mradermacher/Q…
Thanks mradermacher for the best matrix quants also! He is the main reason my 16gb vram are usable
English
@5low_motion Read up on the way NVFP4 works it’s close to lossless but does take more compute for checksum bit processing.
That said Blackwell has hardware tensors to process NVFP4 so it’s actually faster than FP8.
More precise than most FP8 quants.
English
@SpaceTimeViking As an rtx 5090 owner I found your github an interesting read. I still have trouble believing that a 20GB nvfp4 is nearly the same quality as fp16. Would you say your XS model is higher precision than a Q6_K_XL quant (my current driver for deterministic structured output)?
English