Post

Sudo su
Sudo su@sudoingX·
i'll put this here!. every "the dgx spark is slow" take benchmarks it the exact same wrong way: one request at a time. single stream, sure, laguna s 2.1 does a modest 19 tokens a second on it, and people screenshot that number and call the box a dud. but nobody serves one request at a time. that's not what this machine is for. run it the way it's actually meant to run, under load, and the story flips. i pushed it to 16 concurrent streams and it scaled almost perfectly linear, 35 tok/s at 2, 58 at 4, 88 at 8, 140 at 16, the chart is the whole thing. that's the part the spec sheet hides. the spark was never a single-stream sprinter, it was built as a serving box, big memory to hold a model nothing else on a desk can hold, and the throughput to serve a whole team off it at once. judge it on the drag race and it looks slow, judge it on what it's for and it's doing exactly what it was built to do. one honest catch, since i measure the ugly numbers too: the dflash drafter that makes single user feel fast actually hurts you at batch, the wasted draft compute steals from real requests, so you run spec-decode for one user and turn it off when you're serving many. same box, two modes, pick the one that matches the job. the "spark is slow" take isn't wrong because the number is fake, it's wrong because it's measuring the wrong thing.
Sudo su tweet media
Sudo su@sudoingX

i didn't expect the american open weight comeback to actually deliver the thing builders kept asking for. for a year now "open weights" has meant chinese labs, full stop, while the american side went quiet, and every time the west did drop something "open" it was late, or a toy, or it needed a rack of h100s to breathe. builders kept asking for one boring thing the whole time: an open model actually good enough to matter, that i can run myself. on hardware i own. poolside's laguna s 2.1 is the first american drop that just answers it, and i didn't take the launch chart's word for it, i put the whole thing on one dgx spark and measured every number. it's a 118b mixture of experts, 8.5b active per token, open weights under a real license, and agentic coding is the entire point of it. i served it on vllm with the nvfp4 build and the dflash drafter, and the part nobody publishes is what it actually does on your own metal. it fits. 67 gigs of weights on ram, and because only 12 of its 48 layers run full attention, the entire million token context is about 26 gigs of kv, so the full window lands near 100 of 128 gigs with room to spare. a frontier coding model, its whole context, on one box that sits next to a monitor. speed is honest anon. single stream it's modest, about 19 tokens a second, this box was never a sprinter. but the dflash drafter climbs it to 25-30 on normal chat and 45 sustained on code, the workload it's built for, and under load it does 140 tokens a second across 16 streams, because the spark was always a serving box, not a single-stream drag race. it even holds at depth, barely fading from 19 to 14 out at 230k of context while free memory never moves. that's the comeback that actually matters. not a press release, not a leaderboard screenshot, an open model that shows up with what people kept asking for and holds up when you measure it but still america didn't win open source back, china still sets the pace. but this is the first one that landed the ask.

English
14
5
92
9.2K
joseph
joseph@gitcommit90·
i see almost exactly -50% of this. can you explain specifically what you’re serving? my benchmarks; 1 @ 17 tok/s 4 @ 35 tok/s aggregate (drops to 9ts per stream) 10 @ 70 tok/s aggregate (worse per stream performance at just 7tps) Also curiously - where would 10 concurrent requests be useful if each individually is decoding at less than 10 tok/s
English
0
0
1
40
Paylaş