marco

@yume_arasaki Why do you advise to use Nemotron-3-Super 120B on the rtx pro 6000 blackwell instead of qwen 3.6 27B NVFP4 which is better quality and faster?

Everyone's "local ai gateway drug" is the Qwen 3.6 27B dense. It was mine. The model that made local AI real. Fits on a 24GB card, 40 tok/s, flagship-level coding. It's the starting point. Afterwards the questions becomes "what's next"? People recommend the 35B-A3B MoE as "the next step." It's not. It's 4x faster but hallucinates in agent loops. Tool call failures at 18-76% rates. Context hallucinations at 37% usage. Qwen's own benchmarks show 27B beating it on agentic coding. For agents, it's a downgrade. The real upgrade paths depend on which constraint you're breaking: speed, quality, model size, or sovereignty. It depends largely on how much VRAM you want to play with imo Here's the map. - -- PATH 1: Same card, enable MTP (free) The upgrade most people miss. Unsloth shipped MTP GGUFs. Multi-token prediction. The draft head predicts 2 tokens in parallel, main model verifies all 3 in one forward pass. 83% acceptance rate. Your 27B at Q4 running 40 tok/s? With MTP it hits 60-80 tok/s. Same model, same card, same quality. On an RTX 6000 Ada it hits ~160 tok/s. Cost: $0. Download the MTP GGUF, add one flag. --- PATH 2: RTX 5090, 32GB Blackwell (~$4-5K) 32GB GDDR7 at 1,792 GB/s. Same bandwidth class as the $12K RTX PRO 6000. Native NVFP4 tensor cores. What changes: 27B at Q8 fits cleanly with room for long context. NVFP4 quants run 2.5x faster than Q4_K_M on the same silicon. sudoingX benched the 27B on a 5090 laptop: 35.3 tok/s generation, 1,509 tok/s prefill. The prefill is where Blackwell pulls ahead. This is the supposed to be the cheapest card that runs the newest quant format., but it's not, supply chain means you are unlikely to find one at the RSP of $2.5k . Cost: ~$4000-5,000 - Probably not worth it IMO, at its retail price of $2.5 it’s more interesting. Supply issues pushing the card price up makes it unatttractive -- PATH 3: Strix Halo / AI Max+ 395, 128GB unified (~$2.5K) The budget Spark alternative. Geforce VS AMD all over again lol. AMD cheaper but…it’s a huge underperformer without NVIDIA’s efficiency gains due to large community on NVIDIA. AMD's Strix Halo gives you 128GB unified at roughly $2K. Stepfun officially lists it as a Step 3.7 Flash target. Less bandwidth than the Spark. No NVFP4 tensor cores. But half the price for the same memory capacity. If you want to run 198B models and can't afford a Spark, maybe this is a choice Cost: ~$2,500 --- To be honest, it’s not worth paying less for a less serviced ecosystem, a DGX spark is a much better buy. --- PATH 4: DGX Spark, 128GB (~$4,699) Not a dense model speed play, this is where the DGX Spark is weak. Where DGX sparks excel is running MOE models, here it kinda becomes like a KING. It's not a good GPU in terms of memory bandwidth (something dense models need) 273 GB/s bandwidth vs your 3090's 936. Unsloth's NVFP4 quants (released July 10) halve bytes per parameter on Blackwell, which narrows the gap. The Spark IS Blackwell. What it's good at: running models too big for any consumer GPU, and serving many users at once. The trick with it is having mixture of experts. Speed runs DSF4 (easily basically the MOST DIRECT “next step up” from Qwen 27B Dense. Step 3.7 Flash (198B MoE, 11B active): sudoingX's top Spark pick. 198 billion parameters with vision. Full 256K context at 25 tok/s. Knowledge breadth is 7x the 27B. Nemotron-Labs-3-Puzzle-75B-A9B: NVIDIA compressed their 120B Super down to 75B total / 9.3B active. NVIDIA's own post markets it as "perfect for your single GB10." 97%+ of parent quality. NVFP4 weights are 44.5GB. Released July 9. Nemotron-3-Super 120B-A12B (NVFP4): 21-25 tok/s (NVIDIA developer forum). 1M context. 12B active. Native MTP. Built for multi-agent. gpt-oss-120B (MXFP4): 57-60 tok/s pure decode. Fastest single-stream on the Spark. Concurrency: WescheNex1q ran 64 concurrent users on one Spark. 700+ tok/s aggregate. Cost: $4,699 --- Really good buy, but really understand the issues of low memory bandwidth with this one, and it’s trade-offs --- PATH 5: RTX 6000 Ada, 48GB (~$6,800) Single card. 960 GB/s. Same 48GB as dual 3090s but no PCIe overhead, no heat nightmare. Nemotron-3-Super 120B at Q3 (~58GB) fits with offload. Honest take: 5x the cost of a single 3090 for double the VRAM. Rough price per GB. But cleanest single-card 48GB path. Cost: ~$6,800 --- PATH 6: 2 DGX Sparks, 256GB (~$9,500) "Near" Frontier-scale models fully offline: Qwen3-235B-A22B: 17 tok/s batch=1, 36 aggregate at batch=4. Beats NVIDIA's own published number by 45%. MiniMax-M3 (428B MoE + vision): Fits at AutoRound 3.2-bit mixed quant (188.6GB across two nodes). 13.7 tok/s prose, 15 tok/s code, peaks of 20 with EAGLE3 speculative decoding. Vision tower works. Not full precision. 3.2-bit is aggressive. But it runs. GLM-5.2: Needs 4 Sparks for full quality. 2 Sparks is the down payment. This is why I bought mine. 256GB unified holds weights AND parallel agent KV caches. No rate limits. No provider seeing your prompts. Cost: ~$9,500 with DAC cable --- --- PATH 7: RTX PRO 6000 Blackwell, 96GB (~$12-14.5K) 1,792 GB/s. Native NVFP4. This is what @Hikari_07_jp runs (two of them, 192GB total). 6.7x faster than the Spark on the same models. Nemotron-3-Super 120B at Q4 (~70GB) fits on one card with room. Dense models fly. The bandwidth king. $138 per GB of VRAM vs $33 for a 3090. Density buy, not a generalist one. It has a strong upgrade path, but the rig you build if you want high VRAM e.g. 4-6x of these cards, will allow you to literally run the largest models, but cost will run to $70-90k, AND as others pointed out there will be a lot of HVAC things. Cost: $12,000-14,500 --- The decision: Want free speed? Enable MTP. 1.4-2.2x on same hardware. Want speed demon Blackwell on a prosumer budget? RTX 5090. 32GB, NVFP4, 4.5K. Not recommended imo (tiny Vram, very overpriced), might as well just jump higher, or stick with a 3090 or 4090. (and save that $4k) Want frontier MoE offline? Spark. Step 3.7 Flash, Puzzle-75B. $4,699. Want the budget version? Strix Halo. 128GB unified, $2.5K. Not recommended Want frontier scale on your desk? 2 Sparks. Qwen3-235B, MiniMax-M3. $9,500. Want maximum dense speed? RTX PRO 6000 Blackwell. $12K+. Don't buy a Spark to run dense models fast. Don't buy a 3090 to run 235B. Know which axis you're upgrading. Reference links in reply for your own technical DD 👇

@Hikari_07_jp @MichaelGannotti NVFP4 is really great, fast and precise

@MichaelGannotti I knew about NVFP4 before buying the Blackwell GPU, but I didn't realize it would be this powerful. NVFP4 and MXFP4 are bound to become the standard.

NVFP4 is a new, super-compact way for computers to store numbers using just 4 bits (tiny pieces of data). It was introduced with NVIDIA’s Blackwell graphics cards and is designed specifically to make AI models run faster and use less memory. Think of it like a very stripped-down version of how calculators store decimal numbers. Most 4-bit floating-point formats work the same basic way: they use • 1 bit to say if the number is positive or negative, • 2 bits to control how big or small the number can get (the “exponent”), and • 1 bit for the actual digits (the “mantissa”). This format can represent numbers roughly between -6 and +6. The exact values it can store are limited — things like 0, 0.5, 1, 1.5, 2, 3, 4, and 6 (and the same set of negative numbers). The main challenge When you squeeze numbers into only 4 bits, it’s hard to stay accurate if the numbers in your AI model vary a lot in size (some very small, some much larger). This is a common problem with ultra-low-precision formats. How NVFP4 solves it NVFP4 uses two clever tricks to keep accuracy high even with such limited storage: 1. High-precision scale encoding — It records the “scaling” number (the adjustment that stretches or shrinks the values to fit into 4 bits) more accurately than usual. 2. Two-level micro-block scaling — Instead of using one scale for a huge group of numbers, it uses scales at two different levels on very small groups (“micro-blocks”). This lets the system adapt better when the numbers change a lot within the data, so less accuracy is lost. In short, NVFP4 gives developers another practical tool for running AI models quickly on new @NVIDIAAI hardware while still keeping the math reasonably accurate.

Ursula von der Leyen has confirmed that everyone in the EU will need to use the EU's app for identity authentication before being able to access or post on social media websites. 🇪🇺 As an expert in online child safety, I am here to expose the misinformation and misdirection in von der Leyen's statements. Today von der Leyen said: "This is not about whether children can access social media, it is about whether social media can access our children". 💡The first part is true. This isn't about children. It's about surveillance and combating political dissent. A state that can't control its own citizens is more dangerous than a state rife with criminals. The second part is a PR soundbite that politicians are using like a campaign slogan straight out of 1984. 🇪🇺"The question is no longer if children face risks online, but what can we do to give children a safer start online". 💡No. You can't give children a "safer start" online any more than you can offline. In the offline world, the government doesn't enforce curfews or ban children from entering liquor stores, bars or restaurants. That's a parent's responsibility. The digital world should be no different. 🇪🇺"The age verification app is one of the tools to get it done". 💡This is a contradiction because she also said "It won't be foolproof". 🇪🇺"It's easy to use, it is privacy preserving and it is open source". 💡The app was compromised as soon as it was released. "Privacy-preserving" age verification is an oxymoron. You can't verify a person's age without verifying their identity. Where or how that age is shared afterwards is irrelevant. 🇪🇺"This is basically about putting back the power into the hands of parents". 💡More from 1984. The EU is doing the opposite. Parents are having their authority stripped by politicians who think they know better. Many parents are capable and unaffected by peer pressure, and they know how to use parental controls to block any app classified as 13+. Some teens are safe, their parents trust them, and the state has no business overruling that trust. 🇪🇺"We don't give our children keys to the car before they have their licence" 💡Comparing an app to a car is a false equivalence used to justify mass surveillance. Governments don't decide when a young person is ready for car keys, guardians do. 💡Forcing every adult and child into a biometric checkpoint just to use an app or website is not licensing drivers. It's the state seizing everyone's keys, locking the garage, and forcing every driver to ask a private company for permission to take a drive. 💡 This is a gross, unethical overreach that strips authority from parents while imposing state sanctioned identity verification on every adult who doesn't even have a child. 💡Additionally, people who pass a test, obtain a licence and drive a car aren't forced to use an app to constantly authenticate their suitability to drive. 🇪🇺"We do not let them buy alcohol until they are legally allowed" 💡False equivalence. We don't force every person to show ID at a shopping mall entrance just because a few people might buy alcohol with a meal at a restaurant. Some parents are okay with their 12 year-old going to the mall with friends while others aren't. Either way, it's their choice. Whatever irresponsible decisions some parents might make, every adult in the country shouldn't be forced to pay the price. 🇪🇺"It won't be foolproof" 💡This is all the proof we need to show that the EU and every government know that banning social media for teens won't protect them. When pressed by journalists about VPNs being used to circumvent a ban, politicians always state the ban isn't a silver bullet and will take time. The Mayor of London, Sadiq Khan went as far as to say "we know it's not the solution". 💡Either age verification works, or it doesn't. As a technical expert in this space, I can tell you there are no additional steps to take and no progress to be made. Either the approach does what it is supposed to do, or it's not fit for purpose. If they claim a bulletproof solution is coming, it can only mean one thing. They intend to ban or restrict VPNs to people who verify their identity. 🇪🇺"It will take time to invite the cultural change that is already taking shape in our society, just as it took time to outlaw drink driving, just as it took time to use seatbelts in the cars. Great change never happens overnight, but when it comes to our safety it is always worth it". 💡Comparing a social media ban and age verification to seatbelts is a completely broken analogy. Seatbelts are a safety feature that protects children while allowing them to travel in a car. A ban doesn't give kids a seatbelt. It kicks them out of the car entirely. 💡Instead of supporting parents who want to guide their own children through the digital world, this heavy-handed law strips away parental authority by banning the apps and websites that many parents are perfectly fine with and actively monitor. 💡Furthermore, enforcing these bans requires biometric age verification, which means forcing millions of adult citizens to scan their IDs, faces or credit cards just to browse the internet. That isn't a common-sense traffic law. 💡It's a digital checkpoint on every street. True safety means teaching kids how to navigate the digital world safely with real guardrails and parental guidance, not burning down digital spaces for everyone under the guise of protection. 🇪🇺☠️ The EU wants to ban teens from social media so every person is forced to verify their identity before they can read, share or post anything online. In their words, this is to protect children from dangerous content. 🇪🇺☠️ The EU wants to enforce "Chat Control" so every app has to monitor everything people say privately inside it, including apps with end-to-end encryption. In their words, this is to protect children from dangerous criminals. 💡Where this ends 🇪🇺 "If you want a picture of the future, imagine a boot stamping on a human face - forever". George Orwell, 1984.

@0xSero This card is a powerhouse

GLM-5.2 ASrock romed8-2t Epyc 7443P 4x RTX Pro 6000 (workstation 600w version) You can see on the left how power capping the cards effects speeds. At 600w we'd start somewhere around 90 tok/s I need to restrict them to prevent the house from melting down. why oh why

@atomic_chat_hq Which quantization of qwen? FP16?

DFlash makes Qwen 2.2x faster with no quality loss! We ran the same Qwen3.6-27B locally three ways on one RTX 6000: baseline, MTP, DFlash. The tasks only differ in one thing - how predictable the next word is: quicksort, describe a file in JSON, a logic puzzle, a sci-fi story. Outputs: Baseline: 44 tok/s · 1.00x MTP: 65 tok/s · 1.45x · 71% accepted DFlash: 98 tok/s · 2.20x · 30% accepted Baseline writes one token per step. MTP works inside the model itself and guesses 3 tokens ahead. DFlash is a separate small model that writes 15 tokens at once, and the big model only checks them. In JSON the same words repeat all the time, so most guesses were right: 152 tok/s, 3.4x speedup. In the story 9 guesses out of 10 were wrong. DFlash did all that extra work for nothing and became slower than baseline: 42 vs 44 tok/s. MTP guesses only 3 tokens, so a wrong guess costs very little: 46 tok/s and the win in that round. The output is identical in all three modes - DFlash is the pick for tasks with predictable output, like coding, and for chat and creative writing MTP works better. DFlash is now natively integrated into Atomic Chat on llama.cpp - speed up your Qwen models!

@TiffanyFong I see not community notes

Bluesky is for pedophiles

@digitalix Funny boob eyes

My first time "using" blender is asking claude to generate a "funny character".

@LSK94LH @TiffanyFong Not true, communists accepts as factual even more stupid concepts

@TiffanyFong not even Communists like Stalin or Mao would tolerate this retarded leftist shit.

Only a woman would speak like this

@myzccc Ha detto che votava “no”, riferendosi al referendum di quest’anno sulla giustizia (dove ha vinto il no purtroppo dato che in Italia siamo delle capre)

İtalya Başbakanı Meloni ile fotoğraf çektiren genç, ona oy vermeyeceğini belirtti. Meloni: “Ne yapalım… Demokrasi bu işte.”

We are still waiting for Grok Desktop

@yongfook Conversion of backend from python to rust with no errors and also solving pre-existing bugs

This may be shocking to some, but I have never used Fable, and I think Opus is just fine for what I need. Are you all out there sending rockets into space or something?

@kimmonismus Thanks competition 🙏

GPT-5.6 was a success.

@powl_d Ho la model y standard 2025 e anche la model y performance del 2022. Non cambierei la performance con la standard. L’unico aspetto superiore della nuova y standard è l’hardware AI4. Per il resto la performance rimane una spanna sopra.

Bon… mon Model Y Prop de 2023 a 50 000 km. Je l’avais acheté 40 990 €. Hier, en discutant avec un vendeur Tesla, il me dit : « Paulo, c’est peut-être le moment de changer. » J’ai refait les calculs… et il avait raison. • Reprise Tesla : 25 800 € • Bonus reprise Tesla : 5 000 € • Prime CEE : 3 600 € (jusqu’à 5 700 € si on le met au nom de ma mère, qui est éligible et c’est elle qui le roule principalement). Au total, ça ferait 36 500 € d’avantages. Model Y neuf : 40 990 € ➡️ Reste seulement 4 490 € à sortir. Franchement… je pense que la semaine prochaine, ça passe à l’action. 👀

@bryan_johnson First 6 7 hours

I asked for life advice and got 5,500 responses. Here's the how people showed up.

@ivanfioravanti Thru what do you access it with?

Grok 4.5 is ultra fast! How is this even possible, it seems a small model, but it's not! Fast, smart and low cost at the same time 🤯 BTW I'll postpone my deep dive and heavy usage to next week when it should be officially available in E.U

@uzairansar No fucking way I’m going to switch to Grok/ChatGPT so hard

Completely forgot about this.. Claude limits are about to get nuked soon

@DanielleFong Does it boil water?

"so what do you do" "oh, i work on a kind of lightsaber powerplant" "what is that?" "here ... it's easier if i show a picture... ok understand that this is a work in progress"

@chamath @Jason 😰

@Jason I hear Uranus has some fantastic chocolate mousse.

Where can I get the best chocolate mousse in Paris tonight? I need to pair it with a Cuban cigar and some port

@wholemars Lol

so that’s what happened

@AdrianDittmann Neon genesis evangelion vibes

The new clamp retraction mechanism for the Starship launch pad goes so hard