Lean Kin Prak

@spok_vulkan Ok thank you

@LeanKinPrazli And the funny part is, in most cases, you will not see the errors directly the result itself will just be worse overall, you can really notice only in a direct comparison or benchmarks

I just ran into something wild building a local AI agent. Qwen3.5-9B at INT4 (ParoQuant) performs WORSE than Qwen3.5-4B at 8-bit on tool-calling benchmarks. More parameters. Worse results. Here's what we found.

@mauriciob91 @0xSero Thanks! Yeah I did and got around 9 tok/sec but it fell apart in Openclaw

@LeanKinPrazli @0xSero With unified memory you’re better off running MoE models like precisely that Qwen 3.5 35B-a3B one

Qwen3.5-35B compressed 20% with 1%~ performance drop on average. Now you can fit this (4bits) with full context on 24GB of VRAM 700$~ or 1x 3090 huggingface.co/0xSero/Qwen-3.…

@0xSero Not sure if this could interest you x.com/dnhkng/status/…

Guys, it’s time. There is the appetite now for building a future that doesn’t rely on the kindness of private companies. I pledge to upload all my sessions over the last 3 years. To do this right we need a way to anonymise this and strip secrets github.com/0xSero/ai-data…

@0xSero LOVE this! When you asked about research examples I was gonna say it'd be cool to see how much you could quantize/compress models to fit on all sizes of RAM just like this and especially for agentic/tool work in Openclaw etc.

Best models to run on your hardware level I'll be doing this every week, I hope you guys enjoy. ---- 8 GB ---- Autocomplete for coding (like Cursor Tab) - huggingface.co/NexVeridian/ze… - huggingface.co/bartowski/zed-… Tool calling, assistant style - huggingface.co/nvidia/NVIDIA-… ---- 16 Gb ---- Here things get better: Multimodal - huggingface.co/Qwen/Qwen3.5-9B - huggingface.co/Tesslate/OmniC… - huggingface.co/unsloth/Qwen3.… ---- 24 GB ---- - The best model you can get (thanks Qwen) huggingface.co/Qwen/Qwen3.5-2… - Great model (strong agents) huggingface.co/nvidia/Nemotro… - Mine hehe huggingface.co/0xSero/Qwen-3.… I'm doing a weekly series

@NoemiTitarenco @sudoingX How dumb? 9B?

@sudoingX I'm building a superfast browser harness that should be able to work with even dumb models because the tool use/tool discovery is fool proof.

to all of you saying local models aren't there yet because some corporate salesman on an openai paycheck told you so. you're running their bloated tools and blaming the model. the model is fine. the bloated harness is the problem. i've tested literally every harness out there and i have the facts and receipts on my timeline and DM. openclaw is 120K+ lines of typescript bloat backed by corporate, mining your thinking while you pay for the privilege. switch to hermes agent and watch the same model become usable. don't take my word for it. just try. i have DMs from people who made the switch and their "broken" model started working instantly. same hardware, same model, different harness. if you're using hermes agent and someone near you is still on openclaw, help them get away from the bloat. they're frustrated at every step, burning tokens doing completely nothing, paying subscriptions to think on someone else's server. buy a single GPU from ebay. compile llama.cpp. install hermes. replace your openai subscriptions and think free. once you think free you start seeing light. you deserve better cognitive tools than bloat that harvests you.

@ooo000ooo00ooo @0xvallion @DeFi_Hanzo It’s too sensible. The cutting edge folks are here on X

@LeanKinPrazli @0xvallion @DeFi_Hanzo You could open chat gpt and ask. It will guide you through the entire set up and advise you on the best qwen and deepseek models.

I bought a Mac Mini 2 months ago. People laughed. "Why not just use the server?" "Why you play on 1win?" "Local models are a toy." "You'll never match GPT-4 quality on consumer hardware." Google just released TurboQuant. An algorithm that shrinks AI model memory by 6x without losing intelligence. 8x faster. Same number of GPUs. Same quality. My 16GB Mac Mini can now run models that required a $50,000 server 18 months ago. Here is what actually changed: > kv-cache compressed to 3 bits with zero accuracy loss > models that needed 96GB of VRAM now fit in 16GB > the performance gap between local and cloud just collapsed The people who laughed at the Mac Mini are now watching Micron and Sandisk stock fall off a cliff. Because if you don't need 6x the memory to run AI - you don't need 6x the memory chips. $527 billion in combined market cap. Memory prices up 500% on AI demand.

@0xSero I don't know what it means but I'm very excited for where it looks like local LLM's are going.

I need someone who's not been one-shotted to validate these numbers, I have gotten consistent results but I don't have enough expertise to make sure the models are honest. github.com/0xSero/turboqu…

@10xROE @NielsRogge I KNEW my good looks held me back

@NielsRogge Because anyone this good at computer science doesn’t care about there personal appearance lol

Why are local LLM wizards always these weird, anonymous accounts

@0xSero Thank you sir

@LeanKinPrazli Q3 + github.com/0xSero/turboqu… + using mlx + turning off every other app, you might be able to do it.

@Alexintosh @eBay That’s a mac mini in the image

I just bough an M3 ULTRA 4TB SSD 512GB RAM 32-CORE 80-CORE GPU from @eBay at an outragiously low price. I haven't bough on ebay for years, truly hoping I do not receive a bunch of brinks. Wish me luck.

@VraserX Giving up the rights to your voice is not ok.

I genuinely love Suno AI. Making music that actually feels personal instead of generic is such a huge deal. Can’t wait to create even more songs that sound like me, feel like me, and actually mean something to me.

@Aleks00228561 @ChuckYouSuck @suno Hahaha, I PAY to use their service and they've trained on all music ever made so NO, I will not give them the rights to my voice.

@LeanKinPrazli @ChuckYouSuck @suno They (Suno) give you all the rights to use their song which you can make in 10 seconds. Why wouldn't you give them the rights to use your voice. Otherwise, no one is forcing you to use your voice.

Meet Suno v5.5: More expressive, more you. Use your voice, your sound, and your taste to make music that's unmistakably yours, in the best and most personal Suno experience yet.

@LLMJunky No use for Macs I assume?

Two incredible innovations in the local AI space in a span of three days. I am so excited. ComfyUI just shipped "Dynamic VRAM" and it seems like a big deal for anyone running models locally. The problem: large AI models can have many GB of weights. If your system lacks the necessary RAM, you'd normally hit memory crashes or grind to a halt on the page file. Instead of loading the entire model into memory at once, ComfyUI now reads the model file piece by piece directly from your SSD. Only the specific parts needed for the current step get pulled into memory. Everything else stays on disk until it's actually called for. On the GPU side, they built a smart system that loads weight data at the exact moment it's needed. If your GPU runs out of space, it doesn't crash. It uses a temporary workaround to finish the calculation, then cleans up after itself. It also keeps track of what didn't fit so it doesn't waste time trying to reload things that won't fit again. The other big improvement is for workflows that use multiple models. Previously, swapping between models would pile everything into system memory and bog your machine down. Now when a model gets swapped out of the GPU, it just goes back to the "read from disk when needed" state instead of sitting in RAM. The result: a 56GB model can now run on a machine with only 32GB of memory. No crashes, no slowdowns from swap. Available now for Nvidia GPUs on Windows and Linux, with AMD support on the way. No idea how fast this is, but this seems incredible. Cannot wait to get my workstation going.

@Eduardopto @suno well if that's true no serious artist can use the feature, sadly.

Great question! When you upload or record your voice for Suno’s new v5.5 Voice Model feature, yes, you grant Suno a super broad, perpetual, royalty-free, irrevocable license to use your voice data, likeness, and Voice Model forever. You still own your original recordings, but they can use it to train/improve their AI, generate songs, promote the service, etc. (They won’t sell or give your raw Voice Model to other users (only the final tracks that use it)

SO HYPED for @suno v5.5 dropping TODAY 🔥 Finally music that’s more expressive and more YOU than ever! • Add your own voice to the mix (record or upload it actually sounds like YOU) • Create custom models trained on your personal sound • Let “My Taste” learn your favorite genres, vibes, and style so every track feels personal This is the most personal Suno experience yet. Game-changer for creators who want authentic, one-of-a-kind songs! Already queuing up my first voice drop 😍 Who else is jumping in right now? #Suno #SunoV55 #AIMusic”

@0xvallion @DeFi_Hanzo Exactly. Which ones and how? I have the 16GB too and want to learn.

@DeFi_Hanzo So which models are you running on your mac mini?

@demishassabis @LuizaJarovsky @elonmusk @sama @DarioAmodei @sundarpichai @tim_cook @satyanadella @JeffBezos @finkd @IsomorphicLabs If you make this happen, you win EVERYTHING!!!

@LuizaJarovsky @elonmusk @sama @DarioAmodei @sundarpichai @tim_cook @satyanadella @JeffBezos @finkd We are already working on this with tools like AlphaFold and the work we are doing at @IsomorphicLabs

I would like to invite @elonmusk, @sama, @DarioAmodei, @sundarpichai, @demishassabis, @tim_cook, @satyanadella, @JeffBezos, @finkd, and all the other tech CEOs to join the "AI race to cure cancer." The winner gets humanity's forever appreciation.

@NinosMaron @hackuna @suno I meant if someone else can use the voice now willingly or by chance when they just create a song.

@LeanKinPrazli @hackuna @suno It’s your own voice, I don’t think it’s a feature to clone someone else’s

it's about to get personal.