emptyArray

I'm finally happy with my @UnslothAI unsloth/Qwen3.6-35B-A3B-MTP-GGUF, @no_stp_on_snek llama.cpp-turboquant, @_HermesAgent setup, @MajorFAFO params suggestions, and my own Rust/SQL/Hermes app. - @ASUS, 32GB RAM, 8GB VRAM 5070 laptop - MoE fully offloaded to CPU - I've fiddled around with --no-mmap and mlock. It pegs my RAM to 95% instead of 45% and after it spins up, I don't see a difference. --cache-type-k q8_0 \ --cache-type-v turbo3 \ -ot "\\.ffn_.*_exps\\..*=CPU" \ --spec-type draft-mtp \ --spec-draft-n-max 3 \ The video shows it has full hermes skills/toolset access easily, knows my system, can be creative, can print out a quick BASIC function, corrected itself when I told it that it was incorrect, and that it doesn't love me....

@serhumano738477 @raccoon_builds @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS Definitely an upgrade compared to normal. Everything about the turboquant is faster

@ArrayEmpty @raccoon_builds @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS W configs bro, great work! Where did u find a lammacpp mtp turboquant version? Did u had any performance downgrade compared with normal kv quantization?

I'm finally happy with my @UnslothAI unsloth/Qwen3.6-35B-A3B-MTP-GGUF, @no_stp_on_snek llama.cpp-turboquant, @_HermesAgent setup, @MajorFAFO params suggestions, and my own Rust/SQL/Hermes app. - @ASUS, 32GB RAM, 8GB VRAM 5070 laptop - MoE fully offloaded to CPU - I've fiddled around with --no-mmap and mlock. It pegs my RAM to 95% instead of 45% and after it spins up, I don't see a difference. --cache-type-k q8_0 \ --cache-type-v turbo3 \ -ot "\\.ffn_.*_exps\\..*=CPU" \ --spec-type draft-mtp \ --spec-draft-n-max 3 \ The video shows it has full hermes skills/toolset access easily, knows my system, can be creative, can print out a quick BASIC function, corrected itself when I told it that it was incorrect, and that it doesn't love me....

@serhumano738477 @raccoon_builds @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS Exactly. I want the big window along with a decent speed and it seems to work pretty well

@raccoon_builds @ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS Doing this, he can allocate more context window, maybe it can process faster. 3b parameters on CPU is not very slow. He's using 256k context

@SouthLiPeace1 @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS It's not great, somewhere around 40tps. But I'm not doing anything crazy. I can give it a command, do other stuff come back to it.

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS what is your speed

@UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS I'm gonna try this with no-mmap and mlock see what transpires huggingface.co/Jackrong/Qwopu…

@KyleHessling1 Man I wish I didn't have a life so I could just sit on the computer all day and try these LLMs

BREAKING! Qwopus 3.6 27B is LIVE! Thank you for your patience on this one, but I believe you'll find the wait was worth it! We've benchmarked this thing up and down, verified that it holds at least a 75.25% (152/202) in the initial 202 SWE bench solves. Not a full run of 500, but it shows the agentic coding quality from the original 27B is retained while adding all of the additional Qwopus benefits across many domains. As always, Jackrong is absolutely cooking here! COT quality has improved significantly through the inversion techniques from our Negentropy proof of concept. It also went through thorough curriculum training. You can check out the MMLU pro benchmarks on the model card, but it improved a whopping 10 points over the base model in physics, as well as meaningful jumps in Chemistry, business, and computer science. However, the best part is that I was able to build an entire survival shooter game using this local model entirely. I genuinely was blown away by the results, which you can play right now on my HF space (link in comments below). "Qwopus Commander" was completed in 9 turns of Qwopus 3.6! To test the new long context training, I made it re-output the entire 3000+ line program each turn, and it would make fixes and add features that I requested in large prompts, while perfectly replicating the entire rest of the game from context. What's more is that I did it all at Q8 KV cache quantization, and never had an issue over the entire 303k token run! IMPORTANT: Run it at --temp 0.75 to 1. Mess with it in that range for your use case. Higher temp actually lets the fine-tune shine and be exploratory and is also more stable. Swe Bench was run at temp 1, the game was built mostly at 0.8! We're so blessed to have all of you here and using the models! The support means so much! Please let me know what you build with it in the comments! Or if you have any issues getting it up and running, I will try my best to get back to you! Looking forward to seeing what you legends produce with it this weekend! huggingface.co/Jackrong/Qwopu…

@eclecticleaps @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS *no it's not on GitHub it's something I'm messing around with

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS That GUI interface looks outstanding! Is it on github?

No it's just not that I'm messing around with. I had @cursor_ai build it. I gave it complete control and it decided to build a @TauriApps Rust app. To be honest I'm an intermediate JavaScript python coder at best, I know nothing about tauri or rust, so it might be pretty ugly haha

@sakurayukiai @no_stp_on_snek @grok can you explain this to a newbie in AI

@no_stp_on_snek Honestly native MTP is gonna kill traditional speculative decoding. Why babysit a separate draft model and manage two KV caches when the base model can just predict its own future tokens?

Tested out MTP for the first time on my llamacpp fork last night with turbo4 sym. GX10 hardware. using MoE model: llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved +0.41 % PPL vs fp16 baseline

@jun_song You're good man, do your thing. F*** the haters

To my 12,000 followers, I completely understand if you are confused or disappointed by my sudden posts about crypto. Those who already have strong negative feelings probably won't accept anything I say right now. However, I have spent the last few months deeply studying the open source ecosystem, meeting countless developers, and listening to their struggles. The reality is that the open source funding system is completely broken. The top 0.1% of high-profile creators hoard the vast majority of sponsorships, regardless of how critical their project actually is. The rest of the developers are barely getting by, even though many of them have made breakthroughs that could literally change the world. I was incredibly frustrated by this broken system, and I kept searching for a way to fix it. I came to the conclusion that crypto mechanics are actually the most effective tool for this. So, I decided to risk my entire reputation and face all the stigma head-on to flip this into a positive ecosystem. Just like I previously worked to change the narrative around local LLMs and Sovereign AI when they were dismissed. Open source must win, and we don't have much time. If you are too disappointed, you are welcome to unfollow me. But I will welcome you back later, after I have radically revolutionized this deeply rooted problem. I am sorry to have let you down. Sincerely, Jun Song

@obvirm @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS This is actually one thing I haven't tried, I've had NGL set every time I try something. I will try it!

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS jangan pakai -ngl kalok mau ngejar speed

@bepleecanton @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS I'm able to do it with no mmap with my 32 GB ram but it's pretty close to crashing all the time haha. I'm not able to do anything else except use the AI. The whole point of this was to be able to use it and also do other stuff like he's a web browser terminals blah blah

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS I tried no mmap on mine, 32GB RAM, 16Gb VRAM , hard crash so i just ditch that. Gonna try new tbq with mtp now😆

@QuantCapitalX @xbin12345 @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS @grok It's just something I built for myself (using AI), it's not a thing.

@ArrayEmpty @xbin12345 @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS @grok what is this Orchestrator for hermes? give me the github repo link.

@barackomaba @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS Ah I see what you mean. Ok, I'll try the non mtp..won't hurt. I've been doing so many different params, I feel like I've tried it. I should be writting all this shit down.

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS But MTP increases memory requirements, either way. Your prefill is really struggling.

@MooncastOnline @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS I've found any --n-cpu-moe adjustments < 40 crashes llama, so full cpu offload seems to be working. I think i will try that --reasoning on --chat-template-kwargs "{\"preserve_thinking\":true}" though....

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS -ngl 999 -fit off --no-mmap --mlock --no-mmproj --n-cpu-moe 39 --kv-unified --cache-ram 0 --parallel 1 -fa 1 -b 128 -ub 64 --reasoning on --chat-template-kwargs "{\"preserve_thinking\":true}" -t 8 -c 256000 --port 8080 --host 0.0.0.0 Adjust the n-cpu-moe, lower uses mode vram.

@barackomaba @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS I've found that even with the non MTP there's not enough room to offload any of the MoE. If I even change --n-cpu-moe to 39 llama crashes, even if I bring context down to 65k (minimum requirement)

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS wont mtp slow you down because youll have to offload more since youre ram constrained?

@UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS UPDATE: apparently I'm supposed to using --n-cpu-moe 999 instead of -ot "\\.ffn_.*_exps\\..*=CPU"

@firstvostor @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS Какова скорость генерации токенов?

@xbin12345 @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS Thanks - my favorite part is operations

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS wow that's wild. The Orchestrator is also very interesting.

@raccoon_builds @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS @grok Yea my GPU is already pegged with memory with my current setup, changing -ngl would crash it

@ArrayEmpty @UnslothAI @no_stp_on_snek @_HermesAgent @MajorFAFO @ASUS @grok give him some advice please

@elmoche_ @no_stp_on_snek @UnslothAI @_HermesAgent @MajorFAFO @ASUS I honestly just assumed it did after I pulled the latest from repo. So apparently, it does.

@ArrayEmpty @no_stp_on_snek @UnslothAI @_HermesAgent @MajorFAFO @ASUS Wait what? Turboquant llamacpp from tom has mtp support?