I'm finally happy with my @UnslothAI unsloth/Qwen3.6-35B-A3B-MTP-GGUF, @no_stp_on_snek llama.cpp-turboquant, @_HermesAgent setup, @MajorFAFO params suggestions, and my own Rust/SQL/Hermes app.
- @ASUS, 32GB RAM, 8GB VRAM 5070 laptop
- MoE fully offloaded to CPU
- I've fiddled around with --no-mmap and mlock. It pegs my RAM to 95% instead of 45% and after it spins up, I don't see a difference.
--cache-type-k q8_0 \
--cache-type-v turbo3 \
-ot "\\.ffn_.*_exps\\..*=CPU" \
--spec-type draft-mtp \
--spec-draft-n-max 3 \
The video shows it has full hermes skills/toolset access easily, knows my system, can be creative, can print out a quick BASIC function, corrected itself when I told it that it was incorrect, and that it doesn't love me....
English