@0xCVYH I use flux Klein 9B with comfyui . Is there a node to load this new polarquant ?
English
Draneil Mifa
220 posts
@DragonGroky PolarQuant uses a different approach than GGUF. Hadamard rotation + Lloyd-Max optimal centroids give better quality at the same bitrate. GGUF packs bits, PolarQuant optimizes the quantization itself. Working on llama.cpp integration tho
English
PolarQuant agora quantiza modelos de IMAGEM.
FLUX.2 klein 9B com Q5: 0.9986 de similaridade com o original. Praticamente identico.
121 camadas quantizadas, 304 preservadas em BF16. Hadamard + Lloyd-Max funcionando em diffusion models.
Alguem pediu hoje e ja ta pronto.
huggingface.co/caiovicentino1…
Português
@DragonGroky Yes, fits perfectly on a 3090 24GB. PQ5 quality is actually better than standard Q8 at similar size because of the Hadamard rotation + Lloyd-Max centroids. 0.996 cosine similarity vs original. Give it a try and compare
English
@MyopicRaccoon @LottoLabs @no_stp_on_snek i ve a 3090 mais reachs only 20token/s ! can you share your llama.cpp parameters ? i've also turboquant version. llama-server -m models-d/Qwopus3.5-27B-v3-Q5_K_M.gguf -c 128000 \
-ctk turbo3 -ctv turbo3 \
-fa on -np 1 \
--cache-ram 4096 \
--ctx-checkpoints 4 \
--fit on
English
@LottoLabs Here's what I am running, Qwen3.5-27B-UD-Q5_K_XL.gguf, with @no_stp_on_snek's turboquant (turbo4), 128K context (I could double if I drop to Q4), on a single RTX 3090/24GB. Getting a solid 31 tok/s. and Hermes 0.70. Heaven.
English
@outsource_ it is better than Qwen3.5 27B and does it run on a 3090 ?
English
REQUESTED: Qwen3 30B Coder-Next?
The community delivered 🔥
mradermacher/Qwen3-Coder-30B-A3B-Instruct-480B-Distill-V2-Fp32-i1-GGUF
⚡ Distilled from 480B into 30B Coder
📈 SWE-Rebench / agentic coding
✅ runs perfectly local with GGUF quants
Exactly what you wanted, consumer-hardware friendly:
huggingface.co/mradermacher/Q…
English
@DragonGroky @sudoingX - "--cache-type-k"
- "q4_0"
- "--cache-type-v"
- "q4_0"
- "--host"
- "0.0.0.0"
- "--port"
- "8000"
- "--alias"
- "Qwen3.5-27B"
people keep asking me what model to run on a single 3090. it's not even close.
Qwen 3.5 27B dense Q4_K_M. undisputed.
kumikumi (Ankkala)@ankkala
@sudoingX to be clear, which model / quantization did you run on the 3090?
English
@DragonGroky @sudoingX I was waiting for Gemma-4-31B with TurboQuant cache.
To swap out Qwen3.5-27B. My expectation was: Bigger is better. Apparently not for my use case. x.com/TeksEdge/statu…
x.com/leftcurvedev_/…
left curve dev@leftcurvedev_
everyone is wondering the same thing qwen3.5 27b or gemma 4 31b? new benchmarks from @ArtificialAnlys are out, let's dig the numbers: 💻 coding index > gemma wins surprisingly it was the best and scored 42 — it managed to handle more coding tasks successfully than qwen, very interesting! 👀 🤖 agentic index > qwen destroys gemma when it comes to tool calls, multi-step reasoning, and autonomous task execution, there's no need to talk about it — qwen is the absolute winner in the category, scoring 55 (!) vs 41 for gemma 🤯 👑 the winner > qwen3.5 27b stays undefeated gemma could have been an amazing contender but in agentic tasks it's just too much behind, compared to what qwen has to offer — makes no sense to use it if your tasks are heavy and need reasoning what are your thoughts?
English
③ 日本語1クリックで完結！最強の全自動ワークフロー🤖
「設定が難しそう⋯」という方もご安心ください！
最新のAI「Qwen 3.5」を組み合わせれば、日本語で「夕暮れを歩く少女」と入力するだけでOK✨
AIが自動で最適な英語プロンプトやカメラワークの指示を作ってくれます。
面倒な計算もすべてAIにお任せして、最高の一本を作りましょう！
日本語
@loktar00 for a 3090, it is the good parameters for using TurboQuant ? export PATH=/ia/llama-cpp-turboquant-cuda/build/bin:$PATH
llama-server -m models-d/Qwopus3.5-27B-v3-Q4_K_M.gguf \
-c 132000 -n 4096 \
--cache-type-k q8_0 --cache-type-v turbo3 \
-fa on -np 1 -ngl 99
English
@DragonGroky @sudoingX For me 196608 is a sweet spot. Larger amounts get flushed into RAM, dropping the performance.
English
@DragonGroky @sudoingX My Qwen3.5-27B-Q4_K_M implementation (third compose) has a context length of 196608. It runs flawless with OpenClaw, OpenCode/GSD and my home brewed LangGraph agents such as a Docling/LangExtract agent. I was hunting 512k with TQ3_4S.
English
@sudoingX Using Hermes and coPaw, I ve better result with coPaw (LLM : qwen 27B Q5 turbo quant). Using on a 3090
English
if you're still on openclaw bloatware in april 2026, you're not a builder. you're a tourist that has 0 idea how things are accelerating.
try hermes agent for yourself. take that step and make the switch today anon. and i'm not the one calling it bloat that came from builders in my dms and timeline. bloat is their words, not mine.
and if you have already made the switch, let people know your experience so more builders on bloated tools become aware.
you deserve a better tool in this fast changing field.
Teknium (e/λ)@Teknium
Hermes Agent is the third fastest growing GitHub repo this week!
English
Using Qwopus3 Q5 and Hermes is not so good, Hermes is quickly running in loop to try to patch but with copaw, i don't have this problem, it looks much better and doesn't go in loop.
English
English
@sudoingX I'm currently running Qwopus3.5-27B-v3-TQ3_4S, the Qwen3.5-27B 'Opus' v3 fork with TurboQuant cache.
Context length 256k, 35 tokens/sec on a 4090.
huggingface.co/YTan2000/Qwopu…
English
@sudoingX on my 3090, I'm running Q5_K_M. llama-server -m models-d/Qwopus3.5-27B-v3-Q5_K_M.gguf --alias "Qwen3.5-27B-Q5-Qwopus3" \
-c 128000 \
-ctk turbo3 -ctv turbo3 \
--port 8000 --host 0.0.0.0 \
-fa on -np 1 \
--cache-ram 4096 \
--ctx-checkpoints 4 \
--fit on \
--reasoning off
English
@TheAhmadOsman Personally I don’t care it is Chinese , American , French or whatever , I just want an LLM running on my 3090 and able to gen quality code like sonnet 4.5 or Opus 4.6. For now I am using Qwopus3 Q5 + turboquant llama.cpp version but happy to try Gemma if better
English