Sawyer Bowerman

@ZQ_Dev @RedHat_AI @vllm_project @GoogleDeepMind @UnslothAI Yep! Thats a great question! huggingface.co/RedHatAI/gemma… We are constantly updating the models as updates come/are required, but you can see the chat template updates there. Thanks for reaching out!

@RedHat_AI @vllm_project @_soyr_ This is awesome! Is @RedHat_AI watching for and pulling in chat template fixes from @GoogleDeepMind and @UnslothAI for these? I think there were some upstream changes for better tool calling, agentic use, etc. made in the past few days.

What compression looks like on @vllm_project. Same Gemma 4 31B. Red Hat AI's quantized version runs at nearly 2x tokens/sec, half the memory, 99%+ accuracy retained. Open source. Quantized with LLM Compressor. Links in comments. 🙏 @_soyr_ for the 2-minute demo.

@RedHat_AI @mgoin_ @vllm_project the true cracked hacker!!

Michael Goin (@mgoin_) walks through what's new in @vllm_project v0.17, v0.18, and v0.19 in ~8 minutes. Flash Attention 4, new performance modes, zero-bubble async scheduling, online MXFP4 quantization, Gemma 4, and a lot more. 1,592 commits. 682 contributors (163 new). 🎉 🚀

As AI and LLMs become more and more powerful, the boundaries for efficient inference simply break down. If you want to experiment with quantized models as well, you can follow the code in the video, and check out huggingface.co/RedHatAI to pick any model you'd like!

The Wildcard Ever wanted to do your daily stand-up in @Minecraft? @_soyr_ shows us how (and why!) they did it. Don't miss these sessions, May 6-7! Register here: devoxx.co.uk