@bnjmn_marie Tested it on my two RTX 5000 Pros, one model on each, both Gemma 4 31B, both FP8, one with and one without MTP.
The 3x was no lie. Went from 30 to almost 100 tok/s. That's incredible.
English
Kevin
4.2K posts
Kevin
@TheOneKev
Product Owner building independent AI systems. Interested in psychology, politics & the global economy
เข้าร่วม Haziran 2020
475 กำลังติดตาม179 ผู้ติดตาม
Gemma 4 was very slow compared to Qwen3.6. Now, it's probably much faster!
I'll publish my own numbers tomorrow
Google for Developers@googledevs
Gemma 4: Now up to 3x Faster. ⚡ Same quality, way more speed. Our new MTP drafters allow Gemma 4 to predict multiple tokens at once, effectively tripling your output speed without compromising intelligence.
English
Ok, had to try it out. Doing the first runs, both FP8 and each running on a RTX 5000 Pro.
And what can I say? hey weren't exaggerating. ~30 tok/s vs almost 100 tok/sec.
Which meant in that test run reducing the time from 16 to 5 secs. And I can't see any degradation or similar.
Great job @googlegemma
Google Gemma@googlegemma
Gemma 4 just got even faster! We're releasing Multi-Token Prediction (MTP) drafters that deliver up to a 3x speedup, without any degradation in output quality or reasoning logic.
English
Kevin รีทวีตแล้ว
Gemma 4 just got even faster!
We're releasing Multi-Token Prediction (MTP) drafters that deliver up to a 3x speedup, without any degradation in output quality or reasoning logic.
GIF
English
If anyone wants to do this this summer I have giant villa on the Amalfi Coast & Lake Como that I’ll be hosting hackathons from.
ol’ stocky ⛳️@oldstocky
Your soul needs this, not the Met Gala
English
@TheOneKev This is the pro move brother! You gave me inspiration to move to the next level.
English
What LLM gateway are you using?
I built my own to have a single point of configuration for my local AI systems.
How did you all solve the pain point of having many local models over many local machines.
English
@Dozer3000 @gas0linr Stimme ich halbwegs zu. Zumindest was den Status Quo betrifft. Aber wenn man überlegt, wo LLMs herkommen und dass Scaling bis jetzt immer noch was bringt, abgesehen von evtl neuen Architekturen bald, wird bald kein Dev mehr mithalten können. Biologisch nicht möglich.
Deutsch
@gas0linr Die Aussage ist meines Erachtens völliger Quark. Das Studium lohnt sich. Nur weil man mit vibe-coding mit ein paar Agenten effizienter arbeiten kann, werden gute ITler nicht obsolet. Reine Script-Coder haben es schwerer, aber sonst gibt genug zu tun.
Deutsch
Es ist ein wirklich irres Gefühl als Informatiker zu sehen, was der technologische Fortschritt mit der Branche macht. Und es ist beängstigend zu erkennen, dass 90% der Menschen das nicht kommen sehen.
Als Student der Informatik würde ich jetzt (!) abbrechen und mich orientieren.
Deutsch
Sometimes it's hard to understand, when you're right in the middle of it, but this is literally history in the making. And I honestly think just a precursor of what will come.
I think @sama even said it himself, that it will probably get bad first, before it can(!) become good.
Anonymous@YourAnonNews
Kevin O'Leary's massive data center was approved by a county commission in Utah last night without residents' approval of the measure. At 40,000 acres, it would be 2.5x the size of Manhattan. The commission approved the proposal despite opposition from hundreds of locals.
English
@isabelunraveled As a dad, makes me happy reading that reply from you dad. That's how it should be. He's doing a great job.
English
@TheoMediaAI @sama I mostly agree. The only things that come to my mind in that scenario is b) for how long (driving) that will still be a thing, but even more b) Augmented Reality, e.g. HUD. Great combo.
No doubt though, voice only will have it's use cases. Just not standard standalone UI.
English
@TheOneKev @sama Agreed, but also: Driving.
So, yeah: Voice only has a place.
English
pretty excited for voice models to get great
its interesting to watch how people are already starting to change the way they interface with AI
English
@TheAhmadOsman Ngl, that 10x tokens thing...they really know how to get me. Tokenite 😄
English
People keep treating everything like isolated events
- Dario / Anthropic fearmongering
- Policy maker pressure
- Elon’s lawsuit
- Sudden 10x tokens
- SF parties
All just random coincidences? Come on, look more than 2 steps ahead
We’re surrounded by existential risks & Psyops
English
Shout-out to @OpenAIDevs and @sama
I was expecting a stream or something, but this is no doubt a more than pleasant surprise. Would have absolutely preferred to be there, but this is a nice band-aid.
And I'm definitely gonna use it to the max 😈
English
@Beever_AI @TheAhmadOsman Looked interesting, just visitied the repo. Maybe I'll add it to my own implementation. I am personally going a little different route.
Asked my operator to compare itself vs Beever.
English
@TheOneKev @TheAhmadOsman Beever Atlas — chat-native ingestion → typed knowledge graph + MCP-ready memory layer. Apache 2.0. github.com/Beever-AI/beev…
English
Never heard back btw, guess my original intuition about not being invited was right 🤣
Sam Altman@sama
@TheAhmadOsman War is peace. Freedom is slavery. Ignorance is strength. oh wait, we don't believe any of that. how about we democratize a lot of super capable AI, and then we sit back and watch you build the future?
English
@TheAhmadOsman Ah, c'mon. Pretty sure that's just a capacity problem 🙂 As the unofficial representative of the local LLM and buy a GPU gang, you'd be a good addition to the party.
English
@TheOneKev The only difference is that I am already in the Bay Area on a work trip and would easily make it but just doubt the OpenAI folks would enjoy having me over 😂
English
Be honest — how many local models do you have downloaded right now? 👀
English