Thomas Beverley

Today, there is a small Wavebox update to version 10.134.18. It contains important updates to Chromium 134.0.6998.166 and some other fixes for reported bugs. Meanwhile, @thomas_101 is busy working on some bigger extension work under the hood! 🏃‍♂️

@xmlking @jason_mayes @webllm of native implementations. AiBrow has a standard API for using the ext or WebGPU depending on what you need. All this said, I think it's too early to say how compatibility will work out, but we're fortunate that we already have different impl options even at this early stage!

@thomas_101 @jason_mayes do you see any standardization (JavaScript API) efforts on how web developers use on-device AI models? I see 1. Chrome team trying to bundle Gimini Nano in the browser. 2. @webllm trying to to offer OpenAI API Compatibility What would be the future for WebAI? can you share your thoughts?

@xmlking @jason_mayes @webllm This is part of the reason I published the AiBrow extension (github.com/axonzeta/aibrow) as it allows you to specify the model and ensure you're using the one that works best for your use case. You can of course do this with WebGPU/WebML but performance is about half that...

@xmlking @jason_mayes @webllm For pre-canned human tasks like summarize I think you can swap models out without much trouble. It's when you're using the LLM as more of a complex system like in an agentic agent where the LLM can make or break a system...

We got GMail (full interface) working in @ladybirdbrowser, let's fricken gooooooo! 🚀 This one is nice because I've been told by many people "you'll never be able to handle modern web apps like GMail" Don't listen to armchair defeatists. We can build whatever we want! 🤓

Dang I'm excited about new semantic attr() capabilities in CSS! I wrote a post about why this feature is so powerful, how it could change the way you build components, and how I would use it. Advanced attr() is rolling out in Chrome 133 starting next week 🥰 una.im/advanced-attr/

@jason_mayes @Google …it’s all excited about translating something and not chatting with you!

@thomas_101 @Google Ah! So having it in that format allows you to force it to do the things you want - got it!

Engineers, creatives, designers - lend me your ears, and eyes, to check my🥇1st tech demo of 2025: A #WebAI Agent running entirely client side in the browser, that's capable of controlling a flight search webpage, to get the job done! I'm using @Google's Gemma 2 (2B) model in #JavaScript via #WebGPU thanks to the MediaPipe Web LLM library, combined with some extra function calling logic to enable advanced user experiences. If you like what you see, give this post a share, as I'm just one guy trying to show the world the power of Web AI to do great things. Try it for yourself using the link below after watching the video, but do so on a modernish machine (I have verified it runs on my 6 year old Dell XPS with only an Intel integrated GPU) but use a modern browser like @googlechrome as it needs WebGPU to run. Want me to make a demo for your industry? Tell me what you need / want to see next! Let me know what you think, and if you have any questions, then drop them in the comments below! YouTube Video Link: youtube.com/watch?v=IC256K…

@jason_mayes @Google …but using some grammar helped to constrain it to just the translation. #L117" target="_blank" rel="nofollow noopener">github.com/axonzeta/aibro… a little bit like pre LLM chat when it was just completion and you could steer an LLM by starting its reply because by the time it gets to the end of the “translation-fr” key…

@jason_mayes @Google Exactly! I’ve also found you can steer the output a bit more using grammar like this. We played about with getting a LLM to translate text and it would be quite chatty about the whole thing “Sure here’s your text in French…”

So much is shipping in Chrome 133 (stable rollout begins in 2 weeks!) A couple of big ones: 1. Scroll-state queries: check if something is stuck and where — developer.chrome.com/blog/css-scrol… 2. Extended functionality for attr(): assign types beyond strings to attributes and use them in your CSS — developer.chrome.com/blog/advanced-… 3. Text-box trim: control the space above and below text — developer.chrome.com/blog/css-text-… (Honorable mention). Hint popover type: open an unrelated popover while an existing stack popovers stay open — chromestatus.com/feature/507325…

@jason_mayes @Google It's great to chat! Sorry I vanished last night, I'm in the UK so it was time for some 😴 🙂

@thomas_101 @Google PS thanks for the discussion appreciated.

@jason_mayes @Google There is a paper about the methodology here arxiv.org/abs/2307.09702. The extension uses llama.cpp which has an implementation of it with it's Grammars API github.com/ggerganov/llam…

@thomas_101 @Google Understood but how are you enforcing that on the LLM - some custom integration with popular models? I noticed you had to use an extension to make your version work Vs using my Gemma 2 model out of the box for example?

@jason_mayes @Google It works the same with JSON, although after a closing quote the most likely token is a full stop, the grammar filters this out so the LLM only has a choice between comma for another property or } to close the object

But how are you forcing the LLM that is any LLM to generate what you also arbitrarily defined? I had to use the prompt to do that but you are somehow whispering to it at runtime to tell it to go back and generate valid token and then continue? That is what I am not understanding how you are pulling that off? Sorry if it is obvious and I am missing something.

@jason_mayes @Google …more likely to pick one that’s a lower probability (e.g fries -> jelly). The grammar rules filter out the available next tokens before the LLM can select them, so jelly is never available.

@jason_mayes @Google Think of it similar to how temperature works, when it picks the next token the LLM may be presented with 5 possible tokens each with varying probabilities. A lower temp makes it pick the highest probablity next token (e.g fries -> ketchup), a higher temp makes it…

@jason_mayes @Google It’s okay. Hard to explain in short tweets :-). It works by changing the next token selection logic, so the model is only able to pick tokens that fulfill the grammar rules.

@jason_mayes @Google Because of this the LLM wouldn’t really miss a comma, because that wouldn’t be considered a valid token when generating the next. The same with the end of the output, normally LLMs use <end_of_turn> or similar but this would only be valid after the closure of the object

@thomas_101 @Google Ah I see so it acts like a filter on any LLM output to remove any chatty extra stuff. What happens if it forgets a comma in the JSON? It would fix it or output what it can extract from the output into the known JSON grammar you defined?

@jason_mayes @Google After that, freetext tokens to make the key. Until eventually the LLM sees a close quote as the most probable next token, moves on to a : and then a number/array/object/string etc.

@jason_mayes @Google When it’s calculating the probability of the next token, those that wouldn’t be considered valid by the grammar are not included in the options the LLM can consider. If you’re building an object, this means that an output of { would only really consider “ as a next valid token.