
Axel Vanraes
182 posts

Axel Vanraes
@AxelVanraes
Engineer. Data Science. Taking clinical reporting to the next level. RWE for better healthcare.


To help explain the weirdness of LLM Tokenization I thought it could be amusing to translate every token to a unique emoji. This is a lot closer to truth - each token is basically its own little hieroglyph and the LLM has to learn (from scratch) what it all means based on training data statistics. So have some empathy the next time you ask an LLM how many letters 'r' there are in the word 'strawberry', because your question looks like this: 👩🏿❤️💋👨🏻🧔🏼🤾🏻♀️🙍♀️🧑🦼➡️🧑🏾🦼➡️🤙🏻✌🏿🈴🧙🏽♀️📏🙍♀️🧑🦽🧎♀🍏💂 Play with it here :) #scrollTo=75OlT3yhf9p5" target="_blank" rel="nofollow noopener">colab.research.google.com/drive/1SVS-ALf…



















🔥 New (1h56m) video lecture: "Let's build GPT: from scratch, in code, spelled out." youtube.com/watch?v=kCc8Fm… We build and train a Transformer following the "Attention Is All You Need" paper in the language modeling setting and end up with the core of nanoGPT.















