Jared Sleeper

Many investors (including me!) are embarrassingly imprecise when they use AI terms. So I'm starting a daily series where I chip away at fixing that for myself. If you're like me and you use AI a lot/want to deepen your understanding, follow along. :) Day one: tokens. "Tokens" are a critical unit in AI, because they measure context, latency and model usage and relate to costs. Under the hood, tokenizers were built by finding common character sequences. If the word "the" shows up frequently, it might be represented as a single token. In practice, a token can be a word, part of a word, a set of characters, a punctuation mark, a space, a code fragment, etc. Critically, tokenizers are typically model-specific. This has important context-engineering implications- the same prompt can consume more tokens in one model than another, leading to tradeoffs. One distinction worth covering is input vs. output tokens. The words are somewhat self-evident, but output tokens cost more because input is processed at once in a parallel pass, while output tokens are generated one at a time. As a result, output tokens typically cost 3-6x as much as input tokens. On the input side (the meat of today's post), tokenizers convert language into tokens, each of which is mapped to a numerical ID (e.g. 1431), which corresponds to a vector stored in one row of the model's embedding table. This process itself is not particularly computationally intensive. The result looks like this: 1431 → [0.17, -0.42, 0.08, 0.91, ...] A modern model like GLM 5.2 might have 6,144 numbers per token (i.e. the vector above would be 6,144 dimensions). Others have 2-3x more than that. Closed models generally don't publish the number of dimensions they encode. Confusingly, each number means almost nothing on its own- meaning is encoded in patterns and directions across many of the values at once. We could imagine that say, the first number is "gender" and -1 = male / 1 = female, but in practice most concepts are captured diffusely across the vector's values, in part because there are obviously far more than 6,144 concepts in a given language. So the concept "queen" might involve a constellation of features or relationships distributed across the embedding's dimensions, including dimensional patterns that roughly correspond to "female," "ruler," "monarch," etc. This is why human-legible interpretability of embeddings is hard. Dimensions are one aspect of what makes a model "good" but they come with tradeoffs as well (including requiring dramatically more compute). Improving models is much more complicated than simply increasing dimensionality. As it stands, there's a clear problem: words have different meanings depending on context. So even a 6,144-value vector of the word "bank" is context-independent, which matters because "bank" has multiple definitions. This gets us to the core of the LLM innovation: attention and transformers, which let us pull information from the representation of the other tokens around a given token to update its vector. That's day two.

Today three pinned browser tabs became one and I've officially churned from Notion, Roam, and Airtable in favor of an app I built. IMO if you are the sort of person who loves tinkering with your personal software stack (Roam/Notion/Airtable/Evernote/etc.) building your own is a no-brainer today. Now I have: 1) An agent-managed personal CRM (Airtable) 2) A note taker w/tagging, AI search, email ingest, etc. (Roam) 3) A wiki (Notion) All on one platform/browser tab, all accessible via a Telegram agent, Claude and a web UI. All with unique to me features/functionality like: 1) Ingest of contacts from Partiful screenshots 2) Headshots as a core feature of the CRM 3) Accommodation for my poorly tagged legacy Evernotes base 4) The fact that I use X follow-backs as a way to denote people I might want to meet one day, etc. And incremental features/tweaks are just a few minutes away. I still don't think this applies to software shared across teams for various (hopefully obvious) reasons, but the era of power users using packaged software for mostly personal use is over.

Role reversal: The Americans use chopsticks while the Chinese do not.