NK

it's unclear if tech people yet realize that perspectives on ai usage differ dramatically depending on if you are token rich or token poor. the idea of looping comes from a place of incredible privilege. recommending that someone who works at an ai startup with unlimited token budget try looping makes complete sense. however, that same idea told to someone with only a $2k/mo token budget would seem completely implausible. what happens when millions of people start evaluating jobs based on what their token allowance will be? what happens to the longtail of administrative paper pushers that don't adapt? what happens to low throughput artists, engineers, or even lawyers and actuaries that can't be trusted to allocate spend? every form of individual knowledge worker is going to be evaluated from the perspective of "do i trust this person to allocate tokens effectively?" there are going to be massive investments into new hiring processes. organizations will need to rearchitect their structures to find orchestrators like Peter my near term recommendation is to do everything in your power to work somewhere that trusts you to spend tokens. you're ngmi if you only have $2k/mo to play with.

Token costs are becoming one of the hottest topics for any enterprise I talk with right now. It’s very bullish for AI in general because it means these systems are being used at a scale that wasn’t contemplated before. It also gives way to another form of differentiation that will emerge for the applied AI layer, which is model routing. As tokens take on a significant amount of the cost of any given workflow, then companies will inevitably want to ensure that their dollars go into the most efficient use of tokens for the particular job at hand. Frontier intelligence will always be relevant at the high end of tasks, like coding, legal and financial analysis, healthcare, and more. And dollars spent here will only go up over time. But, equally, you can peel off individual tasks to lower cost models (whether they’re from open weights vendors or the major labs) and deliver a more efficient end outcome. To do this effectively, the applied AI layer needs to understand the workflows in their domain better than anyone else, and be able to mix and match models to different jobs. If you’re doing document extraction, you need to know which models perform better or worse for any given document type. If you’re legal analysis, you want to know which models perform various types of tasks best. And so on. This will become one of the bigger differentiation points over time. The companies with the best evals, the best ability to route the workloads, and those that have business models directly aligned to customers financial goals, will be in a great position.