I keep hearing people repeat the zombie factoid that tokens are subsidized and users aren't paying the full cost. This is mostly false — major labs have positive margins. But even where it's true, it doesn't matter very much because costs are falling rapidly.

If costs are falling 10x a year, it doesn't matter that much whether you are paying full price or half price this year — either way you are going to pay less (for a comparable performance level) next year.

@binarybits API or subscriptions? Subscriptions are either subsidized or work on most people not hitting their true limits. API is not subsidized at all

With access to X and web data, Grok is the AI assistant with the most up-to-date information, keeping you informed about the latest news and trends in the world, for any given topic.

@binarybits Are there any good pieces covering this? I see a lot of takes, but not a lot of data backing it up

@binarybits What does the curve look like of cost vs token verbosity. How relevant do you think it is that models now are much more verbose?

@binarybits It’s true in the context of token resellers like Cursor. Their business depends on selling Anthropic tokens for 60-90c on the dollar, then raising massive amounts of money to train their own models and attempting to swap them in for Anthropic tokens

It can be true both that labs have positive margins and that they've market-segmented their enterprise customers into paying so much that personal subscriptions are paying less than the cost of inference. (I checked the logs, my usage of my Codex Pro subscription would have been $1100 last month purchased as credits, even more at API pricing.)

@binarybits The labs are absolutely not positive margin - their 'positive' margins completely ignore training costs. This is like talking about oil and gas margins while ignoring the cost to drill the well. You can't use a software type margin analysis here!

@binarybits My small business consumes about $100+/day and the prices to us are amazingly low. So it *feels* subsidized. We'd pay 5x the price, easily.

@binarybits @robertwiblin Do they have positive cash flow, though? Wilting off data center investments over six years looks good now, but will torpedo margins in a few years when they are still only half way to writing off then near-worthless equipment

@binarybits People are confusing API prices with actual underlying costs of provision, which are probably OOM smaller.

@binarybits Soon we'll have local agents without the need for the big AI companies :)

How do you scale from a single server to a massive AI cluster? Supermicro In-Rack Solutions integrate liquid cooling, high-performance networking, power delivery, and battery backup to support rack density, thermal efficiency, and resilient AI infrastructure.

@binarybits Token cost curves are probably the strongest anti-doomer argument. Even bad margins get rescued fast when inference keeps getting cheaper.

@binarybits per-token cost keeps falling but agents burn through so many more that the bill doesnt actually shrink

@binarybits 100%