Entrée Capital

🤔 Some thoughts on an architectural shift in AI we’ve been seeing in the past months: We're at the beginning of a significant architectural change in how AI actually gets deployed. The dominant approach today is to route everything to a massive cloud LLM, burn tokens, pay per call and this can’t continue due to latency, cost, privacy, dependency and a host of other reasons. Most of the work will shift to the edge. Fit-for-purpose models which are small, specialised, and trained on your domain or workflow will run on-device or servers close to it. It’s fast, cheap and private. The big LLMs (Anthropic, OpenAI, Google) will shift to a ‘backstop’ role for many use cases. They will be the confidence layer and called for outlier requests, complex reasoning, cross-domain synthesis i.e. the things that genuinely need 70B+ parameters. The result will be a hybrid inference architecture. Local models handling ~80% of requests. Some knock-on effects: - Cloud inference costs will drop sharply at scale. - Datacenter and hyperscaler demand for AI inference will plateau sooner than the current buildout assumes. - Open source is a winner in this race: Llama, Mistral, Phi, Gemma and new models will become the default inference layer for most enterprise workflows. - The moat will shift from "biggest model" to "best fine-tuned specialist". Apple Intelligence is already at this architecture. Microsoft Phi-4 runs locally on a laptop. Inference is moving to the edge. We are seeing some of our portfolio co’s adopt open source models, edge inference, multiple LLM orchestration and so on.