avery.apt 🇺🇸@AveryChing
My hot take: Open-source AI models will dominate most general consumer usage in a few years.
Why? Open source models are improving incredibly fast (e.g., see the new Qwen 3.5 models competing head-to-head with Opus 4.6 and GPT-5.2 on several benchmarks) and are closing in on frontier models. The latest research by the Linux Foundation says the gap from closed source to open models is down to 13 weeks from half a year in 2024.
All consumer devices are being designed with inference in mind, with high-end even more so. Mac’s M5 Max is up to 4x faster on LLM prompts than M4 Max. In say 2 years, with a combination of hardware and models improving by 10x means that a high-end MacBook Pro and/or top-tier iPhone will be able to easily handle inference tasks better than today's Opus 4.6. Most likely, even mid-tier devices will be perfectly fine for typical use cases.
Of course, frontier models will improve rapidly and continue to maintain a declining quality benefit over open-source models. The quality gap will close and when 95%+ of the common work we need for LLMs can be handled by open-source models, the other benefits of open-source models will be the differentiator. Cheaper in cost (mostly just electricity at that point), uncensored, and private.
There is a gap in software to make the transition from frontier models to open-source models completely seamless, especially as good as Claude and OpenAI are today. Long-running agent infrastructure needs to be fully developed and polished. But those gaps will easily be closed (not unironically) with AI coding. As you can see, I’m bullish on open-source AI models and the future of private, personalized AI.
I’d be remiss to not mention that open-source and private AI will have a lot of interesting storage and serving workloads. Global, scalable, private data is the key to good decision-making, fine-tuning, and context. But of course, @shelbyserves.
