Chris Simpson

The recipe: 1. Solve Search. 2. Use it to solve everything else.

This "Aha moment" in the DeepSeek-R1 paper is huge: Pure reinforcement learning (RL) enables an LLM to automatically learn to think and reflect. This challenges the prior belief that replicating OpenAI's o1 reasoning models requires extensive CoT data. It turns out you just need to give it the right incentives. We are so back in the AlphaGo excitement era: by playing countless Go games and maximizing the reward function (winning the game) using pure RL, AlphaGo beat the best human players. Now we are entering the LLM RL era. 2025 could be the year of RL.

How to train a 670B parameter model. Let's talk about the DeepSeek v3 report + some comparisons with what Meta did with Llama 405B

I'll get straight to the point. We trained 2 new models. Like BERT, but modern. ModernBERT. Not some hypey GenAI thing, but a proper workhorse model, for retrieval, classification, etc. Real practical stuff. It's much faster, more accurate, longer context, and more useful. 🧵

Never a dull moment in AI. Current market summary: * Big, bigger, biggest: biggest VC round ever (OpenAI), biggest seed round (Safe SuperIntelligence, $1B), biggest acqui-hire (Character, $2.7B), biggest super computer (xAI's Colossus, etc) * Rich valuations all around: NVIDIA at 65 P/E (vs 20 average), several startups at multi-billion valuations pre-revenue (sometime pre-product). Can argue whether deserved or not, but lots of growth expectations built in * Fortunately, this generation of AI startups is growing faster than their SaaS cousins ever did (source: Stripe data) * Despite concerns about over-building of AI infra, Mag 7 companies in recent earnings all attest to very strong (or even "insane") levels of demand for AI. Numbers are comparatively small by Mag 7 standards, however. * Big question around continued progress of AI research/performance, or not. Concerning reports that GPT-5 may not show as much exponential progress as prior versions did. Question around reasoning for LLMs. Fuziness around what AGI and ASI actually mean. Optimistic view is that great products/companies can be built deploying what we currently have, even if progress slowed down dramatically * Foundation/frontier LLM market seems to have crystallized around a few early winners (unclear whether any can build long term differentation). Open question about the impact of Llama 4 and open source on commercial API providers. Plenty of room still for specialized LLMs, per modality (audio, video) or vertical (bio, materials sciences, etc) * AI tooling layer (orchestration, evaluation, RAG, etc) interesting, but crowded and fast evolving * AI consumer application layer wide open. Every paradigm shift saw the rise of dominant consumer companies. Who will build the Uber of Generative AI? * From a customer perspective, enterprise (Global 2000) adoption of AI is still very early: mostly low hanging fruit kind of use cases: chat (GPT/Azure), search (Glean), code (Github CoPilot), or easy to deploy applications (Synthesia for video). For anything more company-specific or industry specific: consultants are the big winners so far. Next: enterprises will go from PoC to production for custom use cases. * From a vendor perspective, AI (intelligence wrapper) is the new SaaS (database wrapper). Furious pace of building across use cases (AI for finance, sales, marketing, custome service, HR etc) and verticals (law, accounting, banking, etc). New business models ("sell the work" vs just selling software) * Brave new world ahead of us. In the enterprise, could mean AI agents connecting with each other to orchestrate tasks end-to-end. Tomorrow's enterprise could need a lot less people and jobs would evolve to managing a few humans and a lot of AI.