Will Killebrew

To build personal superintelligence, our model’s capabilities should scale predictably and efficiently. Below, we share how we study and track Muse Spark’s scaling properties along three axes: pretraining, reinforcement learning, and test-time reasoning. 🧵👇 Let’s start with pretraining. Over the last 9 months, we rebuilt our pretraining stack with improvements to model architecture, optimization, and data curation, enabling us to increase the capability we can extract from every unit of compute. To rigorously evaluate our new recipe, we fit a scaling law to a series of small models and compare the training FLOPs required to hit a specific level of performance. The results: we can reach the same capabilities with over an order of magnitude less compute than our previous model, Llama 4 Maverick, making Muse Spark significantly more efficient than the leading base models available for comparison.

Total amt of flops across all the GPUs in the world has grown about 3x per year for the last few years. Total amt of inference demand has probably grown ~10x per year. What happens when those lines cross? The econ answer is: when demand > supply, price goes up. That might be true (even H100s are more expensive than they have ever been...) but doesn't actually solve the problem on its own here - demand continues to grow as we unlock new use cases and there is only so much additional supply coming. To get to a healthy equilibrium, we also need to shift much more usage to smaller, targeted models. This will happen naturally as the incentives make sense for it: it's much easier to 10x your agent usage if you know that you can now solve 90% of your tasks with good cheap, fast models. SWE 1.6 is not a general model. But we have trained it to specifically be good at the kinds of standard coding tasks that our users run. And thanks to its small size & a little magic from Cerebras it can run very cheaply at ~1000 tokens / sec. Give it a try and let us know what you think!

NEWS: Waymo and Waze have announced a partnership to help cities detect and repair potholes faster by providing data from their autonomous fleet. The pilot will launch in these cities first: • San Francisco Bay Area • Los Angeles • Phoenix • Austin • Atlanta "The pilot program uses Waymo’s perception and physical feedback systems to detect and provide up-to-date information on potholes where Waymo operates. The data will be available to cities and state Departments of Transportation through the free-to-use Waze for Cities platform alongside user-reported pothole information, giving officials an additional view of surface street and highway conditions that enables them to more efficiently and effectively fill potholes. The data will also be visible to Waze users in the cities where Waymo operates, keeping road users safe by alerting them as they approach a pothole. Like other on-road features reportable in the Waze app, users will be able to verify the Waymo-identified potholes, increasing the data’s accuracy. Waymo has already identified approximately 500 potholes. Over time, we’ll expand this partnership to more cities we serve, including those with winter weather and harsh freeze-thaw cycles that exacerbate the pothole problem."

Opus 4.6: I did not find any vulnerabilities. Developer: Phew. Mythos 1.0: I found 3 critical vulnerabilities. Developer: Oh no. I'll fix them. Mythos 2.0: I found 8 critical vulnerabilities. Developer: Oh no. I'll fix them. Mythos 3.0: I found 35 critical vulnerabilities. Developer: Oh no.

Overheard in Silicon Valley: "LinkedIn is prison for middle managers."

Sometimes i see things and think, why did it take sooo long