Random Libertarian Tech Lead

@FelixKerlin @cryptopunk7213 Savings per task would be a concise way to express it but then it would not be immediately clear that it's relative to only the most expensive model on this chart (rather than being some arbitrary metric, or relative to the most expensive model in existence).

@FelixKerlin @cryptopunk7213 Since you already have a ceiling value that's constrained to this specific set of models, you could just frame it as "cost savings relative to the most expensive"... But that's difficult to communicate concisely in that amount of space.

1. theres no way this beats opus 4.6 2. wtf is this chart crime??? 😂

@pirosb3 @fynnso Could be anything, really. If they're working with them directly, it could even be continued pre-training.

@someRandomDev5 @fynnso yeah good point, what form of fine tuning you think they ended up using?

@wesbos ... And there are some leaks which seem to suggest that Cursor's Composer model is no different; it seems that it may just be a fine-tuned version of Kimi K 2.5:

@wesbos Opus and GPT 5.4 are both highly intelligent. But Opus has the edge in understanding people, and GPT 5.4 has the edge in understanding systems. Nearly every open-weights model is basically just a distilled fine-tuned Frankenstein of GPT, Claude, and Gemini.

Composer 2 vs Opus 4.6 vs GPT 5.4 - a totally unscientific test > Create a Twitter clone. Use Better Auth, Vite, Sqlite, Drizzle, Typescript, and React with Tanstack Start. Each one took ~5 mins in plan mode. Each had access to a browser to test. Composer: 5 mins $6.04 1,250 LOC Opus: 19 mins $10.43 1,000 LOC GPT: 22 mins $14.15 2,000 LOC Opus seems to use the cache WAY more than composer, so it's not really 10x more expensive. Composer app ran first try. Other two needed a bit of CORS debugging but did work. Code between all was extremely similar All three done inside cursor - so Claude Code / Codex may have been different Models used: Composer 2.0 (regular, not fast) Opus 4.6 Medium Thinking GPT 5.4 Medium Thinking

@wesbos If you want an interesting test that actually demonstrates brownfield development (without requiring too much set up on your own part), ask the model to expand or modify the behavior of a large complicated open-source project that you actively use.

@wesbos That said, this greenfield test does do a fairly good job of demonstrating how well these models can replace design agency work. If all you're building are businesscard-like or brochure-like landing pages, then sure, this test is fine.

@pirosb3 @fynnso You’re comparing “all forms of fine tuning” to “a specific form of fine tuning”.

@fynnso In this scenario is it actually fine tuning or is it a LoRa adapter? I’m a little new to these concepts but very curious.

was messing with the OpenAI base URL in Cursor and caught this accounts/anysphere/models/kimi-k2p5-rl-0317-s515-fast so composer 2 is just Kimi K2.5 with RL at least rename the model ID

@BlackHC TLDR: How did you know that software was good before the era of AI? Pure vibes. How do you know it's good after the era of AI? Still pure vibes, nothing has changed.

@BlackHC Again, prompt engineering is an actual skill set and there's actually a lot of nuance to it despite people ridiculing it. "Building an intuitive sense of the things that the LLM can actually infer accurately through language" is a key part of that.

A while back, Andrej Karpathy said the app store will be replaced by generated, disposable software," and Amjad Masad predicted that the value of all application software will go to zero I think this "ephemeral software hypothesis" is wrong, though, and I want to explain why: