Matt

14

安叫兽|Bird🕊️ 🔶 BNB@ajs6888·2h

@codermatt @deepseek_ai @OpenAI @SpaceXAI @pidotdev Grok 这成本差距有点离谱了

中文

0

1

38

Matt@codermatt·3h

I gave 3 LLM models the same brief: Build a simple CRM platform based on a design spec and example image. - @deepseek_ai v4 Flash 0731: $0.081 - @OpenAI GPT 5.6 Luna: $0.068 - @SpaceXAI Grok 4.5: $0.750 All run via the @pidotdev harness Videos and images below 👇

English

3

0

8

607

Matt@codermatt·59m

@tushar_koshti @deepseek_ai @OpenAI @SpaceXAI @pidotdev Thanks for the kind words. Anything else you would like to see?

English

1

18

Tushar Koshti@tushar_koshti·2h

@codermatt @deepseek_ai @OpenAI @SpaceXAI @pidotdev This is a nice comparison between #deepseek and #luna and #grok

English

0

1

52

Matt@codermatt·3h

@anrayama For sure! Luna did really well here.

English

8

Anra@anrayama·3h

@codermatt luna on top with this cooking

English

0

6

Matt@codermatt·1d

I gave Luna 5.6 (Max reasoning) and DeepSeek V4 Flash-0731 the same challenge: build Minecraft from scratch. Luna: 343K tokens · $0.35 DeepSeek: 366K tokens · $0.25 Luna absolutely crushed it. DeepSeek wasn’t bad, but Luna’s game looked and felt far better. Videos and prompt below 👇

English

42

10

320

63.6K

Matt@codermatt·3h

@Chris76822191 It’s pretty close between flash and Luna! I’ll let you guys be the judge

English

9

Jason@Chris76822191·3h

@codermatt Yeah that's great. So you'd give the edge to Flash right?

English

0

11

Matt@codermatt·3h

@SergioM80274824 Real world test here: x.com/codermatt/stat… Let me know your feedback

I gave 3 LLM models the same brief: Build a simple CRM platform based on a design spec and example image. - @deepseek_ai v4 Flash 0731: $0.081 - @OpenAI GPT 5.6 Luna: $0.068 - @SpaceXAI Grok 4.5: $0.750 All run via the @pidotdev harness Videos and images below 👇

English

5

Sergio@SergioM80274824·1d

@codermatt stop using this model for this kind of thing, compare them for web dev and that kind of stuff, it doesn't seem to have been trained for 3d nor games.

English

2

0

7

1.6K

Matt@codermatt·3h

@Chris76822191 Did a different test with a real world case here x.com/codermatt/stat…

I gave 3 LLM models the same brief: Build a simple CRM platform based on a design spec and example image. - @deepseek_ai v4 Flash 0731: $0.081 - @OpenAI GPT 5.6 Luna: $0.068 - @SpaceXAI Grok 4.5: $0.750 All run via the @pidotdev harness Videos and images below 👇

English

4

Jason@Chris76822191·1d

@codermatt Most people aren't using these models to build games. Curious why you don't bench real world use cases?

English

4

0

6

1.7K

Matt@codermatt·3h

@vokaysh @WescheNex1q @blueemi99 Maybe check this one out also. Could be of interest. x.com/codermatt/stat…

I gave 3 LLM models the same brief: Build a simple CRM platform based on a design spec and example image. - @deepseek_ai v4 Flash 0731: $0.081 - @OpenAI GPT 5.6 Luna: $0.068 - @SpaceXAI Grok 4.5: $0.750 All run via the @pidotdev harness Videos and images below 👇

English

10

🏴‍☠️jake@vokaysh·10h

@WescheNex1q @blueemi99 oh look, exactly what I said x.com/codermatt/stat…

I gave Luna 5.6 (Max reasoning) and DeepSeek V4 Flash-0731 the same challenge: build Minecraft from scratch. Luna: 343K tokens · $0.35 DeepSeek: 366K tokens · $0.25 Luna absolutely crushed it. DeepSeek wasn’t bad, but Luna’s game looked and felt far better. Videos and prompt below 👇

English

🚀 DeepSeek-V4-Flash Official API is now LIVE in public beta! 🔷 We’ve massively upgraded its Agent capabilities—benchmark scores are now far surpassing the V4-Pro-Preview. Check out the massive performance leap below! 👇 🔷 The official V4-Flash now natively supports the Responses API format and is fully adapted for Codex! Check out the configuration details in our official API docs: api-docs.deepseek.com/quick_start/ag…

0

33

bluedev@blueemi99·2d

5.6 Luna still beats that, and is almost free too btw

DeepSeek@deepseek_ai

English

102

3

444

84.4K

Matt@codermatt·3h

@rennf93 Very true! Ai makes us so fast right now.

English

5

Renzo@rennf93·4h

@codermatt And fix them faster. It does everything faster, yes haha

English

0

1

10

Matt@codermatt·1d

hot take: AI writes code faster. It also makes wrong decisions compile faster.

English

10

0

9

1.2K

Matt@codermatt·3h

@evb84 Good points. Will start to add some more metrics in.

English

10

edvvvv@evb84·5h

@codermatt “More tokens”. So which ended up more expensive? You should try to get it to the same state. Test how much it cost overall. If they end up with the same challenges, find out how long/cost each one takes to solve them

English

0

9

Matt@codermatt·3h

Grok 4.5 High was the strongest overall. Most complete, highly polished, and the functionality worked well. But at $0.750, it cost approx. 11× more than Luna and Deepseek v4 Flash.

English

1

74

Matt@codermatt·3h

GPT-5.6 Luna was the cheapest and most token-efficient at just $0.068 Very polished, but it incorrectly placed two deals in the wrong pipeline stages.

English

0

81

Matt@codermatt·12h

@Chop_Suey0 @OmedVibeCodes I know right. Been hitting it hard today.

English

124

HaSh@Chop_Suey0·12h

@codermatt @OmedVibeCodes The zen one is the exact same thing and COMPLETELY FREE🫡

English

0

2

137

OmedTheVibeCoder@OmedVibeCodes·20h

OpenCode Go for $10 is actually BROKEN I have used DeepSeek V4 Flash nonstop, generated an insane amount of code—and somehow I’ve only used 10% of my weekly limit. I’m literally struggling to hit the limits. Feels like a 200$ plan from OpenAI

English

102

46

1.5K

83K

Matt@codermatt·12h

@TheSpacerr Working on benchmarking multiple models! x.com/codermatt/stat…

I gave Luna 5.6 (Max reasoning) and DeepSeek V4 Flash-0731 the same challenge: build Minecraft from scratch. Luna: 343K tokens · $0.35 DeepSeek: 366K tokens · $0.25 Luna absolutely crushed it. DeepSeek wasn’t bad, but Luna’s game looked and felt far better. Videos and prompt below 👇

English