phil

348 posts

phil

phil

@philcuvin

If not now, when? @SpaceXAI

Katılım Ağustos 2024
267 Takip Edilen846 Takipçiler
phil
phil@philcuvin·
@PrinceVogel put the avalon panorama photo in the bag
English
0
0
2
251
phil
phil@philcuvin·
@imjaredz nightmare on third street
English
0
0
2
120
Shen Zheng
Shen Zheng@shen_zheng25741·
Today is my last day at xAI. My time at xAI has been intense and incredibly fruitful. I had the chance to work on many interesting things, from improving Grok’s search and DeepResearch capabilities to working on longer-horizon and more autonomous ML research. It was especially rewarding to see some of that work translate not only into strong results, but also into a better experience for people using Grok every day. I’m incredibly grateful to all the friends and teammates I met along the way. I learned so much from this exceptionally talented group of people, and I have no doubt that xAI will keep getting better from here.
English
59
10
737
79.3K
phil retweetledi
phil retweetledi
Alexander Yue
Alexander Yue@Alezander907·
I take it all back. We just got access to eval Grok 4.5 and it has landed above GPT-5.6-Sol and just shy of Opus for browser use. Because cache input is expensive, the overall cost is only 10% cheaper than opus. Its overall a bit faster. We have another opus-class model in the competition
Alexander Yue@Alezander907

Some new model release and my eval scores: gpt-5.6-sol is a small improvement on browser tasks, but seems to consume many more tokens Muse spark 1.1 outperforms gemini 3.5 flash at a cheaper price, but is not overall very good Claude sonnet 5 is 5th best performance at higher cost than opus. I am not very impressed with new models browser-use abilities. Maybe I will train my own instead

English
91
124
1.8K
2.9M
phil
phil@philcuvin·
fun fact. Sol stands for what you are after it deletes your worktree
English
2
0
6
1.2K
phil retweetledi
Dean W. Ball
Dean W. Ball@deanwball·
there comes a time in every ai policy professional’s life when they realize they have to read the talmud to make further progress, and for me that time was this week
English
507
170
2.1K
798.9K
phil
phil@philcuvin·
@alexandr_wang you will live in the llm-generated house. you will eat the tokens
English
0
0
4
428
Alexandr Wang
Alexandr Wang@alexandr_wang·
I’d live in some of these homes designed by muse spark post upload, and they only cost a few cents to build
thehype.@thehypedotnews

meta muse spark 1.1 vs gpt 5.6 sol vs fable 5 vs grok 4.5 meta recently dropped muse spark 1.1 – a multimodal reasoning model from meta superintelligence labs built for agentic tasks. key facts: • 1m token context with active self-management – the model compacts its own history and keeps only the steps needed for later work • trained to orchestrate multi-agent systems: as main agent it plans and delegates to parallel subagents, as subagent it sticks to its job and knows when to escalate back • computer use trained to pick between scripting and clicking – writes automation when it's faster, clicks when it's simpler, batches actions per step • first public api from meta: the meta model api is now in preview • benchmarks: sweeps the agent column – mcp atlas 88.1 (opus 4.8: 82.2), jobbench 54.7 (opus: 48.4), humanity's last exam 62.1 (1st). loses coding – deepswe 1.1 53.3 vs gpt 5.5's 67.0, swe bench pro 61.5 vs opus's 69.2 our test – 3 prompts, single-file html, three.js, fully procedural, no assets: 1. norwegian house cantilevered over a fjord in a snowstorm – transmissive glass wall, fully modelled interior 2. beijing siheyuan courtyard house in dawn fog – instanced roof tiles, dougong brackets, glowing paper windows 3. new mexico adobe pueblo in an approaching dust storm – deep window reveals, windward grit accumulation we ran the test on @aimlapi platform results: - cost #1 muse spark 1.1 – $0.20 #2 grok 4.5 – $0.51 #3 gpt 5.6 sol – $1.93 #4 fable 5 – ~$5.20 - output tokens #1 muse spark 1.1 – 41,868 #2 gpt 5.6 sol – 49,139 #3 grok 4.5 – 64,954 #4 fable 5 – 81,849 - lines of code #1 muse spark 1.1 – 1,799 #2 gpt 5.6 sol – 2,377 #3 fable 5 – 3,088 #4 grok 4.5 – 4,216 observations: • muse spark is the cheapest of the four by a wide margin – 2.5x under grok, ~26x under fable per run. output quality tracks the price • only 7.4% of its output tokens are reasoning (3,104 of 41,868) – the model barely thinks before writing. economic, not pedantic: it commits to the first plan and ships it • the low loc is not compression, it's omission – all three prompts demanded instancing, muse spark delivered it in one muse spark's code quality – reviewed by fable 5: upsides: 1. all three files run 2. the adobe grit effect is legit – shader injection via onbeforecompile, windward faces detect storm direction through a normal-dot-wind term and darken procedurally 3. the fjord glass is real meshphysicalmaterial with transmission and ior, not a transparent quad 4. the siheyuan properly instances barrel tiles, dougong blocks and courtyard pavers downsides: 1. in the fjord file the strafe vector is negated – press a, you move right; press d, you move left. exactly the key mix-up we kept hitting with this model 2. all three files ship the model's self-doubt as comments: "// actually yaw orientation: need correct" sits above a direction vector that gets computed, abandoned and recomputed – dead vectors allocated every frame, 60 times a second 3. the siheyuan registers two separate keydown listeners, one containing an empty if-block 4. snow "accumulation" on the norway roof is a sine wobble on a scale value, not accumulation 5. "instanced snow" became 3,500 plain points. zero dispose calls anywhere pattern: minimal reasoning, minimal code, minimal price. it nails the flashy requirements – shaders, transmissive glass – and quietly drops the boring ones: instancing, controls, cleanup. you get a demo that mostly runs and a control scheme you can't trust follow @thehypedotnews for 24/7 ai news, analysis and breakdowns

English
50
21
617
88.1K
phil
phil@philcuvin·
@SeizeEndowments @brianryhuang > distal Gatsby dynamic isn't super varied. all the way down to the parties with in-vogue libraries. they never cut the pages
English
0
0
1
30
Dawson Engler
Dawson Engler@SeizeEndowments·
@philcuvin @brianryhuang Big spread in the substances they buy for cash over the decades ( in my limited ). But probably the distal Gatsby dynamic isn't super varied.
English
1
0
1
36
Brian Huang
Brian Huang@brianryhuang·
Every frontier lab new grad I know started dressing like a fuckboy, that’s how you know they’re post-economic
English
10
3
313
38K
phil retweetledi
Mercor
Mercor@mercor_ai·
Grok 4.5 from @SpaceXAI places #2 on the APEX-SWE leaderboard at 51.2% Pass@1 (±6.0), behind Fable 5 (65.5% ±6.2) on our benchmark for real-world software engineering work. It leads Integration (65.0% Pass@1) and places #2 in Observability (37.3% Pass@1), covering multi-step build tasks and diagnosis/debugging respectively. The Integration lead maps directly to the agentic workflows Grok 4.5 was built for: multi-step coding tasks run in collaboration with Cursor. Grok models have improved 30.2 pp in a year on this benchmark: Grok 4 (21.0% Pass@1) to Grok 4.5 (51.2% Pass@1). Congratulations to the xAI and Cursor teams.
Mercor tweet media
English
89
99
690
1.8M
phil
phil@philcuvin·
@shiraeis joanne's departure clearly still hasn't been recovered from
English
2
0
7
827
shira
shira@shiraeis·
so openai’s model personality team might be going through something
shira tweet media
English
31
17
360
15.4K
phil
phil@philcuvin·
@chrisbarber waiting for the day people clue in to how capable SI is
English
1
0
5
3.2K
phil
phil@philcuvin·
are you on the SWE-1.7 or the Grok 4.5 side of X today
English
7
1
50
2.1K
phil
phil@philcuvin·
I'm glad we finally get to share this model with the world.
Artificial Analysis@ArtificialAnlys

SpaceXAI’s Grok 4.5 scores 54 to place fourth on the Artificial Analysis Intelligence Index following only Fable 5, GPT-5.5, and Opus 4.8. It scores on par with GPT-5.5 in Codex on the Artificial Analysis Coding Agent Index in the Grok Build harness, at much lower cost Grok 4.5 improves 16 points over Grok 4.3 on the Intelligence Index, bringing SpaceXAI to the intelligence frontier behind only OpenAI and Anthropic, and outperforming all open weights models and notably Google’s Gemini models. Key standout areas of performance are agentic knowledge work and coding. Grok 4.5 in Grok Build scores 76 on the Artificial Analysis Coding Agent Index, on par with GPT-5.5 (xhigh) in Codex and just below Fable 5 (max) in Claude Code, and at a small fraction of the token usage and price. Congratulations to @SpaceXAI, @cursor_ai, and @elonmusk on the impressive release! Key Takeaways: ➤ Grok 4.5 performs very strongly on agentic tasks. Grok 4.5 ranks #4 on GDPval-AA v2 with an Elo of 1543, between Claude Opus 4.8 (1600) and GLM-5.2 (1513). It achieves the top score on 𝜏³-Banking of 33%, above 31% from GPT-5.5 (xhigh), and sits on the cost vs performance Pareto frontier across all three agentic evaluations in the Intelligence Index ➤ Grok 4.5 is one of the most cost efficient models to run for near-frontier intelligence. It costs $0.31 per task on the Artificial Analysis Intelligence Index and $2.59 per task on the Artificial Analysis Coding Agent Index within Grok Build ➤ Low cost for Grok 4.5 is driven by both low pricing and token efficiency. Grok 4.5 has a headline price over 60% lower than Claude Opus 4.8 and GPT-5.5, and used ~14k output tokens per Intelligence Index Task - over 60% lower than Opus 4.8. On the Coding Agent Index, Grok 4.5 stands out on the Pareto frontier of Coding Agent Index score vs. Total Tokens, using only 1.9M tokens for the Coding Agent Index while scoring 76 ➤ As a coding agent, Grok 4.5 in Grok Build is on par with GPT-5.5 and offers efficiency benefits: In our Artificial Intelligence Coding Agent Index that consists of DeepSWE, Terminal-Bench v2, and SWE-Atlas QnA, Grok 4.5 in Grok Build ranks third, on par with GPT-5.5 (Codex) and below Fable 5 (Claude Code). It is also very efficient in achieving this result: Grok 4.5 in Grok Build cost $2.49 per task while Fable 5 in Claude Code cost $11.80 and GPT-5.5 in Codex $5.07. This is driven by relatively low token pricing and the model using far fewer tokens than comparable models (1.9M average tokens used per task), significantly less than Fable 5 in Claude Code (7.2M) and GPT-5.5 in Codex (6.2M) Other model details: ➤ Context window of 500k tokens - a reduction from Grok 4.3’s 1M token context, but retaining configurable reasoning and vision input ➤ Pricing of $2/$6 per 1M tokens of input/output; cache hits are discounted by 75% to $0.5 per 1M tokens, and costs still double with long (>200k token) inputs ➤ As Elon Musk has disclosed, Grok 4.5 is 3x larger than its predecessor at 1.5T parameters

English
2
0
61
1.9K
phil
phil@philcuvin·
full natty
Artificial Analysis@ArtificialAnlys

SpaceXAI just released Grok 4.5, and it ranks #4 on GDPval-AA v2 with an Elo of 1543 - behind only the latest Claude releases from Anthropic on real-world agentic knowledge work tasks Grok 4.5 achieved this score at a cost of $0.49 per GDPval task to sit clearly on the Pareto frontier for performance versus cost. This cost is lower than GLM-5.2 and Kimi K2.6, and nearly 90% cheaper than the models ahead of it on our leaderboard. We’re finalizing the remaining Artificial Analysis Intelligence Index evaluations and will share final results soon. Thanks to @SpaceXAI and @elonmusk for their collaboration testing this model ahead of release, and congratulations on the launch!

English
1
1
30
1.4K