Sabitlenmiş Tweet
Datacurve
11 posts
Datacurve
@datacurve
Research and data to advance frontier models.
San Francisco Katılım Şubat 2024
11 Takip Edilen4.1K Takipçiler
Opus 4.8 is a clear efficiency step up over 4.7 on DeepSWE, same-or-better task success with fewer steps and fewer input tokens per task.
Worth noting DeepSWE measures backend/implementation, not frontend.
Hoping to release a deep dive soon.
Datacurve@datacurve
Opus 4.8 is now on DeepSWE. On the default high thinking effort, it scores 6% higher than Opus 4.7 xhigh, while also lowering average cost per task.
English
Full deep dive coming soon.
Check out the full benchmark here → deepswe.datacurve.ai
English
Datacurve retweetledi
DeepSWE reflects what I’m hearing from engineers better than any other benchmark.
They took the hard path to build a good one.
Serena Ge (Datacurve)@serenaa_ge
Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks. On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work.
English
Datacurve retweetledi
This is the new standard for engineering evals
Serena Ge (Datacurve)@serenaa_ge
Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks. On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work.
English
Datacurve retweetledi
Today we’re releasing DeepSWE, a new standard for agentic coding benchmarks.
On public leaderboards, top models often look relatively close in capability. DeepSWE shows where they actually diverge, reflecting the realistic experience of developers in their day-to-day work.
English
Datacurve retweetledi
I presented today at Demo day Day 2 and @TechCrunch featured us @datacurve! Just been reading TC and listening to TC Daily Crunch since high school mornings... a surreal feeling to see us on it. Also, post-demo sadness cuz now YC is coming to an end
English