Vals AI

1.4K posts

Vals AI banner
Vals AI

Vals AI

@ValsAI

Public LLM Evaluation // https://t.co/FjWabQY2jk @8vc @BloombergBeta @pearvc

San Francisco, CA Katılım Mart 2024
264 Takip Edilen13.4K Takipçiler
Vals AI
Vals AI@ValsAI·
In case you missed it, this was a big week for model releases. Grok 4.5, GPT-5.6 Sol, and Muse Spark 1.1 all landed within days of each other, and all three cracked the top 10 on the Vals Index. The pace of progress right now is hard to overstate. Here's our week in review
Vals AI tweet media
English
3
7
95
5.8K
Vals AI
Vals AI@ValsAI·
Across the newly completed benchmarks, Grok 4.5 picked up five top-10 finishes: #8 on CorpFin, #8 on MMLU-Pro, and #9 on LegalBench, Code Migration, and the Vals Multimodal Index.
English
2
0
2
966
Vals AI
Vals AI@ValsAI·
Full results for Grok 4.5 are in and live on Vals! Compared to Grok 4.3, it gained 34 points on Excel Modeling Benchmark, 30 points on Code Migration Benchmark, and 13 points on MortgageTax.
Vals AI tweet media
English
3
6
92
5.7K
Vals AI
Vals AI@ValsAI·
📢Vals is at ICML! Last night we hosted a small dinner and it was one of the highlights of the trip. Grateful for everybody who attended and already looking forward to more conversations. If you're at ICML, DM us to say hi!
Vals AI tweet mediaVals AI tweet mediaVals AI tweet media
English
3
0
41
4.9K
Xiaopu Peng @ ICML
Xiaopu Peng @ ICML@SuperNyx1024·
@ValsAI It looked like a great event! I got to meet Rayan at the RL + Agents lunch during ICML, really enjoyed the conversations!😆
English
1
0
1
137
Zhiqing Sun
Zhiqing Sun@EdwardSun0909·
Muse Spark 1.1 is #4 on the overall vals index eval and fastest model in top 10!
Zhiqing Sun tweet media
Vals AI@ValsAI

Muse Spark 1.1 ranks #4 on the Vals Index and is the fastest model in the top 10, running roughly 3x faster than the top three models having an average latency of 388s.

English
9
40
270
37.4K
Lisan al Gaib
Lisan al Gaib@scaling01·
GPT-5.6-Sol is a clear step-up from GPT-5.5 on ProgramBench
Lisan al Gaib tweet media
English
18
8
452
21.4K
Vals AI retweetledi
Alexandr Wang
Alexandr Wang@alexandr_wang·
Muse Spark 1.1 is ranked #4 on the Vals Index, ahead of GPT-5.5 and Grok 4.5
Vals AI@ValsAI

Muse Spark 1.1 ranks #4 on the Vals Index and is the fastest model in the top 10, running roughly 3x faster than the top three models having an average latency of 388s.

English
62
74
1.1K
377.9K
Vals AI retweetledi
Alexandr Wang
Alexandr Wang@alexandr_wang·
Muse Spark 1.1 is SOTA on Harvey's Legal Bench, TaxEval, and MedScribe. It's cool to see that our model outperforms even Fable in a few areas :)
Vals AI@ValsAI

Meta just released Muse Spark 1.1 and is the new SOTA on MedScribe and TaxEval, taking the top spot from Fable 5 while being 10x cheaper and twice as fast. Meta currently holds the top 2 spots on TaxEval It is also the new #1 on Harvey's Legal Agent Bench, dethroning Grok 4.5 less than 24 hours after it took the top spot.

English
108
99
1.5K
379.5K
Bindu Reddy
Bindu Reddy@bindureddy·
👑 GPT 5.6 SOL LAUNCHES AS THE BENCHMARK LEADER ON LIVEBENCH AI 👑 GPT 5.6 sol crushes all benchmarks and comes #1. Fable is still the go-to model for complex multi-turn agentic loops. 5.6 is an INSANELY good model and we expect 80% of all Opus API workloads to move there It's faster, cheaper and better than Opus class moodels
Bindu Reddy tweet media
English
41
30
305
19.7K
OpenAI Developers
OpenAI Developers@OpenAIDevs·
GPT-5.6 is rolling out in the API. Sol is our flagship model, leading in coding, knowledge work, cybersecurity, and science. Terra delivers performance competitive with GPT-5.5 at lower cost. Luna is our fastest, most affordable model for high-volume tasks.
English
120
184
2.7K
227.9K
OpenAI
OpenAI@OpenAI·
GPT‑5.6 is available starting today across ChatGPT, Codex, and the OpenAI API. The rollout is starting globally now and will continue gradually toward full availability over the next 24 hours. In ChatGPT, Plus, Pro, Business, and Enterprise users access GPT-5.6 Sol through medium and higher effort settings. Pro and Enterprise users can also select GPT‑5.6 Pro for the highest-quality results on complex tasks.
English
47
158
1.2K
195.8K
OpenAI
OpenAI@OpenAI·
Sol, Terra, and Luna, our GPT‑5.6 family of models, are starting to roll out now in ChatGPT, Codex, and the API.
English
584
1.4K
12.4K
4.1M
Vals AI
Vals AI@ValsAI·
GPT 5.6 - Sol has a 1M context window, 128k max output tokens, and 748.4 latency on the Vals Index. We ran the model on OpenAI’s default provider settings, with max reasoning effort on all benchmarks but Terminal Bench.
English
1
0
5
1.5K
Vals AI
Vals AI@ValsAI·
OpenAI’s most anticipated model of the year is here, and it ranks #2 on Vals Index and Vals Multimodal Index. Although Fable 5 is still ahead on several benchmarks, GPT 5.6 is clearly in the same class, and is able to complete tasks like our CyberBench that Fable refuses.
Vals AI tweet media
English
5
13
143
18.8K