Vals AI

Today @xai just rearranged our leaderboards… Grok 4.3 jumped 25 points to take #1 on CaseLaw v2 and climbed 21 spots to lead CorpFin at 68.5%. Congrats @xai @elonmusk 🚀

Full results and benchmark breakdown can be found on vals.ai/models/mistral…

Medium 3.5 has a 262k context window. For all benchmarks, it was run with 80k output tokens, temperature=0.7, top P =0.95. On the index, it costs $2.06/per test, which is expensive compared to other open-weight models. .

This week Mistral dropped Medium 3.5 model and it landed at #10 on open-weight Vals Index.

Grok #1 in law

We have updated the results of GPT 5.5 on our site - it is now the #1 model on Terminal Bench 2. Its ranking on the Vals Index has not changed.

After reaching out, we were able to confirm with OpenAI that “tool_choice”: “none” injects an additional steering instruction into the model system prompt, in a way that tools: [] does not. This instruction seemingly hurts the model’s ability to use the Terminus 2 harness effectively, which, despite not using native-tool-calling, is still agentic.

We noticed discrepancies between our Terminal Bench scores and those reported by others. After a detailed investigation, we determined the cause of the delta was a surprising, undocumented behavior of the tool_choice parameter. See our full findings below.

After the discovery below, GPT 5.5 is now the #1 model on Terminal Bench 2, improving by +11%. It is still the #2 model on the Vals Index. See full results on vals.ai/models/openai_…

@scaling01 It's here and we evaluated it- x.com/ValsAI/status/…

looks like we are getting grok 4.3 api very soon

Full results and domain specific benchmarks can be found at vals.ai/models/grok_gr…

We evaluated 4.3 with the default xAI hyperparameters: temperature=0.7, Top P: 0.95, and Top K= default. This model has 1M context window and output tokens. Latency is 584.24 seconds per test and costs $0.38/test on the index.

Grok 4.3 has launched at #13 on the Vals Index. It ranks #1 on CaseLaw and #1 on CorpFin but it struggles on general coding benchmarks.