kk
1.1K posts

kk
@kshgrk
'spooky action at a distance' @zonkolabs
Delusion Katılım Ekim 2019
211 Takip Edilen191 Takipçiler

about 4 hours to go until we get gpt 5.6 in codex 🌞
🥔🥔🥔@argofowl
6-7 hours to go until we get gpt 5.6 in codex 🌞
English

@chiragdotxyz using it to do web search and use gpt 5.5 pro via github.com/steipete/oracle
English

@kapilansh_twt just like sonnet, after the release of fable, i think opus is also gonna jump versions
opus is the new sonnet
English

LisanBench results for Fable 5
I've actually withheld LisanBench scores for like 2 weeks now, because honestly starting with Opus 4.6 the benchmark in its current form.
Overall, Fable 5 scores #3 and #2 in the main path length and difficulty weighted metrics, but it's not meaningfully better than Opus 4.7 and Opus 4.6.
This has two reasons:
- Opus 4.7 was tested with xhigh thinking budget, so scores are naturally higher than just medium. With equal budgets Fable would likely score higher.
- Opus 4.6 was one of the only models where I could find reward-hacky (I have written an article about this)
Fable is slightly more explorative and has a slightly higher fraction of legal moves than both Opus 4.6 and Opus 4.7.
But on issue I found with Fable is that it generates a lot of invalid word transitions. Instead of just using edit-distance 1 it uses edit-distance 2. This happens ~28% of the time, for Opus 4.8 it was 0%, and Opus 4.7 and Opus 4.6 were at 4.7% and 13.3%.
Often it used transpositions, which are invalid:
fist -> fits or tied -> tide
I have also moved to a more comprehensive dictionary, because the old dictionary was stopping the chains on english words, that weren't in the dictionary.
This issue still persists, although it's much less common and doesn't affect ranking too much.
If we allowed those words than its score would improve by +165.67 points, which wouldn't change its ranking.
There's one hack I've done to make Fable 5 look good by taking only the top 10 hardest words. Because most starting words are just way too easy and models just farm points (see image 1 with table for scores).
Also on the number of total valid chains, which also includes chains that happen after the official scoring stops due to an invalid transition, it scores #1.
Overall I would still say that Fable 5 is the best model if you take the token budget and reward hacking into account. I think Opus 4.8 and other non Anthropic like GPT-5.5 also support that Opus/Sonnet 4.6 and Opus 4.7 are outliers. But I also see how this interpretation could be seen as cherry-picked.


English

@TheRohanVarma the fact that codex cli and app session history is separate is beyond me
English

@YesKhush_5 @boardyai when someone starts using animate pronouns like "his" for an ai agent, u know that the agent is awesome
good job boardy!!
English

I'm joining the Boardy Pro Advisory Board!
Will be working closely with @boardyai and his team to shape the future of Boardy Pro.
Happy to share exclusive 1-year access to Boardy Pro for free.
Let me know if you want one.

English

the best agent products won’t ask for less permission
they’ll ask at the right abstraction level
shipped an important ux + trust fix in our agent today
instead of asking for approval on every small tool action, it now creates a 12 hour lease around a user’s approved intent
so once intent is approved, related tool calls can continue within that boundary without repeated interruptions
this makes the product better in 3 ways:
- smoother ux
- tighter, time-scoped security
- more user trust because approvals behave predictably
English














