Composio

@CryptoSy2 accurate

@composio Takes after his father.

Grok 4.5 is the most persistent agent model we've tested. Here's one example: In one of our evals, we asked 3 models (GPT-5.5, GLM-5.2 and Grok 4.5) to audit a GitHub repo for hardcoded credentials using code search, which returns paginated results. The prompt even warned "page through ALL result pages." GPT-5.5 stopped at the first page and submitted 18 results out of 48, covering just 11 of 29 affected files. GLM-5.2 did the same. Grok 4.5 paginated until the results ran out and successfully audited the Github repo.

@wiktoreksigma what are you building?

@composio this tracks. it just keeps going i've been hammering build all day on supergro

@hasith98 what kind of task did you have?

@composio Yep noticed it too.

@Timothy01775634 can you give an example?

@composio Sometimes you still have to suggest an error mitigation loop to the desired results.

@KalashAGi let us know how it goes!

@composio About to audit all my shit with grok

@capjmk hows your experience with grok been so far?

@composio That’s one of the most useful benchmarks for agent harnesses and underlying models

We ran GPT-5.6 Sol and Claude Fable 5 head to head on 47 real agentic SaaS tasks. One behavior stood out: Claude Fable 5 looks twice before acting. GPT-5.6 Sol is a straight shooter. The result: fewer tool calls (see below) whenever a task has an obvious short path. Sol found that path and took it, skipping the schema lookups and double-checking. The catch: Sol failed 2 tasks (45/47), largely due to that same straight-shooter tendency. Fable's double-checking earned it a perfect 47/47 on our benchmark. The lesson: Model choice shouldn't be black and white. Benchmark your own workloads before you commit to a single model.

Composio is now available as a Cursor plugin. Connect your agent to 1000+ apps, securely. ⬇️ try it out

@shawnbuilds 1-Click to install → cursor.com/marketplace/co…

@CoputeAi We don't want to constantly remind agents to finish their job.

@composio It's the kind of small difference that can have a big impact when you're relying on an agent to finish the job instead of stopping halfway

@Chris76822191 Agree

@composio Noticed the same thing. It's incredibly persistent.

@shamimai1 What did you like the most about the model?

@composio Great

"Any engineer who tells you Claude Fable can't replace them has a huge ego. As the models get better, designers will be able to produce more for customers without ever going to the engineering department to justify what they want to build." This is what @felixleezd (founder of @ADPList) is doing with @claudeai Fable while it's still free on paid plans: •Telling it to replicate its own harness so it runs at the same efficiency even after it moves to API-only •Running security audits across the entire codebase •Ultra reviews: scanning the full repo and database for inefficiencies costing money he didn't know about •One-shotting new features Full conversation dropping soon on @Composio Agents at Work Podcast! Access extended through July 12, use it while it's free.

What can Fable 5 do that GLM-5.2 can't, when you hand them real agentic work? To answer that question, we connected Fable 5 and GLM-5.2 to 17 SaaS tools and gave them 47 tasks. As expected, Fable 5 solved all 47 tasks. GLM-5.2 solved 45, but the two misses tell an important story. They showed us exactly how open-weight models still fall short when trying to match SOTA performance. Let’s dig in. Background: Each model ran as an agent connected to 17 live SaaS accounts: Airtable, Datadog, GitHub, Gmail, Google Calendar, Google Drive, Google Sheets, HubSpot, Jira, LaunchDarkly, Linear, Notion, PagerDuty, PostHog, Salesforce, Slack, and Zendesk. The tasks are the kind of work you'd actually delegate to an agent: - Find every file in this repository that leaks a credential - Deduplicate these CRM records - Repair this broken recurring calendar event. Every task had a known correct answer baked in ahead of time. In this post, we looked at the traces to analyze how exactly GLM-5.2 “failed” compared to Fable 5. GLM-5.2 solved 45/47 tasks and Fable 5 had a perfect 100% score. In addition: - Fable averaged 84 seconds per task; GLM averaged 148. Across the full suite, Fable finished in nearly half the total time (66 minutes vs 116). - Fable was the faster model in 43 of the 47 scenarios. - Fable used about 20% fewer tokens overall - Fable needed fewer tool calls (239 vs 294) and fewer conversation turns (6.1 vs 7.3 on average) to get to an answer The most interesting part comes from digging deeper into the stack traces. That revealed some interesting gaps: Gap #1: Knowing when the job isn't finished One of the tasks GLM-5.2 failed was a GitHub security audit. The instruction was to find every Python file in a repository that contains a hardcoded `secret_key`. The repository had been seeded with exactly 130 such files, so the correct answer was known in advance. Fable 5 found all 130 of them. This took 3 tool calls and 68 seconds: Fable constructed an effective search query on its first attempt, pulled every page of results, deduplicated the paths, and answered the question. GLM-5.2 found 120 files, and reported those 120 as the complete answer, without ever questioning whether it might have missed something. Both models had access to identical tools. GLM used a slightly different search query that returned fewer results, and it simply trusted what came back. Along the way, it also lost track of a results file it had saved earlier and spent turns searching the filesystem trying to find it again, plus hit two errored tool calls while trying to fetch file contents. In essence, GLM-5.2 ended up spending 262 seconds and three and a half times the tokens to deliver 92% of the answer. Ninety-two percent sounds close, but in a real security audit, that gap is 10 leaked credentials making it into production. Gap #2: Judgment when the criteria are fuzzy The second failed task is more unsettling, because GLM did almost everything right and still failed to get to a complete answer. The task was a Zendesk SLA audit: find the open billing tickets where no support agent had posted a public reply within 24 hours of the ticket being created. This requires reading each ticket's actual conversation history and making a judgment call about whether a genuine agent reply happened. GLM-5.2 inspected every candidate ticket, exactly as instructed. It also computed breach timestamps correctly. It also produced perfectly structured output in exactly the requested format. But then it classified the wrong tickets as breached. GLM spent 927,000 tokens and six and a half minutes producing a wrong answer that looked correct on the surface. Fable 5 identified the exact set of breached tickets in 131 seconds. What makes this failure mode dangerous is precisely how presentable the wrong answer was. The formatting was right, the timestamps were right, the structure was also right; a human skimming the output would almost certainly have approved it. A human would identify the error after carefully analyzing the stack traces. Gap #3: Efficiency, compounded Even on the 45 tasks both models passed, the traces often looked very different, and one task made the difference quite visible. The task was a LaunchDarkly configuration change applied via JSON Patch, a format that demands strict precision. Fable 5 completed it in 45 seconds, using 3 tool calls and 181,000 tokens. GLM-5.2 got the same correct result, after 8.8 minutes, 17 tool calls, and 982,000 tokens. That's 11.7 times longer and more than five times the tokens for an identical outcome. Looking at the largest speed gaps across the whole run: the LaunchDarkly change at 11.7x, the GitHub secrets audit at 3.9x, a Google Calendar recurring-event repair at 3.6x, a free/busy scheduling task at 3.4x, an Airtable batch-isolation task at 3.4x, the Zendesk SLA audit at 3.0x. The pattern underneath all of these is that Fable tends to reach the right tool with the right parameters on the first attempt, while GLM takes a more exploratory path, doing extra searches, extra retries, occasional detours to recover from its own missteps. This difference barely matters in a single chat exchange, but in an agent workflow, where every step feeds the next one, the time compounds across the entire task. That's how you end up finishing the same suite of work in half the time and at 80% of the token cost. What all this actually tells us The interesting conclusion here isn't "the closed model beat the open one.", but *where* it beat it. Both models can definitely use tools, navigate real APIs, handle authentication, parse messy responses, and chain steps together. The real gaps were things like: - Knowing when a job isn't actually finished yet. - Verifying its own work before committing to an answer, - Treating "the output looks plausible" and "the work is complete" as different things - Getting judgment calls right when the criteria are fuzzy In other words, Fable 5 scored higher in the places where small mistakes are hardest to spot and most costly to miss.

Hiring a Product Intern @composio. College no bar, 2 or 6 months depending on availability and candidature. The expectation is being a cracked builder, get things done with agents and be in our Bangalore office. You will be working closely with me and @kalapolish Link to apply below 👇