Today we're releasing HiL-Dynamics, the first open-source tool that measures how production agents actually collaborate with humans under uncertainty. Not just whether they got the answer. Now you can measure exactly when your agent asks for help, when it makes assumptions, and when it'll confidently ship the wrong answer. Our findings 🧵

We evaluated five production agent stacks: Codex, Claude Code, Google Antigravity, ADK, and OpenCode, with and without customization and skill tuning. Even with the best enhancements, agents still struggle to escalate to humans at the right moment. State-of-the-art agents remain far from reliable human collaboration.

@ScaleAILabs Finally, agents that know their limits.

@ScaleAILabs Open-source transparency for agent decision-making finally a tool that measures when to ask, assume, or ship. Huge step.

@ScaleAILabs This is happening at one of the companies; we need you to look into it. youtube.com/watch?v=8M4ODr…

@ScaleAILabs 您好，我是一名来自中国河南省的人，在数据标注行业工作了6年，目前职位是分公司负责人和供应商管理负责人，请问我有机会去贵司工作吗？

@ScaleAILabs under uncertainty. how do you measure the cost of a wrong assumption that ships anyway?