Judgment Labs

@JudgmentLabs is hosting a pop-up poker night at @NeurIPSConf for RL folks tomorrow night (Tuesday, December 2)! ♠️ Spots are limited and we're prioritizing attendees who are passionate about/actively working on AI research in the RL, evals, and rubrics space. If you're looking for sharp and deep convos with elite RL talent, we'd love to see you! 🧡 Event link in replies

@DakotaJMcKenzie @shaiunterslak sent a DM @shaiunterslak

@shaiunterslak Meet w/ @JudgmentLabs

Half the good AI startups I know have tried one of the 100 evals platforms and then built their own in-house.

At @JudgmentLabs we've had the opportunity to work with countless AI agent teams building fantastic products. Measuring and understanding agent behavior has become a bottleneck to agent improvement and everyone knows it. However, few get this process right and most teams fall into common pitfalls. In this piece from our research team, we cover some failure modes of teams attempting to monitor their agents and what a proper future of agent quality measurement will look like. Link in replies

@alexshander03 Good read 🫡

@hallerite we do! sent you a DM

who's building the wandb for LLM-RL trajectories