PostHog

Finally updated our OS

Finally updated our OS

everyone is talking about agency, not enough people are talking about information. right now, everyone is talking about loops, planning, tool use and sandboxes. those are all about what an agent _can_ do, but there’s not much focus on what the agent _knows_. loops increase effort, not necessarily effectiveness. a loop let’s the model keep trying. it doesn’t tell the model whether it’s trying to solve something worth solving. there are of course exceptions - we’ve seen this when it applies to simple performance metrics, where loops have produced superb outcomes for posthog recently. we've written about this on our blog. software engineering for most isn’t primarily a coding problem. code is a means to an end. a huge amount of the valuable work is asking questions like what’s broken, how to reproduce x, who does this affect, did this start recently, is this worth fixing now, is this even the real problem. right now, I think we are collectively underestimating how much product engineers rely on ambient context. which systems matter, which systems are fragile, which metrics move the business, what the company wants, what customers want, which bugs can wait. that context isn’t covered well by git and documentation alone. i think this is perhaps where the incentives of the frontier labs and engineers working on products somewhat differ. models are remarkably good at research tasks - they are naturally inclined to read documentation, read multiple files, compare alternatives and reason step by step. the labs are making models great at research in their pursuit of recursive self improvement ending in agi. building products requires something different - understanding users, business priorities, production behavior, and deciding what matters. it’s more about asking the right questions than just being great at answering them. the tooling for this feels much more nascent.