Steffen Hausmann retweetledi
🔎🤖LangSmith Insights Agent
Really excited to launch our first in-product agent
This agent lives inside LangSmith and combs through traces, giving you insights into:
🧑🤝🧑how users are using your agent
⁉️how your agent may be messing up
🛃{your custom insight here}
The problem we saw was that people were launching agents... and didn't know how their users were actually using them! You put a chat box in front of people, and they may ask it anything - the surface area for agents is often super wide
In addition - agents would fail silently. They could give a bad response - this wouldn't show up in error logs, but its good to know.
If you know what look for, you can set up LLM as a judge evaluators. But what if you don't? (most people don't initially)
The best way to figure this out - as @HamelHusain says - "look at your data". But LLMs are really good at looking at your data! So can they do it for you?
This is exactly what insights agent attempts to do. It's live in LangSmith today. You can read more about it here: blog.langchain.com/insights-agent…
English
