Portia AI retweetledi
New tutorials on structured agent development with Portia AI
Just added some new tutorials to my production agents repo covering Portia AI and its evaluation framework SteelThread. These show structured approaches to building agents with proper planning and monitoring.
What the tutorials cover:
Portia AI Framework - Demonstrates multi-step planning where agents break down tasks into manageable steps with state tracking between them. Shows custom tool development and cloud service integration through MCP servers. The execution hooks feature lets you insert custom logic at specific points - the example shows a profanity detection hook that scans tool outputs and can halt the entire execution if it finds problematic content.
SteelThread Evaluation - Covers monitoring with two approaches: real-time streams that sample running agents and track performance metrics, plus offline evaluations against reference datasets. You can build custom metrics like behavioral tone analysis to track how your agent's responses change over time.
The tutorials include working Python code with authentication setup and show the tech stack: Portia AI for planning/execution, SteelThread for monitoring, Pydantic for data validation, MCP servers for external integrations, and custom hooks for execution control.
Everything comes with dashboard interfaces for monitoring agent behavior and comprehensive documentation for both frameworks.
These are part of my broader collection of guides for building production-ready AI systems.
github.com/NirDiamant/age…
Anyone working with structured agent approaches? Curious what planning and evaluation methods others are using.
English
