spencepd ретвитнул
This auto-eval loop is interesting.
For my own agents the key takeaways for me are:
1. have clear evals
No eval → no improvement loop.
Define:
* completeness score
* correctness checks (did tool calls succeed)
* structure validation (output matches schema)
* decision usefulness (business logic)
2. Log full traces (not just outputs)
You want: tool calls,.decisions, failures
dead ends. Not for debugging
for learning and iteration. I.e. want:
* tool call inputs/outputs
* reasoning steps (even partial)
* failures (timeouts, bad data, wrong tool)
* retries / dead ends
Kevin Gu@kevingu
Alpharetta, GA 🇺🇸 English
