
agentgateway
47 posts

agentgateway
@agentgateway
https://t.co/78P0GHdaaC One high-performance gateway for service, LLM, and MCP traffic.












New in agentgateway: LLM budget enforcement. Here's the whole setup in 4 steps 👇 💰 Step 1 — Model cost catalog Per-model USD rates (per 1M tokens) in a ConfigMap. Your entries merge with a built-in catalog, so you only define models where you want to pin your own rates. ⚙️ Step 2 — Load it into the proxy Reference the catalog from EnterpriseAgentgatewayParameters and point your Gateway at it via infrastructure.parametersRef. On startup the proxy confirms: loaded model catalog providers=3 models=80. 🛣️ Step 3 — Route the traffic A standard Gateway API HTTPRoute in front of your LLM backend. 🚦 Step 4 — Budgets + enforcement Two budget modes: • Audit — track spend against a limit (e.g. $5/day) without blocking • Block — a hard circuit breaker (e.g. 2,000 tokens/day) that stops traffic when tripped Attach the policy to the route, let ArgoCD sync, and you've got cost governance for AI traffic as code. Full 30-second demo in the video 🎥 #agentgateway #LLMOps #FinOps #Kubernetes #GatewayAPI #PlatformEngineering #GenAI #AIGateway #SoloIO






