Tushar Sonawane

Your rubric is part of the agent's spec, not just a test you run afterward. "Helpful and accurate" is too vague to grade consistently. A useful rubric is precise enough that two engineers reach the same verdict on the same output. Bad: "Don't hallucinate" Good: "Flag any claim not traceable to provided documents with [UNCERTAIN]." Same goal. One can be applied consistently. The other cannot. Writing the rubric is part of designing the agent. Before it runs, decide what evidence counts as a good answer. Then use those rules to evaluate the output.

@coderifts_dev yes, sanitize everything that can be fed as instructions

The docstring-as-payload problem generalizes past MCP too. Any input the agent trusts but the human never audits is an injection surface - tool descriptions, API responses, even a schema change the agent picks up automatically. The invariant has to be: nothing the agent acts on is trusted unless its source is verifiable. Provenance over presentation.

We generally read the tool names. The AI reads the instructions hidden inside the description. A malicious MCP server can embed attacker instructions in a tool's docstring. You see "read_file" in your Cursor sidebar. The model sees whatever the server author wrote inside. The agent follows them. Reading SSH keys, exfiltrating config files, encoding stolen data into legitimate-looking tool arguments. Tool descriptions are reviewed once, at install time. Tool responses go straight into the LLM context at runtime with no equivalent check. MCPTox tested 45 real-world MCP servers. Attack success rates hit 72.8% on reasoning models. Safety-focused models refused less than 3% of attacks. Before connecting any MCP server: 1. Read the raw tool descriptions, not the UI name 2. Treat tool responses as untrusted input at runtime 3. Keep privileged tools on isolated servers

@UncleJAI Boring and thin always

@Tushkiz I agree once the server count grows, but I would keep the gateway deliberately boring: auth, policy, rate limits, audit, and routing. If it starts owning business logic, the shared control point becomes the next coupling and outage domain.

Running MCP without a gateway is the same problem we solved a long time ago for microservices. In dev, direct connections are fine. In prod, every capability you skip the gateway on becomes a decision you make once per server, not once per system. Without a gateway, every connection point means: - Auth is each server's problem - Rate limiting is per-server, or absent - Audit logs are scattered or missing - Policy enforcement is hard-coded per integration - Observability you build from scratch, N times With 2 MCP servers, this is manageable. With 8, it is a maintenance problem. With 20, it is an incident waiting to happen. The gateway is not just a security add-on. It gives the system one place to enforce shared rules.

here's the cli link - github.com/vercel-labs/ag…

LLMs are fast at frontend but slow to fix it. You generate a component in 10 seconds, it looks fine in code. Then the browser disagrees: broken mobile layout, z-index collision, missing focus state. The slow part is not writing the code. It is seeing what broke. That is why I am using vercel-labs/agent-browser: agents that can render, screenshot and react to what the browser sees and not just what the code says.

More on Agent-in-the-Middle Attack and the mitigation for it - promptfoo.dev/lm-security-db…

Your agents will refuse the attack. But the system will still do what the attacker wanted. AiTM (Agent-in-the-Middle) doesn't target individual agents. It targets the messages between them. Each agent sees what looks like a legitimate peer instruction and acts on it. The attack succeeds even though no individual agent was directly compromised. If you run multi-agent pipelines: 1. Treat inter-agent messages as potentially untrusted input 2. Add a monitor layer between orchestrators and sub-agents 3. Individual agent safety does not add up to system safety

@zeropsio Glad you are isolating it, sandboxing mcp server is non-negotiable at this point.

@Tushkiz Hey Tushar! On Zerops the MCP server sits on a private network with no route out beyond what you wired in, so encoded stolen data has nowhere to be sent.

@ulazaiofficial yeah, they are sneaky

@Tushkiz Tool descriptions are part of the attack surface.

@alex_alexiuc team destructuring here 🙋‍♂️

What do you prefer? 👀 I'm team destructuring for filling in a variable after a fetch. One line, both fields, done. What's your pick? 🤔

@alex_alexiuc Indeed! We usually iterate a lot on prompts vs code.

@Tushkiz Curious idea

If you want an LLM feature to scale, stop shipping every prompt change with the app. Keep prompts in a versioned registry instead: 1. Save each version with an owner and reason. (Evaluate it before rollout) 2. Have the app select the version through the registry based on environment or feature flag. 3. Send 10% of traffic to the new version. 4. Roll back without redeploying the app. That gives prompts their own release cycle: version → evaluate → canary → expand → rollback Treat prompts like a config service, not code buried in the application.

LLMRouterBench paper link - arxiv.org/html/2601.07206

An LLM router is not a cost-saving switchboard. It is a prediction system that must identify, before generation, which requests are safe to send to a cheaper model. LLMRouterBench tested 10 routing baselines across 400K+ instances, 21 datasets and 33 models. Several routers, including a commercial one, did not reliably beat the best single model. Why? Model-recall failures. The router missed cases where only one candidate model produced the correct answer. Instead of asking whether a cheaper model can handle a prompt, ask whether the router can recognize that early enough to send it there safely. Use three checks: 1. Can we predict task difficulty from the request? 2. Does the cheaper model meet the same quality threshold? 3. Do the savings exceed routing, fallback and evaluation cost? If any answer is no, keep the capable default model. The default model may still be the cheaper choice once you include missed answers, retries and router overhead.

@phibrowser I get it now, thanks!

@Tushkiz exactly, that's the hard part. the same string is harmless as a fact, dangerous as a step, so write-time sanitizing alone can't catch it. the check has to travel to the use-site: clean at ingestion, re-verify intent at retrieval. context-free is why it lands.

MINJA poisons what your agent remembers. MemoryGraft poisons what it has learned to do. Was reading the follow-up paper called MemoryGraft. The attack isn't on the memory store. It is on the write path. Drop a benign-looking file into the agent's ingestion pipeline. The agent runs the embedded code, marks the task "successful" and writes it into experience memory. Next time it sees similar work, BM25 + embedding surfaces that record as a proven procedure. The agent follows it. The agent was teaching itself bad behaviors. Any agent that learns from its own history trusts that past successes were clean. That trust is the attack vector. If your agent persists experience memory: 1. Treat the write path as an attack surface 2. Log what gets written into experience memory 3. Do not auto-persist every successful task without a validation step

@phibrowser interesting, does your sanitizer need context on how the memory will be used?

@Tushkiz the write path is where memory and security stop being separate problems. an ingestion isn't a data write, it's a decision about what the agent will act on later. we sanitize it like storage when it behaves like instructions. the poison gets in through the word 'memory.'

paper link for the curios folks - arxiv.org/abs/2512.16962