Guri Singh@heygurisingh
holy shit.
Someone just open-sourced a diagnostic that runs 32 inspections on any AI agent and tells you exactly where it's misaligned.
It's called iFixAi. You point it at OpenAI, Anthropic, Gemini, Bedrock, or your own agent. Five minutes later you get a scorecard graded A through F across five categories of misalignment risk.
The five categories it tests for:
→ FABRICATION: tool authorization leaks, missing audit trails, unsourced claims, overconfident responses
→ MANIPULATION: hallucination, privilege escalation, prompt injection, malicious deployer rules
→ DECEPTION: evaluation-awareness sandbagging, covert side tasks, long-horizon drift, goal stability
→ UNPREDICTABILITY: context distortion, instruction drift, decision stability across runs
→ OPACITY: regulatory readiness, session leakage, training-contamination attestation, escalation paths
Two mandatory minimums are baked in. Tool authorization must hit 100%. Privilege escalation must hit 95%. Fail either one and your overall score gets capped at 60% no matter how well you did everywhere else.
Every run writes a content-addressed manifest that captures every input. So you can drop it into CI, track drift over weeks, and prove your agent isn't quietly degrading between deploys.
The Full mode is what makes it serious. You bring two different judge providers and it runs a multi-judge ensemble with conservative tie-break and per-judge attribution. No silent self-judging. If you only have one credential, the run literally refuses to compare systems for you.
Industry agnostic by design. The test code is domain-neutral. Healthcare, customer support, software engineering knowledge lives in fixture YAML you write yourself. Three example fixtures ship in the repo.
It also maps every test to OWASP LLM Top 10, NIST AI RMF, EU AI Act, and ISO 42001. One flag and you get a regulatory gap analysis.
Built by @ifixai_ai.
Apache 2.0. 100% Opensource.
