Why it's useful
You write simple test cases and it checks your AI's answers, compares models side by side and tries to trick it into leaking or misbehaving. Good for anyone shipping something with a model in it.
What it needs
API keys for whichever models you test, or a local model
Image: Promptfoo (now OpenAI). Promptfoo belongs to Promptfoo (now OpenAI); we just think it's worth knowing about.
More tools like this
Open source · MIT 37.6k stars on GitHub
Coding
Claude Code Router
by musistudio
Runs Claude Code against whichever model you pick, including local and cheaper ones.
Open source · Apache-2.0 37.4k stars on GitHub
Coding
claude-code-setup
by Anthropic
Anthropic's official plugin that looks over your project and suggests the Claude Code setup to match.
Open source · Apache-2.0 69.9k stars on GitHub
Coding
Cline
by Cline
An open-source coding agent that lives in VS Code, or on the command line.