Promptfoo

Promptfoo is an AI security platform that finds and fixes application vulnerabilities during development.

Promptfoo is an AI security platform designed to catch vulnerabilities during the development of AI applications, agents, and RAGs. It automates red teaming and testing by simulating real users to uncover direct and indirect prompt injections, jailbreaks, data leaks, and business rule violations. Developers and security teams can integrate it into CI/CD pipelines, GitHub, GitLab, and Jenkins to receive security findings and remediation guidance directly in pull requests. The platform offers a free open-source community version for local testing alongside enterprise options with team collaboration, continuous monitoring, and dashboards. It is built on a user community and trusted by enterprise organizations across industries.

Promptfoo at a glance

Best for: Software engineer
Runs on: CLI, API, self-hosted, open source
Pricing: Community open-source plan is free; Enterprise plan has custom pricing based on team size and needs.
Released: April 2023
Category: AI coding tool
Website: promptfoo.dev

Key features

Automated red teaming for agents and RAGs
Custom attack generation for direct and indirect prompt injections
CI/CD pipeline integration
Security findings and remediation guidance in pull requests
Continuous monitoring and tracking of fixes
Vulnerability scanning for over 50 vulnerability types
Team sharing and collaboration controls

Use it for

Catching AI vulnerabilities during development
Simulating user attacks on agents and RAGs
Running security scans in CI/CD pipelines
Tracking remediation progress across development teams

Alternatives to Promptfoo

code2prompt — Written in Rust for high speed and efficiency when handling large codebases.
PromptForce — Cuts token waste and provides relevant codebase context for AI coding agents.
PromptHub — It enables teams to manage, version, and evaluate prompts in a centralized UI.
Grim Prompter — It guides AI coding agents to produce evidence-traced threat models for repositories.
OpenAPPA — Tracks data flows algebraically to prevent prompt injection without breaking agents.
Finbot Ctf — Stops prompt-fooled AI agents from executing unauthorized tool calls and actions.
Composio — Equips AI agents with tools, managed user auth, and sandboxes across 1,500+ apps
Coucou — It provides open-source, real-time agent monitoring and approvals directly from the screen notch.
16x Prompt — Manage code context and compare multiple LLM responses side-by-side easily.
HackerOne — Combines AI agents with expert human researchers to prioritize and fix real risks.
Prompt Flow — It connects language models, prompts, and code for production deployment.
Leaked System Prompt AI — Explores how major AI systems are instructed and configured.

All Promptfoo alternatives