# Promptfoo — AI Coding Tool Review & Pricing

> Promptfoo is an open-source CLI and library for evaluating and red-teaming LLM-based applications and agents. You declare test cases and assertions, compare prompts and models side by side, run automated red teaming and vulnerability scans against agents and RAG pipelines, and wire the checks into CI/CD and pull request code scanning.

Source: https://ai.dosa.dev/tools/promptfoo · Updated Oct 23, 2026 · Category: AI Testing & Quality Assurance · Published by dosa.dev

## At a glance

- **Tool**: Promptfoo
- **Company**: Promptfoo (part of OpenAI)
- **Category**: [AI Testing & Quality Assurance](https://ai.dosa.dev/category/qa-testing)
- **Pricing model**: Open Source
- **Official website**: https://www.promptfoo.dev

## Key features

- Declarative evals with assertions, run via `promptfoo eval` and inspected in a local web viewer (`promptfoo view`)
- Side-by-side model comparison across OpenAI, Anthropic, Azure, Bedrock, Ollama and many other providers
- Red teaming and vulnerability scanning for LLM apps, agents and RAG pipelines
- CI/CD integration for automated regression checks
- Code scanning that reviews pull requests for LLM-related security and compliance issues
- Install with npm, Homebrew or pip, or run without installing via `npx promptfoo@latest`

## Pricing

**Model**: Open Source

Free and open source under the MIT license; install via npm, Homebrew or pip, or run with `npx promptfoo@latest`. You pay your own model provider API costs. Promptfoo joined OpenAI in March 2026 and states it remains open source and MIT-licensed; enterprise offerings are not priced in the repository.

## Who it is for

- **Best for**: Developers and AI teams shipping LLM features or agents who want repeatable evals, model comparisons and security red teaming in local runs and CI.
- **Not ideal for**: People looking for a coding agent or test generator for ordinary application code; Promptfoo tests LLM behaviour and requires writing eval configs and supplying provider API keys.

## Verdict

The most widely adopted open-source toolkit for LLM evals and red teaming, actively released and now backed by OpenAI while staying MIT-licensed; a default choice for testing prompts and agents in CI.

## Recent updates

Released 0.125.0 on October 23, 2026, adding new assertion types (ROUGE-L/ROUGE-S, trace span filters), eval safe mode and stability reporting, and making the Claude Agent and Codex SDK providers opt-in; the repository has 26K+ GitHub stars and appeared on GitHub Trending (TypeScript) on October 23, 2026. Promptfoo announced it was joining OpenAI on March 9, 2026.

## Frequently asked questions

### Is Promptfoo free?

Yes - Promptfoo is Open Source. Free and open source under the MIT license; install via npm, Homebrew or pip, or run with `npx promptfoo@latest`. You pay your own model provider API costs. Promptfoo joined OpenAI in March 2026 and states it remains open source and MIT-licensed; enterprise offerings are not priced in the repository.

### Is Promptfoo open source?

Yes - Promptfoo is open source. Free and open source under the MIT license; install via npm, Homebrew or pip, or run with `npx promptfoo@latest`. You pay your own model provider API costs. Promptfoo joined OpenAI in March 2026 and states it remains open source and MIT-licensed; enterprise offerings are not priced in the repository.

### Who is Promptfoo best for?

Developers and AI teams shipping LLM features or agents who want repeatable evals, model comparisons and security red teaming in local runs and CI.

### Who is Promptfoo not ideal for?

People looking for a coding agent or test generator for ordinary application code; Promptfoo tests LLM behaviour and requires writing eval configs and supplying provider API keys.

### What are the best Promptfoo alternatives?

The closest Promptfoo alternatives on ai.dosa.dev are GitAuto, AgentDiff, agent-qa - all listed under AI Testing & Quality Assurance.

### Who makes Promptfoo?

Promptfoo is developed by Promptfoo (part of OpenAI). It is listed in the AI Testing & Quality Assurance category on ai.dosa.dev.

## Top alternatives

- [GitAuto](https://ai.dosa.dev/tools/gitauto) (Freemium) — Teams using GitHub that struggle with low test coverage or lack dedicated time for manual test maintenance in their legacy or rapidly evolving codebases.
- [AgentDiff](https://ai.dosa.dev/tools/agentdiff) (Freemium) — Engineers building multi-turn, tool-calling agents who need to verify agent actions and maintain provenance in production environments.
- [agent-qa](https://ai.dosa.dev/tools/agent-qa) (Open Source) — Engineering teams looking for a repository-native AI testing workflow that supports both web and native mobile platforms without relying on proprietary SaaS platforms.
- [e2e](https://ai.dosa.dev/tools/e2e) (Open Source) — Teams looking for a reliable, hybrid approach to E2E testing that combines the precision of scripted assertions with the adaptability of AI agents for complex or frequently changing multi-step flows.
- [iFixAi](https://ai.dosa.dev/tools/ifixai) (Freemium) — Engineering and risk teams running agents that handle sensitive tasks such as issuing refunds, accessing customer data, or calling internal tools.
- [ARTEMIS](https://ai.dosa.dev/tools/artemis) (Open Source) — Mobile developers and QA engineers who want AI assistants to execute and diagnose Android end-to-end tests on real hardware.
- Full alternatives guide: https://ai.dosa.dev/tools/promptfoo/alternatives