Updated

Promptfoo alternatives

For LLM observability & evals, the alternatives to Promptfoo that makers' products we track chose most are Datadog (50), Langfuse (35) and Helicone (13). 15 of the 25 are open source or self-hostable (Langfuse, Helicone, Arize Phoenix and 12 more) and 17 have a free tier. At 10,000 users, the cheapest is Langfuse at $29 a month on Core. Promptfoo also answers AI guardrails & security: those alternatives are below too.

15 open source or self-hostable17 with a free tier9 ready for AI agents

Not a ranking: each option is described by the situation it fits, with its trade-off.

Promptfoo alternatives compared for LLM observability & evals

How many makers' products we track use each for LLM observability & evals, and what each costs at 10,000 users.

OptionKindMakers' productsAt 10,000 usersFree tierOpen source
PromptfooEval and test CLI0—YesYes
LangfuseOpen-source platform35$29/mo · CoreYesYes
HeliconeProxy logging13$116/mo · ProYesYes
LangSmithHosted eval platform11$475/mo · DeveloperYesNo
BraintrustHosted eval platform6—YesNo
Arize PhoenixOpen-source platform0—YesSelf-hostable
DatadogHosted eval platform50——No
Weights & BiasesHosted eval platform3—YesSelf-hostable
RespanHosted eval platform2—YesNo
OpikOpen-source platform0—YesYes
DeepEvalEval and test CLI0—YesYes
RagasEval and test CLI0—YesYes
MLflowOpen-source platform0—YesYes
LaminarOpen-source platform0——Yes
AgentOpsHosted eval platform0—YesYes
OpenLLMetryOpen-source platform0—YesYes
HoneyHiveHosted eval platform0—YesSelf-hostable
GalileoHosted eval platform0—YesSelf-hostable
PromptLayerHosted eval platform0—YesNo
LunaryHosted eval platform0—NoSelf-hostable
Confident AIHosted eval platform0—YesNo
HumanloopHosted eval platform0——No
Patronus AIHosted eval platform0——No
AgentaOpen-source platform0——Yes
Parea AIHosted eval platform0——No
Athina AIHosted eval platform0——No

Promptfoo alternatives compared for AI guardrails & security

How many makers' products we track use each for AI guardrails & security.

OptionKindMakers' productsFree tierOpen source
PromptfooRed-team testing0YesYes
LakeraHosted screening API1—No
NeMo GuardrailsOpen-source library0YesYes
Guardrails AIOpen-source library0YesYes
NightfallEnterprise platform1NoNo
Prompt SecurityEnterprise platform1—No
PangeaHosted screening API0YesNo
ArthurMonitoring with guardrails0YesSelf-hostable
HiddenLayerEnterprise platform0NoNo
Lasso SecurityEnterprise platform0NoNo
Protect AIEnterprise platform0—No
CalypsoAIEnterprise platform0—No
Robust IntelligenceEnterprise platform0—No

When each fits: LLM observability & evals

The whole decision →

Open-source platform you can self-host, or a hosted eval product. Open-source tools keep traces on your own servers and cost nothing to run yourself, but you maintain them. Hosted eval platforms give you polished experiment and scoring workflows for a monthly bill, with your data in their cloud.

Promptfoo (Eval and test CLI): Running prompts and models against test cases from the command line or CI, including red-team tests for prompt injection and data leaks. Trade-off: A testing tool rather than production tracing, so you pair it with something that logs live traffic.

Langfuse

Open-source platform
Open sourceFree tierFrom $29/mollms.txtCLI

Best forTracing, prompt management and evals in one tool you can run on your own server for free.

Trade-offSelf-hosting means running Postgres, ClickHouse, Redis and object storage alongside it; otherwise it's a usage-based cloud plan.

Used by 35 makers' products for this

Helicone

Proxy logging
Open sourceFree tierFrom $79/mollms.txt

Best forGetting request logs, costs and latency by changing one base URL, with no SDK instrumentation.

Trade-offProxy logging sees individual requests well but agent steps and eval workflows less deeply; acquired by Mintlify in March 2026, so check its roadmap before building on it.

Used by 13 makers' products for this

LangSmith

Hosted eval platform
Free tierFrom $39/seat/mollms.txt

Best forApps built on LangChain or LangGraph, where tracing works with almost no setup.

Trade-offPaid per seat beyond the free tier; self-hosting is enterprise-only.

Used by 11 makers' products for this

Braintrust

Hosted eval platform
Free tierFrom $249/mollms.txtMCPCLI

Best forEval-driven work — scoring outputs and comparing prompts and models side by side in experiments.

Trade-offClosed source, and the paid tier is priced for teams rather than hobby projects.

Used by 6 makers' products for this

Arize Phoenix

Open-source platform
Self-hostableFree tierFrom $50/mollms.txt

Best forOpenTelemetry-based tracing and evals you can start locally in a notebook, then self-host or move to its cloud.

Trade-offSource-available under the Elastic License rather than a permissive open-source license.

No maker product tracked for this yet

Datadog

Hosted eval platform

Best forTraces, token costs, errors and quality and safety evaluations for LLM apps and agents, next to the APM and logs you already have in Datadog.

Trade-offBilled as another Datadog product on top of your plan, and only worth it if the rest of your monitoring lives there; formerly called LLM Observability.

Used by 50 makers' products for this

Weights & Biases

Hosted eval platform
Self-hostableFree tierFrom $60/mo

Best forTracing and evaluating LLM apps and agents with W&B Weave, in the same account you use for experiment tracking and a model registry when you train or fine-tune models.

Trade-offSelf-hosting needs a licensed W&B Server on Kubernetes with ClickHouse; the free plan ingests 1 GB of trace data a month; now sold as CoreWeave Forge, which names its Agent Lens product as Weave's successor for agent tracing.

Used by 3 makers' products for this

Respan

Hosted eval platform
Free tierFrom $199/mo

Best forTracing, monitoring, prompt management and LLM-judge evals for agents, with a built-in gateway that logs every call it routes.

Trade-offCloud-only below Enterprise; the free plan keeps logs 7 days, and Team starts at $199/mo billed yearly.

Used by 2 makers' products for this

Opik

Open-source platform
Open sourceFree tierFrom $19/mollms.txtMCP

Best forTracing, evaluations and prompt optimization in an Apache-licensed platform you self-host with Docker or Kubernetes, or use on Comet's cloud.

Trade-offSelf-hosting means running several services; smaller community than Langfuse.

No maker product tracked for this yet

DeepEval

Eval and test CLI
Open sourceFree tierFrom $200/mo (Confident AI Starter)llms.txtCLI

Best forWriting LLM evals as pytest-style tests in Python, with ready-made metrics for hallucination, faithfulness and answer relevancy.

Trade-offThe metrics use an LLM as a judge, so every run costs tokens and scores vary slightly; dashboards and history are in the paid Confident AI platform.

No maker product tracked for this yet

Ragas

Eval and test CLI
Open sourceFree tierllms.txt

Best forScoring RAG pipelines — how faithful answers are to the retrieved context and how relevant that context is — and generating test questions from your documents.

Trade-offA Python library, not a platform — no tracing of live traffic or UI, and LLM-judged metrics cost tokens per run.

No maker product tracked for this yet

MLflow

Open-source platform
Open sourceFree tierllms.txt

Best forTracing, evaluating and versioning prompts for LLM apps in the same open-source platform many teams already use for ML experiments.

Trade-offGrew out of classic ML tooling, so the UI and concepts are broader than an LLM-only tool; you run the tracking server yourself unless you use a managed version.

No maker product tracked for this yet

Laminar

Open-source platform
Open source

Best forTracing, evals and debugging built for multi-step agents, self-hosted or hosted.

Trade-offSelf-hosting means running its services yourself; the hosted version saves that work.

No maker product tracked for this yet

AgentOps

Hosted eval platform
Open sourceFree tierFrom $40/mo

Best forTracing, replaying and costing multi-agent runs from a Python SDK that hooks into CrewAI, AutoGen, OpenAI and other frameworks.

Trade-offThe Python SDK hasn't had a PyPI release since August 2025, so check that it supports your framework versions.

No maker product tracked for this yet

OpenLLMetry

Open-source platform
Open sourceFree tier

Best forOpen-source OpenTelemetry instrumentation that traces model, vector store and framework calls and sends them to any OpenTelemetry backend.

Trade-offInstrumentation only — you still need a backend to store and view traces; Traceloop, its maker, is joining ServiceNow.

No maker product tracked for this yet

HoneyHive

Hosted eval platform
Self-hostableFree tierFrom Custom (Enterprise)

Best forTracing, online evaluation of live traffic, offline experiments on datasets, annotation queues and prompt versioning for agents.

Trade-offThe free plan stops at 10,000 events a month and 30 days of retention, and the only paid step is a custom-priced Enterprise plan.

No maker product tracked for this yet

Galileo

Hosted eval platform
Self-hostableFree tierFrom $100/mo (billed annually)

Best forScoring every production request with small Luna evaluator models distilled from LLM-judge metrics, then turning those evals into guardrails.

Trade-offNow owned by Cisco and priced through Splunk, billed annually; built for enterprise teams more than small ones.

No maker product tracked for this yet

PromptLayer

Hosted eval platform
Free tierFrom $49/mo

Best forA prompt registry non-engineers can edit and deploy without a redeploy, with request logs, traces and dataset evals tied to prompt versions.

Trade-offHosted only below Enterprise; the free plan logs 2,500 requests a month, and usage past plan limits is billed per request.

No maker product tracked for this yet

Lunary

Hosted eval platform
Self-hostableFrom $20/user/mo

Best forLogging LLM calls with user and conversation analytics, prompt templates and human review.

Trade-offNo free workspace for new sign-ups ($20 per user a month), and self-hosting is now an Enterprise feature.

No maker product tracked for this yet

Confident AI

Hosted eval platform
Free tierFrom $200/mo

Best forDashboards, history and regression tracking for DeepEval test runs, plus tracing, online evals and red-teaming in one hosted platform.

Trade-offThe free plan allows 5 test runs a week and one project; the next step is $200 a month.

No maker product tracked for this yet

Humanloop

Hosted eval platform

Best forTeams managing prompts, running evaluations and watching LLM apps in production.

Trade-offShut down on 2025-09-08 after the team joined Anthropic; all data was deleted, and Humanloop pointed users to Langfuse or Braintrust.

No maker product tracked for this yet

Patronus AI

Hosted eval platform

Best forEvaluating and optimizing LLM apps and agents with research-based evaluators.

Trade-offPivoted to a research lab; there is no self-serve evaluation product on its site.

No maker product tracked for this yet

Agenta

Open-source platform
Open source

Best forAn open-source, self-hostable place to manage prompts, run evaluations and trace LLM apps.

Trade-offPivoted on 2026-07-21 to a workspace for building agents; no longer an observability product.

No maker product tracked for this yet

Parea AI

Hosted eval platform

Best forExperiment tracking, evaluations and human annotation of LLM app logs.

Trade-offThe site returns HTTP 402 and the SDK stopped in February 2025; apparently shut down without notice.

No maker product tracked for this yet

Athina AI

Hosted eval platform

Best forMonitoring LLM apps in production with automatic hallucination detection and evals.

Trade-offThe site and app are down and the SDK stopped in May 2025; apparently shut down without notice.

No maker product tracked for this yet

When each fits: AI guardrails & security

The whole decision →

An open-source library in your code, a hosted screening API, or an enterprise security platform. Libraries run validators and rules inside your app for free, but you tune them and pay the latency. Hosted APIs check prompts and outputs against detectors the vendor keeps up to date, billed by usage. Enterprise platforms add discovery, red teaming and policy across every AI tool a company uses, sold through sales. Most of this market has been bought by security companies in the past year, so check who now owns a vendor before you depend on it.

Promptfoo (Red-team testing): Red-team test suites for prompt injection, jailbreaks and data leaks, run from the command line or CI. Trade-off: A test tool, not a runtime filter; you still need a guardrail on live traffic.

Lakera

Hosted screening API

Best forOne API call that screens prompts, retrieved content and outputs for prompt injection, data leaks and harmful content, in under 50 ms.

Trade-offNow Check Point AI Guardrails, with sign-up through a Check Point account and no public price list.

Used by 1 maker's product for this

NeMo Guardrails

Open-source library
Open sourceFree tier

Best forProgrammable rails configured in YAML and Colang flows to block, alter or validate inputs and responses, maintained by NVIDIA.

Trade-offPython only, and Colang is a language of its own to learn; checks that call an LLM add latency and cost.

No maker product tracked for this yet

Guardrails AI

Open-source library
Open sourceFree tier

Best forWrapping LLM calls with validators from the Guardrails Hub for PII, toxicity, hallucination and output format.

Trade-offHarvey acquired the company in September 2026 and said nothing about the framework's future, though the repo is still active.

No maker product tracked for this yet

Nightfall

Enterprise platform
From Contact sales

Best forData loss prevention across SaaS, endpoints and AI tools, plus APIs to scan text and files for sensitive data in your app.

Trade-offPriced per user per year through sales; a DLP product first, not a prompt-injection filter.

Used by 1 maker's product for this

Prompt Security

Enterprise platform

Best forSecuring a company's AI use across employees' tools, its own AI apps and code assistants.

Trade-offAcquired by SentinelOne and sold as part of it; demo-only, no public pricing.

Used by 1 maker's product for this

Pangea

Hosted screening API
Free tierFrom Pay as you go

Best forAI Guard and Prompt Guard APIs that detect injection, redact PII and disarm malicious content, billed per use from a free account.

Trade-offPart of CrowdStrike since 2025, so the roadmap now follows its Falcon platform.

No maker product tracked for this yet

Arthur

Monitoring with guardrails
Self-hostableFree tierFrom $60/mo

Best forModel and agent monitoring, evals and guardrails in one platform, with an MIT-licensed engine you can run yourself.

Trade-offMainly an observability and governance product; the free tier covers 4 use cases with 7-day retention.

No maker product tracked for this yet

HiddenLayer

Enterprise platform
From Contact sales

Best forDiscovery, model and supply-chain scanning, attack simulation and runtime guardrails across a company's AI.

Trade-offSales-led with no public pricing; more than a small team needs for one LLM feature.

No maker product tracked for this yet

Lasso Security

Enterprise platform
From Contact sales

Best forControlling employee AI use and securing agents, MCP servers and coding assistants with runtime policies.

Trade-offSales-led with no public pricing; built for security teams, not for adding a check to your own app.

No maker product tracked for this yet

Protect AI

Enterprise platform

Best forScanning LLM prompts and outputs with the open-source LLM Guard, plus enterprise model and AI app security.

Trade-offAcquired by Palo Alto Networks and folded into Prisma AIRS; LLM Guard has had no release since 2025-05-19.

No maker product tracked for this yet

CalypsoAI

Enterprise platform

Best forEnterprise guardrails that inspected prompts and responses in real time, plus red-teaming of AI apps.

Trade-offAcquired by F5 and now sold as F5 AI Guardrails.

No maker product tracked for this yet

Robust Intelligence

Enterprise platform

Best forValidating AI models for risks before launch and an AI firewall in production.

Trade-offAcquired by Cisco and folded into Cisco AI Defense.

No maker product tracked for this yet

Questions

What is the best alternative to Promptfoo?

It depends on what you need it for; we don't rank them. The one makers' products we track chose most for LLM observability & evals is Datadog (50 makers' products). You want tracing, prompts and evals in one tool you can self-host for free: Langfuse. You already use OpenTelemetry or want vendor-neutral instrumentation: Arize Phoenix. You want request logs and costs today by changing one base URL: Helicone.

Is there a free alternative to Promptfoo?

Yes. For LLM observability & evals, Langfuse, Helicone, LangSmith, Braintrust, Arize Phoenix, Weights & Biases and 11 more have a free tier a small product can start on.

Is there an open-source alternative to Promptfoo?

Yes. For LLM observability & evals: Langfuse, Helicone, Arize Phoenix (self-hostable), Weights & Biases (self-hostable), Opik, DeepEval and 9 more.

What is the cheapest alternative to Promptfoo?

At 10,000 users, the cheapest is Langfuse at $29 a month on Core. From each vendor's pricing page.