LLM Observability & Evals for Next.js: what 48 makers' products chose
Among 48 makers' products built with Next.js that we know the LLM observability & evals of, Datadog leads: 24 of them (50%). Next come Langfuse (12) and Helicone (4).
Open-source platform you can self-host, or a hosted eval product. Open-source tools keep traces on your own servers and cost nothing to run yourself, but you maintain them. Hosted eval platforms give you polished experiment and scoring workflows for a monthly bill, with your data in their cloud.
What makers on Next.js chose
| Option | Next.js products | All makers | Free for | At 10,000 users |
|---|---|---|---|---|
| 24 (50%) | 43% | — | — | |
| 12 (25%) | 30% | ~5,000 users | $29/moCore | |
| 4 (8%) | 11% | ~1,000 users | $116/moPro | |
| 4 (8%) | 9% | ~520 users | $475/moDeveloper | |
| 4 (8%) | 5% | — | — | |
| 2 (4%) | 2% | — | — |
Live makers' products with a source for both: that they're built with Next.js, and what they use for LLM observability & evals. A product using two options counts in both; not market share. "All makers" is the option's share among every maker's product that made this decision.
Costs are the cheapest plan that fits, from each vendor's pricing page, at the usage our stack planner assumes for a product with that many users. Free for: the users its cheapest plan a business can use stays free for; plans for personal use only, like a non-commercial hobby tier, don't count.
Which to pick, by situation
- You want tracing, prompts and evals in one tool you can self-host for free
Langfuse 12 here - You want request logs and costs today by changing one base URL
Helicone 4 here - Your app is built on LangChain or LangGraph
LangSmith 4 here - Your main question is whether a prompt or model change made outputs better or worse
Braintrust 4 here
Datadog
Hosted eval platform24 Next.js products here (50%).
Best forTraces, token costs, errors and quality and safety evaluations for LLM apps and agents, next to the APM and logs you already have in Datadog.
Trade-offBilled as another Datadog product on top of your plan, and only worth it if the rest of your monitoring lives there; formerly called LLM Observability.
“Our core analytics infrastructure. Everything from dev to marketing analytics to ensure we can give the best uptime and issue resolution.”
“How we know Brew is healthy at 3am. Catches the issues users would have flagged before they get the chance.”
LLM Observability & EvalsMaker says so +1 · source ↗“Our go-to for observability and a team favourite.”
LLM Observability & EvalsMaker says so · source ↗“We use Datadog for system and agent observability in production and sandbox environments.”
LLM Observability & EvalsMaker says so · source ↗“Latency is the product. If the agent hesitates for a beat, the demo stops feeling human and the buyer disengages - so we need to see the whole path from speech to browser action to response, not guess which hop added 200ms. Datadog is where we watch that end to end.”
LLM Observability & EvalsMaker says so · source ↗“Our core analytics infrastructure. Everything from dev to marketing analytics to ensure we can give the best uptime and issue resolution.”
LLM Observability & EvalsMaker says so · source ↗
Langfuse
Open-source platform12 Next.js products here (25%). Free up to about 5,000 users. About $29 a month at 10,000 users (Core).
llms.txtCLIBest forTracing, prompt management and evals in one tool you can run on your own server for free.
Trade-offSelf-hosting means running Postgres, ClickHouse, Redis and object storage alongside it; otherwise it's a usage-based cloud plan.
“Marc gave me an in-person onboarding in SF - I found an issue in our LLM provider config just 30 minutes after the onboarding thanks to Langfuse. 10/10 recommendation”
- Detailed traces show each agent call and the context it pulled, which makes debugging agent behaviour much easier. 3 sources
- Token usage and cost are tracked across providers alongside quality, in one place. 3 sources
- Prompt management and experiments feel basic next to the tracing core. 3 sources
- Views are retrospective dashboards, so explaining one run's cost or failure still means reading trace trees by hand. 4 sources
Status page: 14 incidents in the last 90 days, 1 major (its own page, 2026-10-19).
“Langfuse gives us complete visibility into our AI workflows. From tracking token usage and costs across multiple LLM providers to tracing every generation and embedding operation, it's become essential for understanding how our users interact with AI. The tagging and session tracking features make debugging and optimization so much easier. Thank you for building such a powerful observability platform for LLMs!”
LLM Observability & EvalsMaker says so +1 · source ↗“Detailed tracing and evals for our AI workloads.”
LLM Observability & EvalsMaker says so · source ↗“Marc gave me an in-person onboarding in SF - I found an issue in our LLM provider config just 30 minutes after the onboarding thanks to Langfuse. 10/10 recommendation”
LLM Observability & EvalsMaker says so · source ↗“Best for LLM monitoring.”
LLM Observability & EvalsMaker says so · source ↗“LLM observability platform that just works!”
LLM Observability & EvalsMaker says so · source ↗“Langfuse is our choice when observability and tracing are the key. Our clients love their tool and we do too!”
LLM Observability & EvalsMaker says so · source ↗“Langfuse helps us manage LLM observability, metrics, and prompts.”
LLM Observability & EvalsMaker says so · source ↗
Helicone
Proxy logging4 Next.js products here (8%). Free up to about 1,000 users. About $116 a month at 10,000 users (Pro).
llms.txtBest forGetting request logs, costs and latency by changing one base URL, with no SDK instrumentation.
Trade-offProxy logging sees individual requests well but agent steps and eval workflows less deeply; acquired by Mintlify in March 2026, so check its roadmap before building on it.
“Helicone just works right out of the box – really helpful for us to dig into user issues and understanding how much money we're burning on LLM API calls.”
- Setup takes a couple of lines of code and works as a proxy, so it suits stacks outside Python too. 2 sources
- Custom properties attribute LLM cost, latency and usage to individual customers or features. 2 sources
- Acquired by Mintlify in March 2026; some developers report the product has since moved to maintenance mode. 2 sources
- Per-run cost breakdowns need manual labelling and custom SQL, since views are retrospective. 2 sources
“Great for debugging edge cases coming from real users!”
LLM Observability & EvalsMaker says so · source ↗“Observability, cost tracking and diagnosing how an LLM works is pretty hard without a tool like Helicone. It was a trusty tool in the making of this V2”
LLM Observability & EvalsMaker says so · source ↗“Helicone just works right out of the box – really helpful for us to dig into user issues and understanding how much money we're burning on LLM API calls.”
LLM Observability & EvalsMaker says so · source ↗
LangSmith
Hosted eval platform4 Next.js products here (8%). Free up to about 520 users. About $475 a month at 10,000 users (Developer).
llms.txtBest forApps built on LangChain or LangGraph, where tracing works with almost no setup.
Trade-offPaid per seat beyond the free tier; self-hosting is enterprise-only.
“Shoutout for all the observability and prompt management capabilities!”
- Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. 1 source
- Traces, datasets, annotation queues and evals live in one platform widely used by enterprises. 2 sources
- It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. 4 sources
- Traces show which agent failed but not why, so root-cause analysis stays manual. 4 sources
Status page: 0 incidents in the last 90 days (its own page, 2026-10-19).
“We absolutely could not build this product with LangSmith for traces, datasets, annotation queues, and evals. We've become big power users! I use to build AI apps before nice tracing tools like this existed and it's like night and day having a tool like this. We considered Langfuse, but I already had experience with LangSmith so for speed purposes we went with LangSmith.”
LLM Observability & EvalsMaker says so · source ↗“Evals / tracing / monitoring is awesome and out of the box”
LLM Observability & EvalsMaker says so · source ↗“Shoutout for all the observability and prompt management capabilities!”
LLM Observability & EvalsMaker says so · source ↗“For monitoring AI Model performance.”
LLM Observability & EvalsMaker says so · source ↗
Braintrust
Hosted eval platform4 Next.js products here (8%).
llms.txtMCPCLIBest forEval-driven work — scoring outputs and comparing prompts and models side by side in experiments.
Trade-offClosed source, and the paid tier is priced for teams rather than hobby projects.
- Setting up datasets and running evals against different models is quick. 3 sources
- A solid eval platform for checking whether prompts produce consistent results. 3 sources
- Its core dataset-and-pass-rate workflow is simple enough that teams say they could build it themselves. 2 sources
- Unlike Promptfoo or Laminar, it is closed source. 1 source
Status page: 8 incidents in the last 90 days, 7 major (its own page, 2026-10-19).
Respan
Hosted eval platform2 Next.js products here (4%).
Best forTracing, monitoring, prompt management and LLM-judge evals for agents, with a built-in gateway that logs every call it routes.
Trade-offCloud-only below Enterprise; the free plan keeps logs 7 days, and Team starts at $199/mo billed yearly.
“We run a lot of agents at octolane. slash-command skills, deal detection, follow-up generation. respan is how we see what they're actually doing in production. evals + observability in one place, built for agent workflows.”
“The easiest way to monitor Nia's LLM workflows”
LLM Observability & EvalsMaker says so · source ↗“We run a lot of agents at octolane. slash-command skills, deal detection, follow-up generation. respan is how we see what they're actually doing in production. evals + observability in one place, built for agent workflows.”
LLM Observability & EvalsMaker says so · source ↗Also chosen, and not seen yet
Real options for LLM observability & evals not seen among these Next.js products yet: Arize Phoenix.
Who switches, and to what
Public GitHub pull requests whose title says "X to Y", by developers in general, not only on Next.js.
- LangSmith → Langfuse 9 pull requests
Getting LLM observability & evals right
Start by logging every request with its prompt, output, latency and cost — a day of real traces teaches more than any benchmark. Save the bad outputs you find as a small dataset, and rerun it every time you change a prompt or model. Add automated scoring only after you've read enough traces to know what "good" means for your app.
- Logging full prompts and outputs that contain users' personal data to a third-party service without masking it or listing the vendor as a subprocessor.
- Changing a prompt in production with no saved examples to rerun, so a fix for one case silently breaks five others.
- Trusting an LLM-as-judge score before checking that it agrees with your own judgment on a few dozen examples.
The rest of the Next.js stack
What makers' Next.js products chose for their other decisions, the most common pick for each:
- Hosting: Vercel 1,078 of 1,312
- Transactional Email: Resend 493 of 830
- Database: Supabase 171 of 382
- Payments: Stripe 288 of 359
- LLM API: OpenAI API 222 of 301
- Authentication: Clerk 176 of 211
- Web Analytics: Google Analytics 105 of 197
- Product Analytics: PostHog 123 of 168
Questions
What do most Next.js apps use for LLM observability & evals?
Among 48 makers' products built with Next.js that we know the LLM observability & evals of, Datadog leads: 24 of them (50%). Next come Langfuse (12) and Helicone (4).
What does LLM observability & evals cost for a Next.js app with 10,000 users?
On our usage model at 10,000 users, Langfuse comes to about $29 a month (Core); Helicone comes to about $116 a month (Pro); LangSmith comes to about $475 a month (Developer). Prices are from each vendor's pricing page.
Which LLM observability & evals option stays free longest for a Next.js app?
Langfuse: on our usage model its cheapest plan stays free up to about 5,000 users.
How this is counted: a product is on Next.js when a live-product source says so (the maker, its website, a customer story or a subprocessor list), and the same for its LLM observability & evals. How evidence is collected.