LangSmith vs Ragas
Of the 117 makers' products we know the LLM observability & evals of, 11 chose LangSmith (9%) and 0 Ragas (0%). Among the 33 launched in 2025, LangSmith is in 18% and Ragas in 0%.
Two sides of the LLM observability & evals decision: hosted eval platform and eval and test CLI. When each fits, what it costs, who moves from one to the other, and what makers who chose it say.
Which fits you
- Your app is built on LangChain or LangGraph
Use it whenYou already use the LangChain stack.
Trade-offPaid per seat beyond the free tier; self-hosting is enterprise-only.
- Scoring RAG pipelines — how faithful answers are to the retrieved context and how relevant that context is — and generating test questions from your documents.
Use it whenYour product answers from your own documents and you need numbers to compare chunking, retrieval or prompt changes.
Trade-offA Python library, not a platform — no tracing of live traffic or UI, and LLM-judged metrics cost tokens per run.
At a glance
| Used by | 11 makers' products · 62 open-source projects | No maker's product yet · 14 open-source projects |
|---|---|---|
| Cost at default usagetraces 100k traces | $475/mo Developer | — |
| Downloads | 6.2M/wk−8% vs npm | 239k/wk |
| Pricing | Free tier with a monthly trace limit; paid per-seat plan with usage overage; custom Enterprise, including self-hosted. · paid from $39/seat/mo | Free (open source). |
| Free tier | Yes | Yes |
| Open source | No | Yes · self-hostable |
| Incidents, 90 daysfrom its status page | 0 | no public status feed |
Which makers on your framework pick
Makers' products built with each framework that chose either one, for LLM observability & evals.
| Framework | |||
|---|---|---|---|
| 4 | 0 | All 48 Next.js products' picks → |
New launches vs older products
A tool that came first has products from every year since, so its total runs ahead. Here is what products launched each year chose, as a share of those that made the LLM observability & evals decision (by the year they launched on Product Hunt).
| Launched | Products | ||
|---|---|---|---|
| 2024 | 19% 4 | 0% 0 | 21 |
| 2025 | 18% 6 | 0% 0 | 33 |
Cost as you grow
Both cost $0 up to 1k traces; from 10k traces Ragas costs less ($0 vs $25); and still does at 10M traces ($0 vs $49,975). They're different kinds of tool — hosted eval platform and eval and test CLI — so the prices don't buy the same thing.
The numbers, plan by plan
| Traces per month | LangSmith | Ragas |
|---|---|---|
| 1,000 | $0 Developer | $0 Self-hosted (open source) |
| 10,000 | $25 Developer | $0 Self-hosted (open source) |
| 50,000 | $225 Developer | $0 Self-hosted (open source) |
| 100,000 | $475 Developer | $0 Self-hosted (open source) |
| 500,000 | $2,475 Developer | $0 Self-hosted (open source) |
| 1,000,000 | $4,975 Developer | $0 Self-hosted (open source) |
| 5,000,000 | $24,975 Developer | $0 Self-hosted (open source) |
| 10,000,000 | $49,975 Developer | $0 Self-hosted (open source) |
What makers say
Makers on using it for LLM observability & evals, from Product Hunt and Starter Story interviews, each linked to the source. Products with a page of their own and fuller notes first.
LangSmith’s real-time analytics and versioning keep our AI agents rock-solid -- so everything just works better.
You can't build AI agents without monitoring. Metadata filtering is strong.
I deployed the manage Pig agent on LangGraph and it's been smooth sailing!
No maker quote about Ragas for LLM observability & evals yet.
Loved and watch-outs
Themes that recur in makers' words and Hacker News comments, each linked to what it summarises.
- It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. HNHN 2HN 3HN 4
- Traces show which agent failed but not why, so root-cause analysis stays manual. HNHN 2HN 3HN 4
- Viewing your own traces requires a cloud account, with no local-first option. HNHN 2
Who uses each
What makers pair each with
Neither quite right?
See the whole LLM observability & evals decision →Questions
LangSmith or Ragas: which do makers choose?
Of the 117 makers' products we know the LLM observability & evals of, 11 chose LangSmith (9%) and 0 Ragas (0%). Among the 33 launched in 2025, LangSmith is in 18% and Ragas in 0%. Choose LangSmith if your app is built on LangChain or LangGraph; choose Ragas if scoring RAG pipelines — how faithful answers are to the retrieved context and how relevant that context is — and generating test questions from your documents.
When should I pick LangSmith over Ragas?
Pick LangSmith if: Your app is built on LangChain or LangGraph.
How do LangSmith and Ragas compare in cost as you grow?
Both cost $0 up to 1k traces; from 10k traces Ragas costs less ($0 vs $25); and still does at 10M traces ($0 vs $49,975). Cheapest usable plan at each scale, at list prices.

