“For monitoring AI Model performance.”
LLM Observability & EvalsMaker says so · source ↗
LangSmith
Observability and evaluation platform for LLM agents, with tracing, monitoring, and dataset-based testing; framework-agnostic but built by the LangChain team.
11 makers' products we track use LangSmith. For LLM observability & evals it's 4th most chosen of 7 options: 11 of the 117 makers' products we know the LLM observability & evals of (9%). 62 open-source projects also declare it in their code. What makers like most: Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. Their most common complaint: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. On GitHub, 0 public pull requests moved projects to LangSmith from tools we list, and 9 moved them away.
Profile checked
Works with AI agents:llms.txtHow makers use LangSmith
Makers' products we track use LangSmith for LLM observability & evals (11), most often alongside OpenAI API (45%).
What LangSmith is used for
LangSmith vs the alternatives makers chose
For LLM observability & evals, 11 of the 117 makers' products we know the choice of picked LangSmith (9%), behind Datadog (43%).
| Option | Makers' products | Free for | At 10,000 users | Open source | |
|---|---|---|---|---|---|
| 50 43% | — | — | No | ||
| 35 30% | ~5,000 users | $29/mo · Core | Yes | Compare → | |
| 13 11% | ~1,000 users | $116/mo · Pro | Yes | Compare → | |
| 11 9% | ~520 users | $475/mo · Developer | No | ||
| 6 5% | — | — | No | Compare → | |
| 3 3% | — | — | No |
For LLM observability & evals: how many of the 117 makers' products we know the LLM observability & evals of chose each option, and what each costs at 10,000 users.
Costs are the cheapest plan that fits, from each vendor's pricing page, at the usage our stack planner assumes for a product with that many users. Free for: the users its cheapest plan a business can use stays free for.
Often paired with
Share of the 11 makers' products on LangSmith that show this choice.Markets that build with it most
Product Hunt markets where LangSmith is in far more of the stacks we know than across every market (0%).
When to use LangSmith, and when not
Pick LangSmith if
- Your app is built on LangChain or LangGraph. LLM Observability & Evals →
Something else fits better if
- You want tracing, prompts and evals in one tool you can self-host for free: Langfuse.
- You already use OpenTelemetry or want vendor-neutral instrumentation: Arize Phoenix.
- You want request logs and costs today by changing one base URL: Helicone.
- Your main question is whether a prompt or model change made outputs better or worse: Braintrust.
LangSmith is LangChain's hosted platform for watching and testing LLM agents. It records each run of your app as a trace of nested steps (model calls, tools, retrieval) so you can see why an agent did what it did, then turns those runs into datasets and evaluations so you can check that a prompt or model change made things better rather than worse.
More about LangSmith
You instrument code with its SDKs for Python, TypeScript, Go or Java, typically by wrapping your LLM client or marking functions as traceable; LangChain and LangGraph apps switch it on with an environment variable. It also accepts OpenTelemetry traces on an OTLP endpoint, and the Vercel AI SDK has a dedicated integration. In JavaScript, runs are sent in the background by default so tracing doesn't add latency.
For a small team the useful parts are trace search with dashboards and alerts, online evaluators (LLM-as-a-judge) that score production traffic, offline experiments over datasets, and a prompt playground. The same account can also host agent deployments. The managed cloud runs in US, EU and APAC regions on GCP and a US region on AWS, and you pick the region at sign-up.
Traces are kept 14 days at base retention, with longer retention billed separately. Self-hosting is only available as an Enterprise add-on, and an organization can't move between regions after sign-up.
How LangSmith itself is built
1 tools, from its own code, website and Product Hunt page.
LangSmith pros and cons
What makers like most: Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. Their most common complaint: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in.
Pros
Cons and watch-outs
- It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. HNHN 2HN 3HN 4
- Traces show which agent failed but not why, so root-cause analysis stays manual. HNHN 2HN 3HN 4
- Viewing your own traces requires a cloud account, with no local-first option. HNHN 2
- The prompt engineering workflow is laborious, gets expensive fast and handles multi-turn prompts poorly. HNHN 2
Themes that recur in makers' words and Hacker News comments, each linked to what it summarises, with how Product Hunt tags its reviews.
4.8★ from 19 reviews on Product Hunt ↗Reliability and open issues
LangSmith's status page posted no incidents in the last 90 days.
Incidents on its status page
0 in the last 90 days · 0 in the last yearFrom its own status page, 2026-10-19. Vendors decide what they post; one page can cover several products.Who switches
More projects move away from LangSmith: 0 pull requests in, 9 out; the most leave for Langfuse.
LangSmith → Langfuse9 PRs
- Migrate observability from LangSmith to Langfuse and trim logged fieldsHOSH19/HarnessLab · 2026-08-29
- feat: migrate observability from LangSmith to Langfuse Cloud (v4 SDK)icekarim/momo-assistant · 2026-08-05
- feat(trace): migrate LangSmith to Langfuse v4 and fix env key priority1935494577/Xiaoxin-Agentic-RAG-System · 2026-08-01
- [codex] Migrate from LangSmith to Langfuse, remove Python legacy, fix review issuesBunnyRabbit8mile/codex-tee · 2026-07-14
- Reapply "feat: migrate observability from LangSmith to Langfuse"sinuarlowbaby/RAG-PDF-Chatbot · 2026-07-14
- refactor: migrate observability from LangSmith to Langfusewei-yiting/fin-lab-x · 2026-03-18
- Migrate observability from LangSmith to LangfuseAneeshPulukkul/hybrid-rag-solution · 2026-03-14
- feat:migrate LLM observabilty from LangSmith to LangFuse100-hours-a-week/17-JinyUs-Q-Feed-AI · 2026-02-26
- Refactor: Migrate from Langsmith to Langfusedylangamachefl/fantasy-football-chatbot-v2 · 2025-12-01
Public pull requests on GitHub since Oct 2024 whose title says "X to Y" — real code changes moving a project from one tool to another, by developers in general.
Who uses it
11 makers' products, each linked to the source that shows it, and 62 open-source projects that declare it in their code. The best known on Product Hunt: Watchman AI (874 followers), GitLaw (868 followers), Meet-Ting (803 followers), DryMerge (688 followers), NeuralAgent (638 followers).
“Evals / tracing / monitoring is awesome and out of the box”
LLM Observability & EvalsMaker says so · source ↗“We absolutely could not build this product with LangSmith for traces, datasets, annotation queues, and evals. We've become big power users! I use to build AI apps before nice tracing tools like this existed and it's like night and day having a tool like this. We considered Langfuse, but I already had experience with LangSmith so for speed purposes we went with LangSmith.”
LLM Observability & EvalsMaker says so · source ↗“Shoutout for all the observability and prompt management capabilities!”
LLM Observability & EvalsMaker says so · source ↗“You can't build AI agents without monitoring. Metadata filtering is strong.”
LLM Observability & EvalsMaker says so · source ↗“I deployed the manage Pig agent on LangGraph and it's been smooth sailing!”
LLM Observability & EvalsMaker says so · source ↗“LangSmith’s real-time analytics and versioning keep our AI agents rock-solid -- so everything just works better.”
LLM Observability & EvalsMaker says so · source ↗“Our preferred way to track and analyze tool usage. Really easy to handle.”
LLM Observability & EvalsMaker says so · source ↗“Promptius <3 Langchain + Langgraph + Langsmith, this combination has been instrumental in building Promptius in the past 5 months! The abstractions provided by Langchain and Langgraph make building agents as easy as writing a prose. Langsmith provides unmatched observability and also help track our costs.”
LLM Observability & EvalsMaker says so · source ↗“We started building Ting without LangSmith and we were flying blind. Diagnosing issues downstream felt like guesswork. Since bringing LangSmith in, we’ve been able to trace problems through the entire prompt path and actually understand what’s going on. We’re still evolving how we use it, but even early on it’s helped us move faster and get closer to that “this feels good” moment.”
LLM Observability & EvalsMaker says so · source ↗None of these in makers' own words.
Open source: a project that declares LangSmith as a dependency in its public code — verifiable, but not necessarily a live product.
What its projects tell coding agents
15 of the 30 open-source products built with LangSmith that were worked on in the last 180 days keep instruction files for coding agents — 50%, against 53% of all the projects we read. AGENTS.md is in 11, CLAUDE.md in 8.
Alternatives to LangSmith
All alternatives by situation →Questions makers ask about LangSmith
Is LangSmith good for production?
11 makers' products we track run on it. Its own status page posted 0 incidents in the last 90 days; vendors decide what they post there, so this doesn't compare across tools. On GitHub, 0 public pull requests moved projects to LangSmith from tools we list, and 9 moved them away. The watch-out makers raise most: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in.
What is LangSmith used for?
Observability and evaluation platform for LLM agents, with tracing, monitoring, and dataset-based testing; framework-agnostic but built by the LangChain team. Makers' products we track use it for LLM observability & evals (11).
What are the pros and cons of LangSmith?
Pros makers name: Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. Traces, datasets, annotation queues and evals live in one platform widely used by enterprises. Pairs tightly with LangChain and LangGraph for debugging stateful agent workflows. Cons: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. Traces show which agent failed but not why, so root-cause analysis stays manual. Viewing your own traces requires a cloud account, with no local-first option.
What are the alternatives to LangSmith?
For LLM observability & evals, the other options makers' products we track chose most are Datadog (50), Langfuse (35), Helicone (13), Braintrust (6), Weights & Biases (3). Open source or self-hostable among them: Langfuse, Helicone, Weights & Biases. Each alternative, with the situation it fits, is on the LangSmith alternatives page.
How much does LangSmith cost per month?
At 10,000 users, LangSmith costs $475 a month on the cheapest plan that fits (Developer), from its pricing page.
Do developers switch to or away from LangSmith?
On GitHub, 0 public pull requests moved projects to LangSmith from tools we list; 9 moved them away, most to Langfuse (9).
When should you use LangSmith?
Your app is built on LangChain or LangGraph (LLM observability & evals). Something else fits better if: you want tracing, prompts and evals in one tool you can self-host for free, pick Langfuse; you already use OpenTelemetry or want vendor-neutral instrumentation, pick Arize Phoenix.
Do I have to use LangChain to use it?
No. The SDKs trace any code and are documented as usable without LangChain; LangChain and LangGraph apps only get a shortcut, enabling tracing with a single environment variable. source ↗
Can I keep my data in the EU?
Yes. LangSmith has an EU instance in the Netherlands (GCP europe-west4), plus US and APAC instances, available on every plan including the free one. Billing data stays in the US, and you can't switch an organization between regions later. source ↗
Is it SOC 2 and HIPAA compliant?
LangChain states LangSmith is SOC 2 Type 2 certified, HIPAA compliant and GDPR compliant, and offers a DPA through support. It has no EU legal entity for contracting. source ↗
Can I send OpenTelemetry traces instead of using its SDK?
Yes. LangSmith accepts OTLP traces at its /otel endpoint, and you can use an OpenTelemetry Collector to send the same spans to LangSmith and another backend such as Datadog or Grafana. source ↗
Will tracing slow down my serverless functions?
The JS SDK sends traces in the background by default. In serverless runtimes you either turn background tracing off or await pending trace batches before the function returns, so data isn't lost when the runtime freezes. source ↗
How long are traces kept?
Base traces are kept 14 days and extended traces 180 days. Some actions, like online evaluators and automation rules, can upgrade a trace to extended retention; after retention ends, trace inputs and outputs are deleted. source ↗
Can I self-host it?
Only as an add-on to the Enterprise plan, installed on Kubernetes with a license key from LangChain's sales team. source ↗
Is LangSmith free?
Yes — there is a free tier a small product can run on; paid use starts at $39/seat/mo. source ↗
Is LangSmith open source or self-hostable?
Not open source, and hosted only.
Can AI coding agents work with LangSmith?
It serves an llms.txt docs index.
Who uses LangSmith?
11 makers' products we track, each with a source, and 62 open-source projects declare it in their code. The best known of them on Product Hunt: Watchman AI (874 followers), GitLaw (868 followers), Meet-Ting (803 followers), DryMerge (688 followers), NeuralAgent (638 followers). source ↗