Updated

LangSmith

Observability and evaluation platform for LLM agents, with tracing, monitoring, and dataset-based testing; framework-agnostic but built by the LangChain team.

11 makers' products we track use LangSmith. For LLM observability & evals it's 4th most chosen of 7 options: 11 of the 117 makers' products we know the LLM observability & evals of (9%). 62 open-source projects also declare it in their code. What makers like most: Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. Their most common complaint: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. On GitHub, 0 public pull requests moved projects to LangSmith from tools we list, and 9 moved them away.

Free tierPaid from $39/seat/mo

Profile checked

Works with AI agents:llms.txt
Ask your AI about this, with this page as the source:ChatGPT ↗Claude ↗Perplexity ↗

How makers use LangSmith

Makers' products we track use LangSmith for LLM observability & evals (11), most often alongside OpenAI API (45%).

What LangSmith is used for

LLM Observability & Evals · Hosted eval platform →Apps built on LangChain or LangGraph, where tracing works with almost no setup.Trade-off: Paid per seat beyond the free tier; self-hosting is enterprise-only.
Datadog50Langfuse35Helicone13LangSmith11Braintrust6Makers' products on each, for LLM observability & evals

LangSmith vs the alternatives makers chose

For LLM observability & evals, 11 of the 117 makers' products we know the choice of picked LangSmith (9%), behind Datadog (43%).

OptionMakers' productsFree forAt 10,000 usersOpen source
Datadog50 43%——No
Langfuse35 30%~5,000 users$29/mo · CoreYesCompare →
Helicone13 11%~1,000 users$116/mo · ProYesCompare →
LangSmith11 9%~520 users$475/mo · DeveloperNo
Braintrust6 5%——NoCompare →
Weights & Biases3 3%——No

For LLM observability & evals: how many of the 117 makers' products we know the LLM observability & evals of chose each option, and what each costs at 10,000 users.

Costs are the cheapest plan that fits, from each vendor's pricing page, at the usage our stack planner assumes for a product with that many users. Free for: the users its cheapest plan a business can use stays free for.

Often paired with

Share of the 11 makers' products on LangSmith that show this choice.

Markets that build with it most

Product Hunt markets where LangSmith is in far more of the stacks we know than across every market (0%).

When to use LangSmith, and when not

Pick LangSmith if

Something else fits better if

  • You want tracing, prompts and evals in one tool you can self-host for free: Langfuse.
  • You already use OpenTelemetry or want vendor-neutral instrumentation: Arize Phoenix.
  • You want request logs and costs today by changing one base URL: Helicone.
  • Your main question is whether a prompt or model change made outputs better or worse: Braintrust.
From each decision's guide, written from what makers' products chose and why.

LangSmith is LangChain's hosted platform for watching and testing LLM agents. It records each run of your app as a trace of nested steps (model calls, tools, retrieval) so you can see why an agent did what it did, then turns those runs into datasets and evaluations so you can check that a prompt or model change made things better rather than worse.

More about LangSmith

You instrument code with its SDKs for Python, TypeScript, Go or Java, typically by wrapping your LLM client or marking functions as traceable; LangChain and LangGraph apps switch it on with an environment variable. It also accepts OpenTelemetry traces on an OTLP endpoint, and the Vercel AI SDK has a dedicated integration. In JavaScript, runs are sent in the background by default so tracing doesn't add latency.

For a small team the useful parts are trace search with dashboards and alerts, online evaluators (LLM-as-a-judge) that score production traffic, offline experiments over datasets, and a prompt playground. The same account can also host agent deployments. The managed cloud runs in US, EU and APAC regions on GCP and a US region on AWS, and you pick the region at sign-up.

Traces are kept 14 days at base retention, with longer retention billed separately. Self-hosting is only available as an Enterprise add-on, and an organization can't move between regions after sign-up.

How LangSmith itself is built

1 tools, from its own code, website and Product Hunt page.

LangSmith pros and cons

What makers like most: Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. Their most common complaint: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in.

Pros

  • Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. PH
  • Traces, datasets, annotation queues and evals live in one platform widely used by enterprises. PHHN
  • Pairs tightly with LangChain and LangGraph for debugging stateful agent workflows. PHHNHN 2
Product Hunt review tags
monitoring AI model performance3
chain sequence debugging2
evals2

Cons and watch-outs

  • It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. HNHN 2HN 3HN 4
  • Traces show which agent failed but not why, so root-cause analysis stays manual. HNHN 2HN 3HN 4
  • Viewing your own traces requires a cloud account, with no local-first option. HNHN 2
  • The prompt engineering workflow is laborious, gets expensive fast and handles multi-turn prompts poorly. HNHN 2

Themes that recur in makers' words and Hacker News comments, each linked to what it summarises, with how Product Hunt tags its reviews.

4.8★ from 19 reviews on Product Hunt ↗

Reliability and open issues

LangSmith's status page posted no incidents in the last 90 days.

Incidents on its status page

0 in the last 90 days · 0 in the last yearFrom its own status page, 2026-10-19. Vendors decide what they post; one page can cover several products.

Who switches

More projects move away from LangSmith: 0 pull requests in, 9 out; the most leave for Langfuse.

Moving away
LangSmith

Public pull requests on GitHub since Oct 2024 whose title says "X to Y" — real code changes moving a project from one tool to another, by developers in general.

Who uses it

11 makers' products, each linked to the source that shows it, and 62 open-source projects that declare it in their code. The best known on Product Hunt: Watchman AI (874 followers), GitLaw (868 followers), Meet-Ting (803 followers), DryMerge (688 followers), NeuralAgent (638 followers).

The maker says so 10Subprocessor list 1Declared in code 62How evidence is collected →
GitLaw
GitLawAgent + templates + workflow. Making legal documents free.

“Evals / tracing / monitoring is awesome and out of the box”

LLM Observability & EvalsMaker says so · source ↗
Astrid
AstridThe personal stylist programmed just for you

“We absolutely could not build this product with LangSmith for traces, datasets, annotation queues, and evals. We've become big power users! I use to build AI apps before nice tracing tools like this existed and it's like night and day having a tool like this. We considered Langfuse, but I already had experience with LangSmith so for speed purposes we went with LangSmith.”

LLM Observability & EvalsMaker says so · source ↗
Intryc
IntrycAI scores tickets to your SOPs with 90% precision

“Shoutout for all the observability and prompt management capabilities!”

LLM Observability & EvalsMaker says so · source ↗
DryMerge
DryMergeAutomate work with plain English.

“You can't build AI agents without monitoring. Metadata filtering is strong.”

LLM Observability & EvalsMaker says so · source ↗
Pig
PigAutomate Your Windows Computer with AI

“I deployed the manage Pig agent on LangGraph and it's been smooth sailing!”

LLM Observability & EvalsMaker says so · source ↗
Watchman AI
Watchman AICapturing invisible B2B buyers with AI agents

“LangSmith’s real-time analytics and versioning keep our AI agents rock-solid -- so everything just works better.”

LLM Observability & EvalsMaker says so · source ↗
lmChatGPTtfy
lmChatGPTtfyThe ChatGPT pendant of Let me Google that for you

“Our preferred way to track and analyze tool usage. Really easy to handle.”

LLM Observability & EvalsMaker says so · source ↗
Promptius AI
Promptius AIBuild production-grade AI agents using natural language

“Promptius <3 Langchain + Langgraph + Langsmith, this combination has been instrumental in building Promptius in the past 5 months! The abstractions provided by Langchain and Langgraph make building agents as easy as writing a prose. Langsmith provides unmatched observability and also help track our costs.”

LLM Observability & EvalsMaker says so · source ↗
Meet-Ting
Meet-TingAI That Gives Your Schedule a Brain | Availability Agent

“We started building Ting without LangSmith and we were flying blind. Diagnosing issues downstream felt like guesswork. Since bringing LangSmith in, we’ve been able to trace problems through the entire prompt path and actually understand what’s going on. We’re still evolving how we use it, but even early on it’s helped us move faster and get closer to that “this feels good” moment.”

LLM Observability & EvalsMaker says so · source ↗
Aix-DB
Aix-DBAix-DB 基于 LangChain/LangGraph 框架,结合 MCP Skills 多智能体协作架构,实现自然语言到数据洞察的端到端转换。LLM Observability & EvalsIn its code · source ↗
Dify
DifyPlatform for building LLM apps, agents and RAG workflows in a visual editor, hosted as D…LLM Observability & EvalsIn its code · source ↗
LangBot
LangBotProduction-grade platform for building agentic IM bots - 生产级多平台智能机器人开发平台/ Agent、知识库编排、插件…LLM Observability & EvalsIn its code · source ↗
n8n
n8nWorkflow automation tool with a visual node editor, code steps and AI agent nodes, avail…LLM Observability & EvalsIn its code · source ↗
omi
omiOpen-source AI wearable with mobile and desktop apps that records conversations and meet…LLM Observability & EvalsIn its code · source ↗
Yuxi
Yuxi可私有部署的多租户知识智能体平台:统一 RAG、知识图谱、多智能体、MCP/Skills、沙盒与权限管理。Yuxi = Cloud Agents + Knowledge RAG…LLM Observability & EvalsIn its code · source ↗

Open source: a project that declares LangSmith as a dependency in its public code — verifiable, but not necessarily a live product.

What its projects tell coding agents

15 of the 30 open-source products built with LangSmith that were worked on in the last 180 days keep instruction files for coding agents — 50%, against 53% of all the projects we read. AGENTS.md is in 11, CLAUDE.md in 8.

993words in the median file
uvthe package manager most named, in 6 of 13 files
77%say how to test a change
Named more in these files than in other projects' — in how many of the 13:
What AGENTS.md files say about other tools →What open-source products tell their coding agents →

Alternatives to LangSmith

All alternatives by situation →

Questions makers ask about LangSmith

Is LangSmith good for production?

11 makers' products we track run on it. Its own status page posted 0 incidents in the last 90 days; vendors decide what they post there, so this doesn't compare across tools. On GitHub, 0 public pull requests moved projects to LangSmith from tools we list, and 9 moved them away. The watch-out makers raise most: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in.

What is LangSmith used for?

Observability and evaluation platform for LLM agents, with tracing, monitoring, and dataset-based testing; framework-agnostic but built by the LangChain team. Makers' products we track use it for LLM observability & evals (11).

What are the pros and cons of LangSmith?

Pros makers name: Tracing through the whole prompt path replaces guesswork when diagnosing agent issues. Traces, datasets, annotation queues and evals live in one platform widely used by enterprises. Pairs tightly with LangChain and LangGraph for debugging stateful agent workflows. Cons: It keeps pulling users toward the LangChain platform, and LangChain docs push LangSmith, which feels like lock-in. Traces show which agent failed but not why, so root-cause analysis stays manual. Viewing your own traces requires a cloud account, with no local-first option.

What are the alternatives to LangSmith?

For LLM observability & evals, the other options makers' products we track chose most are Datadog (50), Langfuse (35), Helicone (13), Braintrust (6), Weights & Biases (3). Open source or self-hostable among them: Langfuse, Helicone, Weights & Biases. Each alternative, with the situation it fits, is on the LangSmith alternatives page.

How much does LangSmith cost per month?

At 10,000 users, LangSmith costs $475 a month on the cheapest plan that fits (Developer), from its pricing page.

Do developers switch to or away from LangSmith?

On GitHub, 0 public pull requests moved projects to LangSmith from tools we list; 9 moved them away, most to Langfuse (9).

When should you use LangSmith?

Your app is built on LangChain or LangGraph (LLM observability & evals). Something else fits better if: you want tracing, prompts and evals in one tool you can self-host for free, pick Langfuse; you already use OpenTelemetry or want vendor-neutral instrumentation, pick Arize Phoenix.

Do I have to use LangChain to use it?

No. The SDKs trace any code and are documented as usable without LangChain; LangChain and LangGraph apps only get a shortcut, enabling tracing with a single environment variable. source ↗

Can I keep my data in the EU?

Yes. LangSmith has an EU instance in the Netherlands (GCP europe-west4), plus US and APAC instances, available on every plan including the free one. Billing data stays in the US, and you can't switch an organization between regions later. source ↗

Is it SOC 2 and HIPAA compliant?

LangChain states LangSmith is SOC 2 Type 2 certified, HIPAA compliant and GDPR compliant, and offers a DPA through support. It has no EU legal entity for contracting. source ↗

Can I send OpenTelemetry traces instead of using its SDK?

Yes. LangSmith accepts OTLP traces at its /otel endpoint, and you can use an OpenTelemetry Collector to send the same spans to LangSmith and another backend such as Datadog or Grafana. source ↗

Will tracing slow down my serverless functions?

The JS SDK sends traces in the background by default. In serverless runtimes you either turn background tracing off or await pending trace batches before the function returns, so data isn't lost when the runtime freezes. source ↗

How long are traces kept?

Base traces are kept 14 days and extended traces 180 days. Some actions, like online evaluators and automation rules, can upgrade a trace to extended retention; after retention ends, trace inputs and outputs are deleted. source ↗

Can I self-host it?

Only as an add-on to the Enterprise plan, installed on Kubernetes with a license key from LangChain's sales team. source ↗

Is LangSmith free?

Yes — there is a free tier a small product can run on; paid use starts at $39/seat/mo. source ↗

Is LangSmith open source or self-hostable?

Not open source, and hosted only.

Can AI coding agents work with LangSmith?

It serves an llms.txt docs index.

Who uses LangSmith?

11 makers' products we track, each with a source, and 62 open-source projects declare it in their code. The best known of them on Product Hunt: Watchman AI (874 followers), GitLaw (868 followers), Meet-Ting (803 followers), DryMerge (688 followers), NeuralAgent (638 followers). source ↗