Agent observability: see what your AI agent is actually doing

Agent observability is seeing every LLM call, tool call, retry and token of a run, after the fact and while it happens. Classic APM logs a deterministic path. An agent is not deterministic. Same prompt, different graph. Without traces you are staring at the last sentence and guessing.

What you are actually looking at

• Traces: the full path of one request, nested like dolls: model, tool, sub-agent, retry.
• Metrics: tokens, latency per step, cost per request, error rate.
• Evals: did this run do the job, scored by a judge, a script, or a human.

The pipe underneath the brand names is increasingly OpenTelemetry. Langfuse and Phoenix speak it natively. LangSmith can ingest it. If you instrument once against OTel, you can move the backend later.

The field in September 2026

Langfuse: default if you want to own the data

Open source LLM engineering platform, MIT, self-host in Docker. Tracing, prompt versions, datasets, playground, evals. About 34,000 GitHub stars on langfuse/langfuse. Since January 2026 the company sits inside ClickHouse. The README still commits to MIT and self-hosting. That is the trade: maturity plus a database vendor as owner. Ask where the traces land.

Source: github.com/langfuse/langfuse.

LangSmith: deepest if you already live in LangChain

Built by the LangChain team. Waterfall traces, token and latency dashboards, prompt hub. LangGraph Studio sits next to it as the visual debugger: graph lights up, pause, rewind, replay a production trace locally. Public docs in 2026 still treat LangSmith as a cloud product, not a box you run yourself. Fine if prompts may leave the building. Not fine if they may not.

Source: docs.smith.langchain.com.

Arize Phoenix: OpenTelemetry on your laptop

Tracing plus evals, notebook-native, then the same spans in production. GitHub lists about 11,000 stars on arize-ai/phoenix. License on the repo: Elastic-2.0, source-available, not MIT. Self-host is the point. Managed sister product is Arize AX.

Source: github.com/Arize-ai/phoenix.

AgentOps, Helicone, Weave, Opik

AgentOps treats the agent session as the object, not the single LLM call. Helicone sits as a proxy and counts. Weights & Biases Weave is the trace layer if you already pay W&B. Comet Opik is the other open eval-and-trace bench. None of them replace a trace store. They are how some teams enter the same problem.

Three questions, then you pick

1. May prompts leave your machines? No: Langfuse or Phoenix, self-hosted. Yes: LangSmith is on the table.
2. Are you married to LangGraph? Yes: LangSmith plus Studio. No: framework-agnostic first.
3. Is this a debugger for Tuesday or a monitor for next year? Studio wins the afternoon. Langfuse and LangSmith win the month.

Most teams end up with two layers: one for stepping through a graph, one for production traces. That is not indecision. The jobs are different.

Fact table

Claim	Basket	From where
Langfuse ~34,000 GitHub stars, MIT, self-host, part of ClickHouse since January 2026	Fact	github.com/langfuse/langfuse README, September 2026. Acquisition stated there by the project itself.
Phoenix ~11,000 stars, Elastic-2.0, self-host, OpenTelemetry/OpenInference	Fact	github.com/Arize-ai/phoenix, September 2026.
LangSmith is cloud, not a self-hosted box	Fact as of public docs, September 2026	LangSmith docs. If that changes, the docs change first.
Agent runs are non-deterministic, so traces are not optional in production	Thesis	This page. Same input, different path is the whole point of an agent.
OpenTelemetry is the interchange format that will outlive the current brand names	Hypothesis	Langfuse and Phoenix native OTel, LangSmith ingestion. Direction, not a standard vote.

Cut-off: 9 September 2026. Langfuse is no longer an independent company. The traces still self-host under MIT, if you run the box yourself.