How we make this list: we compare pricing pages, product documentation and public user reviews. Ratings come from G2, Capterra or Trustpilot, with the date we checked them.
Key takeaways
- Langfuse is the strongest default for teams that want open-source tracing, self-hosting, and prompt management in one place.
- LangSmith fits best when you already build with LangChain or LangGraph and want deep framework integration.
- Helicone and Portkey help teams control multi-provider costs and latency with a proxy or gateway, often with little code change.
- Braintrust, Arize Phoenix, and Opik stand out for evaluation, not just tracing, which matters once quality becomes the main problem.
- Datadog LLM Observability suits teams already on Datadog, since LLM traces sit beside application traces.
Quick comparison
Is your product missing or out of date?
We review this list regularly. Vendors can request a listing, send updated pricing or ask about featured placement.
Starting priceSee website
Free planYes
Langfuse has become the default shortlist pick for teams starting fresh with LLM observability. It traces LLM calls, tool calls, and agent steps, manages prompt versions, and runs evaluations on production traces. Because the core is open source, teams with data residency needs can self-host and keep traces inside their own infrastructure.
Self-hosting is a benefit and a cost. You gain control, but you also own upgrades, storage, and scaling. Cloud hosting removes that burden for smaller teams. Either way, Langfuse covers the core loop well: see what happened, judge quality, and improve the prompt or model for the next release.
What we like- Open-source core with self-hosting
- Tracing, prompts, and evals in one product
- Framework-agnostic integrations
Watch out for- Self-hosting adds operations work
- Enterprise features and scale pricing need a sales conversation
Starting priceSee website
Free planYes
LangSmith is the natural choice for teams already building with LangChain or LangGraph. Tracing works with little extra code, and the debugging views show agent steps, tool calls, and retrievals in a clear sequence. Evaluation tools and datasets help teams test prompts before they ship, not only after problems appear.
Teams that do not use LangChain can still use LangSmith, but the integration advantage fades. Pricing and self-hosting also favor cloud use for most teams. If LangChain is already central to your stack, LangSmith removes friction that other tools cannot fully match.
What we like- Deep LangChain and LangGraph integration
- Strong evaluation and dataset tools
- Useful debugging views for agents
Watch out for- Best value is tied to the LangChain ecosystem
- Self-hosting requires an enterprise plan
Starting priceSee website
Free planYes
Helicone appeals to teams that want visibility fast. By routing LLM requests through a proxy, it logs prompts, completions, costs, and latency across providers without a major rewrite. That view alone solves a common problem: knowing which model, feature, or customer is driving the token bill.
Helicone is less of an evaluation platform than Braintrust or Langfuse, though it covers basic scoring and monitoring. Teams that need deep evals may pair it with another tool. For cost control and quick multi-provider visibility, it is a practical first step.
What we like- Proxy setup with little code change
- Clear cost and latency views across providers
- Useful caching and rate-limit tools
Watch out for- Proxy model adds a hop to requests
- Deeper eval workflows are less developed than in eval-first tools
#4Braintrust
Best for: Prompt evals and experimentation4.3/5 on
G2 (58 reviews, checked Oct 2026)
Visit braintrust.dev
Starting priceSee website
Free planYes
Braintrust is built around a question many teams face after launch: is the new prompt actually better than the old one. It links traces to experiments, runs evaluations, and helps teams compare models and prompts with data, not impressions. That workflow suits product and engineering teams that iterate on prompts every week.
If you only need basic tracing and cost tracking, Braintrust may feel heavier than Helicone or Langfuse. Its value appears when evaluation becomes routine work. Teams with high prompt churn, agent behavior changes, or quality regressions will get the most from it.
What we like- Strong prompt experimentation and eval workflows
- Trace-backed debugging
- Good for teams iterating on prompts
Watch out for- Pricing rises with scale
- Tracing alone is less of a focus than evals and experiments
Starting priceFree
Free planYes
Arize Phoenix gives teams an open-source way to trace LLM apps and inspect retrievals, embeddings, and agent steps. It fits ML teams that already think in terms of drift, embeddings, and model quality, and it works well for RAG debugging, where retrieval quality often explains a bad answer.
Phoenix alone is a tracing and analysis tool, not a complete hosted evaluation suite. Teams that want managed evals, alerts, and collaboration may need the wider Arize platform or another tool beside it. For open-source tracing with strong ML roots, Phoenix is a solid pick.
What we like- Open-source tracing with OpenTelemetry support
- Strong for RAG and embedding analysis
- Useful for ML teams
Watch out for- Full Arize platform pricing is separate
- Setup for production evals takes more work than a hosted eval tool
Starting priceSee website
Free planYes
Portkey combines an AI gateway with observability, which suits teams using several models or providers. Routing, fallbacks, and caching help control cost and reliability, while request logs give a clear view of usage by model, key, and feature. Guardrails add a layer of control for production apps.
A gateway is extra infrastructure, and teams with one model and one provider may not need it. For multi-model apps, agent systems, or platforms that serve many customers, Portkey centralizes controls that would otherwise spread across codebases.
What we like- Gateway with routing, fallbacks, and caching
- Observability across many providers
- Guardrails and cost controls
Watch out for- Gateway adds architecture complexity
- Small single-model apps may not need a gateway
Starting priceSee website
Free planYes
Opik, from Comet, offers open-source tracing, evaluation, and prompt management for LLM and agent apps. It covers the core observability loop, with metrics, datasets, and experiment views that help teams track quality over time. Teams already using Comet for ML experiments will find the integration familiar.
The community is smaller than Langfuse, which can matter for examples and integrations. Opik still covers the main use cases well, and its open-source model suits teams that want control over their data. Check integration support for your framework before you commit.
What we like- Open-source tracing and evaluation
- Prompt playground and datasets
- Ties to Comet ML experiment tracking
Watch out for- Smaller community than Langfuse
- Enterprise deployment needs planning
Starting priceSee website
Free planFree trial
Datadog LLM Observability makes sense when Datadog is already the center of your monitoring. LLM and agent traces appear beside application traces, database queries, and infrastructure metrics, so a slow agent step can be linked to a slow downstream service in one view. That context helps ops teams more than a standalone LLM dashboard.
For prompt experimentation and deep evaluation, Datadog is less specialized than Braintrust or Langfuse. Teams that already pay for Datadog often accept that trade-off for unified monitoring. Teams starting from zero may prefer a dedicated LLM tool first.
What we like- LLM traces beside APM, logs, and infrastructure data
- Strong for root-cause analysis
- Familiar for ops teams
Watch out for- Less specialized for prompt evals than eval-first tools
- Datadog pricing can grow with volume
#9
Weights and Biases Weave
Best for: ML teams adding LLM traces
Visit wandb.com
Starting priceSee website
Free planYes
Weights and Biases Weave extends a familiar ML experiment platform into LLM tracing and evaluation. Teams that already track models, datasets, and experiments in Weights and Biases can add LLM traces without adopting a separate system. That continuity helps research teams and ML-heavy product teams.
For teams with no Weights and Biases history, Weave is another platform to learn, and its LLM features compete with dedicated tools. The fit is strongest when ML experiment tracking is already central to how your team works.
What we like- Ties LLM traces to ML experiment tracking
- Useful evaluations and prompt versioning
- Strong for research-heavy teams
Watch out for- LLM observability is part of a wider ML platform
- Setup favors teams already using Weights and Biases
How we chose
We compared pricing pages, product documentation, and public reviews on G2, Capterra, and Reddit, and checked each tool for tracing depth, evaluation support, cost tracking, self-hosting, and framework fit. We focused on tools with current LLM or agent observability, not general APM alone, and we excluded discontinued products. We did not claim hands-on testing.
Choosing by your main problem
LLM observability covers several problems at once, and the best tool depends on which problem hurts most right now. Start there, rather than comparing feature lists in the abstract.
If cost is the problem
Pick a proxy or gateway tool such as Helicone or Portkey, and tag requests by feature, customer, and model. Cost views by tag reveal waste faster than a total bill, and caching or routing can cut spend without changing prompts.
If quality is the problem
Pick an eval-first tool such as Braintrust, Langfuse, or Opik, and build datasets from real production traces. Run evals before each release, and track scores over time, so a prompt change that looks better in a demo is proven better in data.
If debugging agents is the problem
Pick a tracing tool with clear agent step views, such as Langfuse, LangSmith, or Phoenix, and check tool calls, retrievals, and errors in sequence. Agent failures often come from a bad tool result or a missing context step, not from the model alone.
Frequently asked questions
What is LLM observability?
LLM observability is the practice of tracing, measuring, and evaluating LLM applications in production. It covers prompts, model calls, tool calls, retrievals, latency, token cost, and output quality. Without it, teams see only API logs and cannot explain why an agent gave a bad answer.
Do I need LLM observability for a simple chatbot?
For a small prototype, basic logging may be enough. Once the chatbot serves real users, handles varied inputs, or uses retrieval and tools, observability becomes useful for debugging, cost control, and quality checks. Start with tracing and cost tracking, then add evals as usage grows.
Is open-source LLM observability worth self-hosting?
Self-hosting suits teams with data residency needs, strict privacy rules, or existing infrastructure skills. It gives control over traces and storage, but adds operations work. Cloud hosting suits smaller teams that want faster setup and less maintenance, even if traces leave their network.
How is LLM observability different from regular APM?
Regular APM tracks latency, errors, and infrastructure for deterministic code. LLM observability adds the semantic layer: prompts, retrievals, tool calls, and output quality. Both matter for agent apps, which is why Datadog-style unified views appeal to ops teams, while eval tools appeal to product teams.
Which tool is best for RAG debugging?
Arize Phoenix and Langfuse are strong for RAG, because they show retrieval steps, document chunks, and embedding views beside model traces. That detail helps teams see whether a bad answer came from retrieval, ranking, or generation. Pair tracing with evals that test retrieval quality over time.
Can one tool cover tracing, evals, and costs?
Langfuse comes closest for many teams, with tracing, prompt management, evals, and cost views in one product. Other teams pair a proxy tool for costs with an eval tool for quality. A single tool is simpler, but a pair can be stronger when cost and quality are both critical.
Sources we checked (9)
- https://www.langfuse.com
- https://www.langchain.com
- https://www.helicone.ai
- https://www.braintrust.dev
- https://www.arize.com
- https://www.portkey.ai
- https://www.comet.com
- https://www.datadoghq.com
- https://www.wandb.com
Is your product missing or out of date?
We review this list regularly. Vendors can request a listing, send updated pricing or ask about featured placement.
Shihab Mia · Founder, Rankite
Shihab has 5+ years in B2B and SaaS SEO. He writes Rankite's software buying guides from pricing pages, documentation and public user reviews. More about Shihab.
Langfuse
LangSmith
Helicone
Arize Phoenix
Portkey
Opik
Datadog LLM Observability