OpenLLMetry: Open-Source Observability for LLM Applications Built on OpenTelemetry
Traceloop Team April 9, 2026 product-announcement medium credibility
View source
Referenced in catalog
OpenLLMetry: Open-Source Observability for LLM Applications Built on OpenTelemetry
Source: GitHub — traceloop/openllmetry | Author: Traceloop Team | Published: 2023-09-01 (repository, ongoing) Category: product-announcement | Credibility: medium
Executive Summary
- OpenLLMetry is an Apache-2.0 library that wraps OpenTelemetry’s Python SDK with LLM-specific instrumentation for 16+ providers (OpenAI, Anthropic, Bedrock, Gemini, etc.), 7 vector databases, and 10 frameworks (LangChain, LlamaIndex, CrewAI, LangGraph), sending traces to any OTel-compatible backend
- The project has 7,000+ GitHub stars and 252 releases; its semantic conventions were contributed back upstream and are now part of the official OpenTelemetry GenAI SIG specs, giving it standards-body legitimacy beyond just Traceloop’s marketing
- Traceloop (the maintainer) was acquired by ServiceNow in March 2026 for an estimated $60–80M, with a stated commitment to keep OpenLLMetry open-source — but this introduces genuine governance uncertainty for teams building on the library
Critical Analysis
Claim: “Because it uses OpenTelemetry, it can be connected to your existing observability solutions — Datadog, Honeycomb, Grafana, New Relic, and others”
- Evidence quality: benchmark
- Assessment: This claim is technically accurate and independently verifiable. OpenTelemetry’s OTLP protocol is now natively supported by Datadog (with GenAI semantic conventions), Honeycomb, Grafana, and most major APM vendors. The value proposition is real: teams that have already standardized on one of those platforms avoid deploying a separate LLM observability silo.
- Counter-argument: The integration advantage is narrower than the marketing implies. OTel-native does not mean zero-configuration — teams still need to initialize the SDK, configure exporters, and map OpenLLMetry’s span attributes to their backend’s dashboarding conventions. The README’s “two lines of code” claim applies only to the Traceloop SaaS backend, not to arbitrary OTLP destinations. Independent reviews (ZenML, Firecrawl) consistently note that Langfuse and Arize Phoenix provide richer native UI for agent debugging.
- References:
Claim: “OpenLLMetry’s semantic conventions are now part of the OpenTelemetry specification”
- Evidence quality: peer-reviewed (standards body contribution)
- Assessment: Verified. The OpenTelemetry GenAI Semantic Conventions SIG was co-driven by Traceloop and the OpenLLMetry project; the resulting
gen_ai.*attribute namespace (coveringgen_ai.system,gen_ai.request.model,gen_ai.usage.input_tokens, etc.) is now part of the official OTel spec. This means using OpenLLMetry today aligns with where the ecosystem is converging, reducing lock-in risk. - Counter-argument: “Part of the OTel spec” understates how experimental this area still is. The GenAI conventions are marked as “experimental” in the OpenTelemetry specification, meaning they can change between minor releases. Teams who instrument deeply against
gen_ai.*attributes today may need migration work as the spec stabilizes through 2026–2027. - References:
Claim: “We no longer log or collect any telemetry in the SDK or in the instrumentations” (since v0.49.2)
- Evidence quality: vendor-sponsored
- Assessment: This is a self-reported privacy commitment with no independent verification. The change was made in v0.49.2 after earlier versions silently phoned home, which caused community pushback. The explicit changelog note and version requirement suggest the prior behavior was a real issue, not a hypothetical one.
- Counter-argument: The library’s instrumentation by design captures LLM prompts and completions as trace attributes. Teams operating in regulated environments (HIPAA, GDPR, financial services) must audit whether prompt payloads containing PII flow through their OTel pipeline and into the chosen backend. OpenLLMetry does not currently provide a built-in prompt sanitization or redaction layer — that is delegated to the OTel Collector or the backend.
- References:
Claim: OpenLLMetry is suitable for complex multi-step agent tracing
- Evidence quality: anecdotal
- Assessment: Multiple independent tool comparison reviews (ZenML, Firecrawl, PostHog) identify this as OpenLLMetry’s weakest point relative to peers. LangSmith and Langfuse provide first-class UI for inspecting chains of tool calls, agent decision trees, and memory reads across a session. OpenLLMetry emits correct OTel spans for each step, but the visualization and correlation experience depends entirely on the downstream backend, which may not have LLM-agent-native views.
- Counter-argument: This is not an architectural limitation — it is a tooling gap. Teams using Honeycomb or Grafana with custom dashboards can approximate agent trace visualization. The OTel GenAI SIG’s 2025 work on agent spans (
gen_ai.agent.*conventions) is specifically designed to close this gap, and OpenLLMetry is tracking those conventions. - References:
Credibility Assessment
- Author background: Traceloop is an Israeli AI observability startup (YC W23 cohort), founded by Nir Gazit and Ran Isenberg. The project is a marketing-backed open-source library designed to funnel users toward Traceloop’s commercial platform; this creates promotional bias in the README.
- Publication bias: Vendor blog / GitHub README — primary source is self-promotional. However, the technical claims about OTel integration are independently verifiable and confirmed by third parties.
- Verdict: medium — The core technical approach is sound and independently validated; the marketing framing (particularly “two lines of code” and “complete observability”) overstates ease of integration. The ServiceNow acquisition in March 2026 adds governance risk that must be weighed.
Entities Extracted
| Entity | Type | Catalog Entry |
|---|---|---|
| OpenLLMetry | open-source | openllmetry |
| Traceloop | vendor | traceloop |