Grafana Tempo

Default: no authentication (network-isolate in production). Basic auth: configure via reverse proxy. Grafana Cloud Tempo: Bearer token via Grafana Cloud API key. HTTPS enforced on Grafana Cloud. Self-hosted: operator-managed security. OpenTelemetry SDK: configure exporter endpoint (OTEL_EXPORTER_OTLP_ENDPOINT=http://{tempo}:4318) — no auth by default for self-hosted. Multi-tenancy: X-Scope-OrgID header for tenant isolation.

Grafana Tempo is the cost-efficient tracing backend in the Grafana observability stack — stores traces in object storage (S3/GCS/Azure) with no search index, making it significantly cheaper than Jaeger or Zipkin at scale. TraceQL is a powerful query language for finding traces by span attributes, duration, and status. Pairs natively with Grafana (trace visualization, exemplar links from Prometheus metrics → traces). For agents: ingest traces via OTLP (OpenTelemetry Protocol), query traces by ID or TraceQL expression, and correlate with Loki logs via trace ID. Self-hostable (single binary) or Grafana Cloud (50GB/month free). Confidence is docs-derived.

HTTP API at {tempo_host}:3100. Trace ingestion: POST /otlp/v1/traces (OTLP HTTP). Also accepts Jaeger (14268), Zipkin (9411), and OpenCensus protocols via separate ports. Trace query: GET /api/traces/{traceID} retrieves a specific trace. TraceQL search: GET /api/search?q={traceql_expr}&start={}&end={}&limit=100. TraceQL examples: { span.http.status_code = 500 } finds error spans; { duration > 1s } finds slow traces; { resource.service.name = "api" && status = error } finds errors in service. GET /api/tags retrieves indexed tag names. GET /api/tag/{name}/values retrieves tag values. Metrics generator: Tempo can generate service graph metrics (RED metrics from traces) written to Prometheus.

Standard HTTP status codes. TraceQL errors: 400 with syntax detail. Trace availability: traces queryable after ~10-30 seconds ingestion lag. Object storage backend: reliability depends on S3/GCS/Azure uptime. Trace retention: configurable (default 336 hours / 14 days). TraceQL performance: very fast for simple queries (trace ID lookup); complex structural queries slower. Grafana Cloud uptime at status.grafana.com. Self-hosted: operator-managed. Tempo single binary: simple to operate; cluster mode for production scale. Search index: Tempo maintains a minimal bloom filter index — no full attribute indexing (unlike Jaeger) — search is sampling-based.

grafana.com/docs/tempo covers TraceQL reference, trace ingestion guide (OTLP, Jaeger, Zipkin), self-hosting configuration, object storage setup, and Grafana integration. Getting started: Grafana Cloud free account, configure Tempo datasource in Grafana, instrument app with OpenTelemetry — first traces in under 10 minutes. OpenTelemetry auto-instrumentation for zero-code traces. Community via Grafana Community and GitHub (4k+ stars). Excellent Grafana integration documentation.