Arize

Arize provides production ML observability — log model predictions, monitor for data drift, evaluate embedding clusters, and trace LLM application behavior in production. For agents: use the Phoenix SDK (open-source, local) for development-time LLM tracing, or Arize Cloud for production monitoring. Phoenix instruments OpenAI, LangChain, LlamaIndex, and DSPy automatically. Arize Cloud adds drift detection, eval scheduling, and alerting for production ML models. Two products: Phoenix (open-source local dashboard) + Arize (hosted cloud platform). Confidence is docs-derived.

Phoenix (open-source, local): pip install arize-phoenix. Launch: import phoenix as px; session = px.launch_app(). Auto-instrumentation: from opentelemetry.instrumentation.openai import OpenAIInstrumentor; OpenAIInstrumentor().instrument(). LangChain: from phoenix.trace.langchain import OpenInferenceTracer; tracer = OpenInferenceTracer(); chain.invoke({"query": "..."}, config={"callbacks": [tracer]}). Arize Cloud Python SDK: pip install arize. Log predictions: from arize.api import Client; arize_client = Client(space_key=..., api_key=...); arize_client.log(model_id="my-model", model_type=ModelTypes.SCORE_CATEGORICAL, prediction_id="1", features={"feature1": 0.5}, prediction_label="positive"). LLM tracing (Arize Cloud): OTel-compatible — set ARIZE_SPACE_KEY + ARIZE_API_KEY env vars + use Phoenix OTel exporter pointed at Arize endpoint.

Arize Cloud auth: Space Key + API Key from app.arize.com → Settings. HTTPS enforced. Phoenix (local): no auth — localhost only. OTel export to Arize: credentials via environment variables. No OAuth2. Role-based access in Arize Cloud: admin, member roles. Data: Arize Cloud stores prediction logs and traces — review data retention policy. Phoenix: all data local.

Phoenix local: in-process — no network calls; always available. Arize Cloud: async log ingestion — predictions available in dashboard within minutes. SDK retries on transient network errors. Large prediction volumes: Arize handles high-throughput logging via batched SDK calls. Drift detection: Arize compares production distributions to training baseline — requires uploading training data separately. Eval scheduling: Arize Cloud supports scheduled LLM evaluations (faithfulness, relevance, toxicity) using LLM-as-judge. Phoenix uptime: local (self-managed); Arize Cloud: status.arize.com.

docs.arize.com covers Phoenix quickstart, Arize Cloud SDK reference, LLM tracing guide, drift detection setup, and eval configuration. Getting started (Phoenix): pip install arize-phoenix, px.launch_app() — local observability dashboard in 2 minutes. Arize Cloud: free tier (limited predictions/month). GitHub: Arize-ai/phoenix (4k+ stars). Community via Arize Community Slack (active). Good documentation; Phoenix particularly well-documented for local dev use.