Vellum Ai

LLM application development and operations platform for teams building production AI features. Prompt versioning with deployment environments (sandbox/staging/production). Visual Workflow builder for multi-step chains, conditional branching, loops, API calls, and inline code execution. A/B testing across providers and models. Evaluation test suites with custom metrics. Trace monitoring for production debugging. Confidence is docs-derived.

REST/SDK API: execute_prompt(deployment_name, inputs) for prompt execution; execute_workflow(deployment_name, inputs) for workflow execution; streaming variants for token-by-token output; Python SDK (vellum-ai) and TypeScript SDK; Vellum API key as Authorization header; Webhook events for async workflow completion; REST API for managing deployments, documents, and evaluation runs programmatically.

Vellum API key per workspace (scoped to organization); environments (sandbox/staging/production) for deployment isolation; role-based access control (admin, editor, viewer) within organization; prompt and workflow content stored in Vellum — no customer prompt content used for training; audit log for deployment changes; documents uploaded to Vellum vector store are organization-scoped.

execute_prompt returns structured output with generation, tokens used, and finish reason; workflow execution is synchronous for short chains or async with webhook for long-running workflows; evaluation runs batch-execute against test cases and return metric scores; streaming uses SSE for incremental token delivery; error responses include error_type and message for programmatic handling; rate limits per plan.

Documentation covers prompt editor and versioning guide, Workflow builder reference (node types, branching, code execution), evaluation test suite setup, Python and TypeScript SDK quickstart, deployment environment guide, and monitoring/trace search. Designed for ML engineers and product teams iterating on production LLM features. Confidence is docs-derived.