Baseten

Each deployed model exposes a REST endpoint: POST /model_versions/{id}/predict (Baseten format) or OpenAI-compatible /v1/completions / /v1/chat/completions for compatible models. Request/response shape is model-defined — agents must know the expected schema for their deployed model. Model lifecycle management API: list deployments, get version status, trigger rollback. Webhook support for async inference results on long-running models.

Baseten provides infrastructure for deploying and serving custom ML models — PyTorch, TensorFlow, Hugging Face, ONNX — via Truss (their open-source model packaging format) and managed GPU infrastructure. For agents, key capabilities: invoking deployed models via REST API (each model gets a unique endpoint), managing model versions, and accessing popular open-source models (Llama, Mistral, Stable Diffusion) that Baseten hosts. OpenAI-compatible inference endpoints are available for LLM models, enabling drop-in replacement workflows. Good fit for agent pipelines that need to call proprietary or fine-tuned ML models without managing GPU infra. Confidence is docs-derived.

API key auth via Authorization: Api-Key {key} header. Keys are account-scoped; no per-model key scoping in the base tier. RBAC available at the workspace level for team access. API key management via Baseten dashboard. HTTPS enforced for all inference and management API calls. Truss deployment via CLI uses the same API key.

docs.baseten.co covers model deployment (Truss guide), inference API, model management, and OpenAI compatibility. Truss documentation is thorough for packaging custom models. Getting started requires a Baseten account and model deployment. API reference covers inference and management endpoints. Tutorials for popular models (Llama 2, Stable Diffusion, Whisper). Community support via Slack.

Inference errors return HTTP status with JSON error body including error type and message. 401 for invalid API key, 422 for input validation errors (model-specific), 503 for model cold-start timeout. Cold start is a real concern for GPU models — first request after scale-to-zero can take 30-120s. Baseten provides warm instance configuration for latency-sensitive workflows. Rate limits per deployment tier. Uptime SLA on enterprise plans.