Vast Ai

Vast.ai aggregates spare GPU capacity from individual hosts and datacenter operators, offering GPU compute at 3–5x lower cost than AWS/Azure/GCP. The tradeoff: instances run on third-party hardware with variable reliability and a decentralized trust model (vs. cloud provider SLA). For cost-sensitive ML workloads (training runs, batch inference, experimental pipelines), the price advantage is substantial. For agents: the REST API enables searching available GPU offers by GPU type, VRAM, price, and reliability score — then launching an instance, monitoring it, and terminating it when done. Docker-based workloads; SSH access to running instances. Interruptible instances lower cost further but may be reclaimed by the host. Confidence is docs-derived.

REST API at console.vast.ai/api/v0. Resources: bundles (offers), instances, volumes, ssh-keys. GET /v0/bundles searches available GPU offers (query params: gpu_name, num_gpus, min_gpu_ram, max_dph_total, reliability2, rentable=true). Response: offers array with per-offer pricing, specs, and availability. POST /v0/asks/{offer_id}/ launches an instance from an offer (body: {image, disk, label, onstart, env, extra, ...}). GET /v0/instances retrieves running instances (filtered by owner). PUT /v0/instances/{id}/destroy terminates an instance. SSH key management: PUT /v0/users/current adds SSH public key for instance access.

API key auth: Authorization: Bearer {api_key} header. Keys from Vast.ai Account Settings → API Keys. Account-scoped. HTTPS enforced. No OAuth2. Billing: credit-based; add credits before launching instances (automatic deduction per GPU-hour). Security caveat: instances run on third-party hosts — Vast.ai vets hosts (reliability score) but does not provide cloud-provider-level hardware isolation. Treat Vast.ai instances as untrusted compute for sensitive workloads.

Instances: launched → running → destroyed lifecycle. Host reliability score (0–1) predicts uptime quality; filter offers by reliability2 >= 0.9 for stable hosts. Interruptible instances: can be terminated by host — implement checkpointing for long runs. SSH access: connect via public IP + port (instance exposes SSH port). Instance health: monitor via GET /v0/instances; check status field. Vast.ai platform uptime at status.vast.ai. Credit exhaustion: instance suspended, not destroyed — add credits to resume. API rate limiting: reasonable limits for standard offer search and instance management.

vast.ai/docs covers the REST API reference, CLI (vastai) command reference, instance launch guide, and Docker image configuration. Getting started: create account, add $5 credits, install vastai CLI (pip install vastai), search offers and launch first instance in under 10 minutes. Python SDK: vastai client library wraps the REST API. Community via Vast.ai Discord and community forums. Documentation is adequate for technical users; less polished than cloud-provider docs. CLI is the most ergonomic interface for interactive workflows.