Runpod

RunPod democratizes GPU access for AI/ML teams: rent a GPU pod (persistent Docker container on a GPU) for interactive development, or deploy a Serverless endpoint (custom Docker image invoked via REST, billed per execution) for production inference. For agents: the REST Serverless API enables invoking ML inference jobs — send input, get output — without managing pod lifecycle. The GraphQL API manages pod lifecycle. Competitive pricing vs. AWS/Azure/GCP GPU instances, especially for open-source model inference (Llama, Stable Diffusion, Whisper). Confidence is docs-derived.

Two surfaces: (1) GraphQL API at api.runpod.io/graphql: mutations to create, start, stop, terminate pods; queries to list pod status. (2) Serverless REST: POST https://api.runpod.ai/v2/{endpoint_id}/run submits a job (async) or /runsync for synchronous execution. Job response includes id for async polling. GET /v2/{endpoint_id}/status/{job_id} polls job status. GET /v2/{endpoint_id}/output/{job_id} retrieves completed output. Serverless input: JSON body with input field matching handler schema.

API key auth: Authorization: Bearer {api_key} header (Serverless REST) or api_key in query param (GraphQL). Keys from RunPod Account Settings → API Keys. Account-scoped. HTTPS enforced. No OAuth2. Spending limits configurable per-account to prevent runaway costs. Spot pods can be interrupted when host reclaims GPU; persistent pods avoid this at higher cost.

Serverless: job status transitions: IN_QUEUE → IN_PROGRESS → COMPLETED/FAILED. Timeout per job configurable in endpoint settings. Failed jobs return error message in output. GPU availability varies by type and time — common 503 when no capacity available. Spot interruptions logged; handle via checkpointing for long-running jobs. RunPod community cloud uptime depends on individual host availability; Secure cloud is more stable. Status at runpod.io/status.

docs.runpod.io covers pod management, Serverless endpoint development (handler function pattern, Docker image requirements), GraphQL API reference, and network volume management. Getting started: create account, add credits, launch first pod from UI in under 5 minutes. Serverless worker templates (Python, Node.js) on GitHub for HuggingFace Transformers, diffusers, Whisper. Community via RunPod Discord (active ML community). No official language SDK; API is clean enough for direct use.