Celery

Celery is the de facto distributed task queue for Python, processing async tasks and cron-scheduled jobs via Redis, RabbitMQ, or SQS message brokers. For agents, Celery is typically a dependency within Python services rather than an externally-called API. However, Flower (the Celery monitoring tool) exposes a REST API for querying task state, worker status, and triggering tasks. Agents building or monitoring Python async pipelines will encounter Celery; direct agent-to-Celery interaction is via Flower or by publishing messages to the broker. Open-source, battle-tested, widely deployed. Confidence is docs-derived.

Celery itself has no auth — security is at the broker layer (Redis AUTH, RabbitMQ user/vhost, SQS IAM). Flower supports HTTP basic auth and optional SSL. The Flower REST API inherits Flower's auth configuration. Production Flower deployments should be on private networks or behind a reverse proxy with authentication. No API key model for Celery native; Flower basic auth is the agent-accessible pattern.

Documentation at docs.celeryq.dev is comprehensive for the Python SDK: task definition, routing, retries, scheduling, and configuration. Flower documentation is separate (flower.readthedocs.io) and covers the REST API and monitoring UI. Getting started guides assume Python familiarity. Community support via GitHub Issues and Stack Overflow. Documentation quality is high for experienced Python developers; the Flower REST API is adequately but not extensively documented.

Celery's primary interface is the Python SDK: task definition with @app.task decorator, and task invocation with .delay() or .apply_async(). No REST API natively. Flower adds a REST API: GET /api/tasks (list tasks), GET /api/workers (worker status), POST /api/task/apply/{name} (trigger task), GET /api/task/result/{task-id} (poll result). The Flower API is optional and must be deployed separately. Celery configuration is Python-based; broker connection strings are environment variables. Retry policies, rate limits, and routing are configurable per task.

Celery tasks return PENDING, STARTED, SUCCESS, FAILURE, RETRY, or REVOKED states accessible via Flower or the Celery result backend. Failed tasks store the exception and traceback in the result backend. Retry logic is configurable per task with exponential backoff. Flower surfaces task failure details via the REST API. Broker connection failures cause workers to retry with backoff. Dead letter queues are broker-dependent. Error visibility is good for Python developers; less polished for cross-language agent consumption.