Dagster

Dagster is a data orchestration platform distinguished by its asset-centric model — rather than defining pipelines as task DAGs, Dagster defines software-defined assets and their dependencies, with the scheduler ensuring assets stay fresh. For agents coordinating data workflows, Dagster's asset model provides richer semantic information than job-run-based orchestrators: agents can query asset freshness, trigger materializations for specific assets, and understand data lineage through the asset dependency graph.

The API covers jobs, runs, assets, asset materializations, and schedules. The asset API is Dagster's distinctive surface — agents can retrieve asset materialization history, check asset freshness status, and trigger targeted materializations. This asset-awareness enables agents to make decisions based on the freshness of specific data assets rather than just whether pipeline runs succeeded.

Authentication uses Dagster Cloud API keys for the managed service. API key management follows the same patterns as other orchestration platforms. The GraphQL API is the primary programmatic interface for Dagster — the REST API wraps key operations, but some advanced Dagster operations require the GraphQL surface.

Reliability for Dagster Cloud is vendor-managed with enterprise SLAs available. Self-hosted Dagster reliability is the team's operational responsibility. Dagster's architecture includes run launchers, code locations, and the Dagster daemon — more moving parts than Prefect for self-hosted deployments, which teams should account for in their operational planning.

Documentation is comprehensive and reflects Dagster's engineering investment. The asset-centric concepts are explained well, which is important because the asset model represents a conceptual shift from traditional task-DAG orchestration. Teams integrating Dagster for agent-driven data pipeline coordination should invest time in the software-defined asset documentation before building integrations.