Argo Workflows
Kubernetes-native workflow engine that runs each step of a pipeline as a container, orchestrating parallel jobs and multi-step DAGs directly on a cluster.
Overview
Argo Workflows is an open-source, CNCF graduated workflow engine for orchestrating jobs on Kubernetes, implemented entirely as Kubernetes Custom Resource Definitions rather than a separate system bolted on top of a cluster. A workflow is defined as a sequence of steps or a directed acyclic graph (DAG) of tasks, where each step runs as its own container - so anything that can run in a container, from a training job to a data transform, becomes a workflow step without extra glue code.
It's commonly used for machine learning pipelines, batch and data processing, infrastructure automation, and CI/CD, taking advantage of Kubernetes' own scheduling and scaling to run many parallel steps across a cluster rather than a single worker.
Argo Workflows is a fit for teams already running Kubernetes who want to orchestrate container-based pipelines using the cluster they already operate, a different niche than Temporal (also in this catalog), whose workflows are general-purpose functions in a language SDK rather than Kubernetes-native container steps.
- Categories
- Automation
- Languages
- Go
- License
- Apache-2.0
Related Projects
Prefect
Apache-2.0Python-native workflow orchestration platform for building, scheduling, and monitoring data pipelines with ordinary Python code.
- Automation
Argo Events
Apache-2.0Event-driven automation framework for Kubernetes that triggers workflows, functions, or other actions in response to events from any source.
- Automation
Kestra
Apache-2.0Declarative, YAML-based orchestration platform for scheduling and orchestrating data pipelines, ETL jobs, and business workflows.
- Automation
Temporal
MITDurable execution platform that lets developers write reliable, long-running workflows as ordinary code, with retries, state, and failure recovery handled automatically.
- Automation
Spotted an error? Suggest an edit on GitHub.