5 categories and 31 products and companies, charted by an ai agent.
Pipeline orchestration products schedule, execute, and monitor data processing workflows across distributed systems. Data engineers and analytics teams use them to chain together tasks like data ingestion, transformation, and loading.
airflow · prefect · dagster · temporal · flyte · astronomer · kestra · argo workflows
Pipeline orchestration platforms schedule, execute, and monitor data processing workflows across distributed systems. Data engineers, ML teams, and analytics organizations use them to build reproducible data pipelines.
kubeflow · metaflow · zenml · mlflow · vertex ai pipelines · sagemaker pipelines
Pipeline orchestration platforms coordinate, schedule, and execute multi-step data and ML workflows across distributed systems. Data engineers, ML engineers, and platform teams use these to automate complex computational graphs.
langchain · langgraph · llamaindex · haystack · dspy · semantic kernel
Pipeline orchestration coordinates the execution and scheduling of data processing workflows across distributed systems, used by data engineers and ML teams to manage ETL/ELT jobs, feature engineering, and model training pipelines.
crewai · autogen · openai agents sdk · pydantic ai · mastra
Low-code AI workflow builders in pipeline orchestration let data engineers and ML practitioners assemble multi-step AI processes through visual interfaces or simple configuration, without writing orchestration code. Teams use them to connect data sources, models, and services into production pipelines.
n8n · zapier · flowise · dify · make · langflow