Dagster

Open-source data orchestrator that models pipelines as the data assets they produce, with schedules, sensors and lineage, plus a hosted Dagster+ service.

Visit dagster.io ↗Alternatives

Works with AI agents:llms.txt

Ask your AI about this, with this page as the source:ChatGPT ↗Claude ↗Perplexity ↗

Dagster orchestrates data pipelines: the jobs that pull data in, transform it and produce tables, files and machine learning models on a schedule. Where most orchestrators start from tasks, Dagster starts from assets, the logical pieces of data a pipeline creates, so you can see what each table depends on and when it was last built.

You define assets in Python (3.10 or later), along with schedules that run them on an interval, sensors that trigger them on events such as a file upload, and resources for database and API connections. dg dev starts a local UI at localhost:3000. Code in other languages or on other platforms, such as Spark, Databricks or Kubernetes jobs, can be run and reported back through Dagster Pipes, as long as it can be triggered from Python.

What you get: lineage across assets, a UI for runs and logs, and native integrations with dbt, Snowflake and Fivetran. The open-source version is self-hosted as three services, a webserver, a daemon that runs schedules, sensors and the run queue, and one or more code location servers, with SQLite storage by default that can be swapped for Postgres; Docker and Kubernetes deployments are documented. Dagster+ is the hosted control plane, either Serverless, where your code runs in Dagster's environment, or Hybrid, where it runs on your infrastructure.

The limits: definitions are Python-only. Dagster+ has no free plan, only a 30-day trial, and is billed per credit, where each asset materialization and each op execution counts as one. Self-hosting means running and upgrading the webserver, daemon and database yourself.

Where it fits

Background Jobs & Cron · Data pipeline orchestratorData pipelines described by the tables, files and models they produce, with lineage between them, schedules, sensors and native dbt integration, in Python.Trade-off: Definitions are Python only; Dagster+ has a 30-day trial but no free plan and bills per asset materialization and op, while self-hosting means running a webserver, a daemon and a database.

How Dagster itself is built

2 tools, from its own code, website and Product Hunt page.

Next.js

Vercel

Who uses it

3 makers' products, each linked to the source that shows it, and 1 open-source projects that declare it in their code.

Job posting 3Declared in code 1How evidence is collected →

Sort

PostHog Product analytics with session replay, feature flags, A/B tests and surveys in one open-…Background Jobs & CronIn its code · source ↗

RelayOnline business banking with up to 20 checking accounts, cards, expense management and A…Background Jobs & CronJob posting · source ↗

BraintrustObservability and evaluation platform for AI agents, letting teams trace requests, score…Background Jobs & CronJob posting · source ↗

LegoraLegora is the collaborative AI powering lawyers to review and research faster, draft sma…Background Jobs & CronJob posting · source ↗

Open source: a project that declares Dagster as a dependency in its public code — verifiable, but not necessarily a live product.

Alternatives to Dagster

All alternatives by situation →

Trigger.devLong-running TypeScript tasks — AI calls, media processing — that would time out in a serverless function.Open sourceFree tier23 makers · background jobs & cron

InngestEvent-driven, multi-step workflows where each step retries on its own and your code stays in your existing app.Open sourceFree tier16 makers · background jobs & cron

TemporalLong-running, multi-step workflows — payments, onboarding, AI agent loops — whose state must survive crashes and deploys, in Go, Java, Python, TypeScript and more.Open sourceFree tier35 makers · background jobs & cron

UpstashSimple scheduled calls and delayed HTTP jobs on serverless or edge hosting, via QStash.Free tier2 makers · background jobs & cron

BullMQNode apps on a long-running server that already have Redis — free and fully in your control.Open sourceFree tier1 makers · background jobs & cron

RabbitMQA dedicated message broker for queues and pub/sub between services, in any language, or as the broker behind Celery.Open sourceFree tier3 makers · background jobs & cron

PrefectTurning existing Python scripts into scheduled, retried, monitored workflows with decorators, using ordinary Python control flow instead of a fixed DAG.Open sourceFree tier1 makers · background jobs & cron

Questions makers ask about Dagster

What is an asset, and how is it different from a task?

An asset is a logical unit of data such as a table, dataset or machine learning model, while an op is a unit of computation. Jobs execute a selection of assets or a graph of ops, and schedules and sensors start them on an interval or when an event happens. source ↗

Can I self-host it, and what does that involve?

Yes. An open-source deployment runs a webserver for the UI, a single daemon for schedules, sensors and run queuing, and code location servers for your definitions. Run storage defaults to SQLite and can be switched to Postgres. source ↗

How much does Dagster+ cost?

Solo is $10 a month plus $0.040 per credit for one user, Starter is $100 a month plus $0.035 per credit for up to three users, and Pro is priced by sales. A credit is one asset materialization or one op execution, Serverless compute adds $0.010 per minute, and both paid self-serve plans start with a 30-day free trial. source ↗

Where does my code run on Dagster+?

On Serverless it runs in Dagster's environment. On Hybrid you run an execution environment that connects to the Dagster+ control plane, so code runs on your infrastructure, and Hybrid has no compute charge. source ↗

Do I have to write everything in Python?

Definitions are Python, but Dagster Pipes can invoke external code, including other languages on remote machines, and that code can send information back to Dagster. It does not need to know about Dagster, but it must be triggerable from Python. source ↗

Is Dagster open source?

Yes. The core project in the dagster-io/dagster repository is licensed under Apache 2.0; Dagster+ is the separate commercial hosted service. source ↗

Is Dagster free?

Yes — there is a free tier a small product can run on; paid use starts at $10/mo. source ↗

Is Dagster open source or self-hostable?

Open source, and you can self-host it. source ↗

Can AI coding agents work with Dagster?

It serves an llms.txt docs index.

Who uses Dagster?

3 makers' products we track, each with a source, and 1 open-source projects declare it in their code. source ↗