Every pipeline needs something that decides what runs, when, and what happens when a step fails. In Snowflake that something can be a dynamic table that keeps itself fresh, a single task on a schedule, a task that wakes up when a stream has data, or a whole graph of tasks that runs as one unit.

Picking the wrong one is how you end up babysitting schedules at 2am. Picking the right one usually means writing less code.

This guide is the concepts tour. It explains each primitive, when to reach for it, and how the pieces fit together, with one short snippet per idea. The centre of the guide is the task graph object: a database object that owns a pipeline's schedule, retry policy and member defaults, and lets you edit a running pipeline and release the change with a single commit.

If you'd rather build one end to end first, the Getting Started with Task Graphs guide walks through a working graph, and this guide is where it sends you for the theory. To build a graph visually instead, see Getting Started with Pipeline Builder.

This is a concepts guide, so there's no single build. The snippets work on one small pipeline: DEMO_ORDERS_PIPELINE, an hourly task graph with an extract root, a transform child and a finalizer. Along the way you'll schedule it, version it, branch it on conditions and configuration, and wire it to notifications.

The snippets are illustrative, not a script to run top to bottom. They assume a scratch schema and a warehouse like the ones below; every object uses a DEMO_ prefix so it's easy to find and drop.

Later snippets refer to a few small tables (DEMO_RAW_ORDERS, DEMO_ORDERS_CLEAN, DEMO_RUN_LOG), a stream on the raw table, a dynamic table and a stored procedure. Create your own versions if you want to try a snippet.

The whole guide works with three tasks: DEMO_T_EXTRACT, DEMO_T_TRANSFORM and DEMO_T_FINALIZE. When a snippet shows a new idea, it redefines one of them rather than adding another task.

Start with the shape of the work, not the tool. Snowflake gives you three ways to decide when something runs:

Dynamic tables first. If a step can be written as a query, a dynamic table usually needs less code: Snowflake refreshes a pipeline of them in dependency order, an incremental refresh computes only the rows that changed, and Snowflake can skip refreshing downstream tables whose upstream data hasn't changed. The Comprehensive Guide to Dynamic Tables covers them in depth.

Tasks for procedures, dbt Projects and Code Bundles. When a step is a stored procedure, a dbt Project or a Code Bundle, it doesn't fit a SELECT. Run it from a task.

Task graphs when there's more than one step. A graph runs steps in dependency order, runs siblings in parallel, and gives the whole pipeline one schedule, one retry policy and one place to change settings.

You can mix them. A common shape is dynamic tables for the transformations and a task graph around them for loads, exports and notifications, which is the final concept section of this guide.

A task runs one SQL statement, a Snowflake Scripting block, or a call to a stored procedure. Everything a task graph does is built from these settings, so they're worth knowing well.

A task runs either on serverless compute, where Snowflake predicts the resources from recent runs and sizes them for you, or on a user-managed warehouse you name with WAREHOUSE. Leave out the warehouse and the task is serverless.

Serverless tasks bill for the compute they actually use and suit stable workloads where keeping to the schedule matters. If a run overruns its interval, Snowflake sizes up. The largest size is the equivalent of an XXLARGE warehouse. Warehouse tasks suit warehouses that are already busy with concurrent work or loads that are hard to predict, and they bill like any warehouse, with a 60-second minimum each time it resumes.

On a serverless task, TARGET_COMPLETION_INTERVAL sets how soon a run should finish. Snowflake scales resources to meet it, and ignores it once the task is already at its largest size. You can also bound the size with SERVERLESS_TASK_MIN_STATEMENT_SIZE and SERVERLESS_TASK_MAX_STATEMENT_SIZE.

The snippets in this section are three versions of one standalone task, DEMO_T_EXTRACT, which later becomes the root of a graph.

A schedule is either an interval such as '60 MINUTE' or '10 SECONDS', or a CRON expression with a time zone for calendar-based runs. Use an interval when you mean "every N minutes", and CRON when you mean a time of day or a day of the week.

Snowflake runs only one instance of a scheduled task at a time. If a run is still going when the next one is due, that scheduled time is skipped.

A new task is created suspended. RESUME starts its schedule, and EXECUTE TASK runs it once on demand, which is handy for testing.

A WHEN clause is a Boolean condition checked before each run. If it's false, the run is skipped.

When the condition is SYSTEM$STREAM_HAS_DATA and there's no schedule, you have a triggered task: it runs when the stream has new data and doesn't use compute until then. Triggered tasks work on streams over tables, views, dynamic tables, Iceberg tables, data shares and directory tables, but not on hybrid tables or streams on external tables.

A few behaviours to know:

How streams, CHANGES and row timestamps capture what changed is its own topic, covered in Incremental Processing with Streams, CHANGES and Row Timestamps.

Three settings decide what happens when things go wrong:

A task also accepts any session parameter, such as TIMEZONE or TIMESTAMP_INPUT_FORMAT, set directly on the task. Account and user parameters aren't supported.

On a standalone task these settings live on the task. In a task graph, the retry and suspend parameters move to the graph, as the next section shows.

A task graph is a directed acyclic graph of tasks that runs as one orchestrated unit. Steps run after their predecessors finish, and siblings run in parallel.

For a long time a graph had no object of its own. Its schedule, overlap policy and retry behaviour lived on the root task, which was both the first step of the pipeline and the settings panel for the whole thing. Changing anything meant suspending that root first.

The task graph object gives the pipeline its own identity. A graph object owns:

The graph-level settings really are graph-only. Setting the schedule, overlap policy, either integration, the configuration, or the retry and suspend parameters on a member task returns an error, and that includes the root task once it belongs to a graph. Even the finalizer is declared on the graph with FINALIZER rather than on the finalizer task.

A WHEN condition on the graph decides whether a run starts at all. If the root task has its own condition, both must be true. The graph's condition isn't checked again for the other members, and members can still carry their own WHEN clauses, which is how branching works.

OVERLAP_POLICY on the graph decides whether a new run can start while the previous one is still going:

Allow overlap only when overlapping runs can't produce duplicate or incorrect data. Otherwise, widen the schedule, move heavy steps to serverless, or size the warehouse up so a run finishes before the next one is due.

A task graph has exactly one root, named in ROOTS. Member tasks are ordinary tasks, created with CREATE TASK and connected with AFTER. The graph object sits over a set of tasks that are already linked. Add a task with an AFTER that reaches the root and it joins the graph; remove its last link and it leaves.

The task graph documentation sets the limits at 1,000 tasks per graph, and up to 100 parents and 100 children for a single task.

These first CREATE OR REPLACE statements are fine because the graph doesn't exist yet. The members carry no compute settings of their own here. The warehouse, timeout and log level come from the graph's TASK_DEFAULTS:

Then the graph object holds everything that describes the pipeline:

Watch the clause order. On a task, COMMENT goes before AFTER; the other way round is a syntax error. On a graph, TASK_DEFAULTS goes before WHEN.

CREATE OR ALTER TASK GRAPH is declarative: anything the statement leaves out is unset or reset on an existing graph, so omitting FINALIZER removes the finalizer. A later ALTER TASK GRAPH ... SET TASK_DEFAULTS merges into the existing defaults instead of replacing them, and you remove a single default by naming it in UNSET TASK_DEFAULTS.

This is the feature that changes how you run pipelines day to day. You no longer suspend a graph to change it.

A running graph executes against a committed version: a frozen snapshot of the graph's shape and settings, captured when you commit. Once a graph has a committed version, every edit to the graph or its members is staged. In-flight and scheduled runs carry on with the committed version until you release the staged changes.

The lifecycle is five statements:

A newly created graph doesn't run. RESUME needs a committed version, so a first deployment is COMMIT followed by RESUME. A graph with neither a schedule nor a WHEN condition can't be resumed, but you can still commit it and run it on demand. If you run EXECUTE TASK GRAPH on a graph that has never been committed, Snowflake commits it once and runs that first version.

Staged edits aren't in the next manual run. EXECUTE TASK GRAPH always runs the committed version, so commit first if you want your edits included.

Here's the loop for changing a running graph: stage an edit, look at what a commit would release, then either commit it or throw it away.

TASK_GRAPH_CHANGES returns one row per added, removed or changed entity, with the DDL before and after the change. It needs the graph to have been committed at least once. If you change your mind, REVERT instead of COMMIT. Member tasks that were added but never committed are disconnected from the graph, not dropped.

Member edits work the same way. Use CREATE OR ALTER TASK or ALTER TASK on a member and the change is staged with everything else. Here the transform step gets a longer timeout than the graph default:

SHOW TASKS shows your latest, possibly staged, definitions. SHOW COMMITTED TASKS shows what is actually deployed. If the two differ, there are staged changes waiting for a commit. SHOW TASK GRAPHS also has a has_pending_changes column that tells you the same thing at a glance.

The documentation says a task in the graph's committed version can't be dropped or replaced while it's part of that version. On a resumed graph, Snowflake enforces this: DROP TASK and CREATE OR REPLACE TASK on a member both fail with an error that the task is part of a running task graph.

Don't suspend the graph to get around it. The error suggests suspending, but on a suspended graph both the drop and a CREATE OR REPLACE of the member go through, and the graph's committed version is discarded with them. Remove a member in three steps instead, which works while the graph keeps running: sever it, commit, then drop it.

The next section adds the transform step back with CREATE OR ALTER TASK and an AFTER clause.

This is also why member edits use CREATE OR ALTER TASK or ALTER TASK rather than CREATE OR REPLACE: altering keeps the task in the graph and stages the change, while replacing it is rejected on a running graph. Every member snippet from here on uses CREATE OR ALTER TASK for that reason.

A note on stored procedures. The version freezes task definitions, not the code a task calls. If a procedure changes mid-run, the run can pick up the new procedure body.

A graph is more than "run B after A". Members can branch on outcomes, read shared configuration, hand values to each other and find out where they are. This section explains the theory; Getting Started with Task Graphs builds a graph that uses all of it.

By default a child runs only when all its predecessors succeed. If one fails, the child isn't scheduled, never appears in task history, and nothing downstream of it runs.

Advanced branching lets a child run on other outcomes. Call one of two functions in the child's WHEN clause:

The two snippets below are alternative definitions of the transform step, and both still refresh the daily table. As a fallback that runs only when the extract failed, it refreshes from the rows already loaded and records why:

This works on the root: in this guide's test run, a child with ALL_FAILED ran when the root task itself failed. In a real graph you would usually keep the normal transform and add a separate fallback task beside it; here the transform is redefined to keep the graph at three tasks.

You can combine the functions with each other and with other WHEN expressions, such as a stream check or a predecessor's return value. The task runs only when the whole expression is true. This version refreshes once the extract has finished, whether it succeeded or failed, but only if new orders are waiting:

Three rules change once any task in a graph uses either function:

1. Every task is scheduled. Each one evaluates its own condition and either runs or is recorded as SKIPPED, so the graph produces more task history rows. Each evaluation also carries a small cloud services charge, even when the task is skipped.
2. Conditions wait for everyone. A task waits until all its resumed predecessors reach a terminal state before it checks its condition, so AT_LEAST_ONE_FAILED doesn't fire on the first failure. Suspended predecessors aren't counted.
3. Tasks without a condition keep their old behaviour. They act as if their condition ended with ALL_SUCCEEDED.

If the root is skipped or cancelled, nothing else runs, with or without branching.

Retries change too. A retry starts from the earliest failed tasks, those with no failed ancestors. Snowflake then re-evaluates successors using the states their predecessors reached across all attempts, so a task that was skipped on the first attempt can run on the retry.

A finalizer is one task that runs after every other task in the graph has finished or failed to finish. Use it for cleanup and for recording or announcing the outcome. Declare it on the graph with FINALIZER, as in the graph definition above.

A finalizer can't have children. It's scheduled only when nothing else in the run is running or queued, and it doesn't start if the root was skipped. Because SYSTEM$TASK_COMPLETION_STATE works in a finalizer's WHEN clause, a finalizer can react to what a specific step did.

CONFIG on the graph is a JSON string every member can read with SYSTEM$GET_TASK_GRAPH_CONFIG. To change it, set the whole string again; you can't update one key. For a one-off run, EXECUTE TASK GRAPH ... USING CONFIG merges your values over the graph's for that run only and leaves the definition alone:

A task can hand a string to its children with SYSTEM$SET_RETURN_VALUE. A child reads it with SYSTEM$GET_PREDECESSOR_RETURN_VALUE, naming which predecessor if it has several. Return values also show up in the RETURN_VALUE column of task history, which makes them useful for logging.

The root below reads its region from the graph configuration, loads that region, and passes the region on:

A child can use that value in its body or in its WHEN clause:

Task names in these functions are case-sensitive, and unquoted names are stored in uppercase, so pass the uppercase form.

SYSTEM$TASK_RUNTIME_INFO tells a running task about its run: for example the original scheduled time of the graph run, or the name and id of the current graph through CURRENT_TASK_GRAPH_NAME and CURRENT_TASK_GRAPH_ID. The finalizer above uses it to stamp the graph name into the run log.

Set ERROR_INTEGRATION and SUCCESS_INTEGRATION on the graph to send graph-level notifications through a notification integration. The role needs USAGE on each integration. Tasks that use error notifications send one for each failed retry attempt.

This assumes a notification integration named DEMO_PIPELINE_ALERTS already exists. For anything richer than "tell me when it fails", such as a custom message or a summary of every step, have the finalizer send it.

To watch for patterns across runs rather than single failures, create an alert from a template. An alert template is a ready-made condition and action that Snowflake renders into an alert from a few parameters in TEMPLATE_PARAMS, so you write no monitoring query yourself. Two templates cover tasks:

SYSTEM$LIST_ALERT_TEMPLATES lists the templates your account offers, and SYSTEM$GET_ALERT_TEMPLATE shows each one's variables, defaults and allowed values.

Before you create one, check three things:

Set the email integration in two places: as the EMAIL_NOTIFICATION_INTEGRATION variable and in notification_config, together with the recipients. In this guide's test run, with the integration only in notification_config, the rendered alert had no integration and would have emailed its creator instead of the listed recipients.

The alert is created suspended; RESUME it to start the schedule. To test it without waiting for the schedule, run EXECUTE ALERT DEMO_TASK_ERROR_RATE_ALERT. Here DEMO_PIPELINE_ALERTS must be an email notification integration. This alert is serverless; add WAREHOUSE = DEMO_WH to run it on the warehouse instead. CREATE OR ALTER re-applies TEMPLATE_PARAMS in full, so any variable you leave out resets to its default.

The task graph object adds its own inspection commands, and the existing task monitoring surfaces understand graphs too:

Pass either a graph argument or a root-task argument to these functions, not both. For a task graph object the graph columns are filled in and the root-task columns are NULL. For an old root-task graph it's the other way round, so a filter on the root task id finds nothing for a graph object.

Take the id from the id column of SHOW TASK GRAPHS and swap it in for the placeholder:

To see whole runs, query COMPLETE_TASK_GRAPHS by graph name. Its GRAPH_RUN_GROUP_ID is what you pass to EXECUTE TASK GRAPH ... RETRY GRAPH RUN GROUP to retry an older run. To retry the latest failed run from the tasks that failed, use RETRY LAST. Either works for runs whose first attempt started in the last 14 days.

Snowsight also shows tasks and task graphs, including each run's graph with the state of every task. See View tasks and task graphs in Snowsight.

Tasks run as their owner. By default a task runs as a system service with the privileges of the task's owner role, not as the person who created it, so it keeps running if that person is dropped or locked out. If a task's SQL works in a worksheet but fails as a task, the owner role is usually missing a privilege.

The rules that matter for graphs:

Run as a user when governance needs a person. EXECUTE AS USER runs tasks on behalf of a named user instead of the system service. This is useful when masking or row access policies look at the querying user, when you need secondary roles, or when you want every run attributed to an account in the audit trail. The owner role needs IMPERSONATE on that user, and the user needs the owner role. For production, use a dedicated service user for the team rather than a person. On a graph, set it once in the member defaults:

This assumes a service user named DEMO_PIPELINE_SVC that has been set up as described.

Warehouse tasks bill as warehouse time while they run. Serverless tasks bill for the compute they use, scaled by Snowflake from task history. A few habits keep the bill small:

You can write the same graph three ways. They all produce a task graph object, so everything above applies to each:

The Snowflake Python API documents tasks and root-task graphs (its DAG classes), not the task graph object, so this guide doesn't cover it.

In a DCM project you declare members with DEFINE TASK and the graph with DEFINE TASK GRAPH, which takes the same clauses as CREATE OR ALTER TASK GRAPH. It adds one thing: a desired state, STARTED or SUSPENDED, placed at the end of the statement. It belongs to the graph only. A member task can't have its own desired state, while a standalone DEFINE TASK outside any graph still can.

DEFINE statements run only inside a DCM project's definition files, not as standalone SQL. The desired state is DCM metadata and doesn't appear in the deployed graph's DDL. A single deployment of a task graph can include up to 1,000 tasks. A project can hold both task graph objects and root-task graphs, and a task can move between them only while the graph that contains it is suspended.

DCM Projects for Tasks deploys a graph this way and changes it through a second plan and deploy.

Pipeline Builder is a visual designer in Workspaces. You drag notebooks, Python files, ML Jobs and SQL onto a canvas and connect them. Connect two or more and it creates a task graph object with a schedule, run policies and task defaults. It packages project code into Code Bundles for you, and every pipeline is a DCM project, so you plan before you deploy and keep the pipeline in Git.

It orchestrates; it doesn't build transformations. Dynamic tables and views go in the same DCM project's definition files, and stored procedures run from a SQL task. All tasks in a graph must be in one schema, defined in the same project, owned by one role, and named with fully qualified names.

See Getting Started with Pipeline Builder for a walkthrough.

If you already have a graph defined by its root task, you don't rebuild the members. A task graph object sits over tasks that are already connected by AFTER, so migrating means moving the graph-level settings off the tasks and onto a new graph object.

Suppose DEMO_T_EXTRACT and DEMO_T_FINALIZE had started life as a root-task graph, with the schedule on the root and FINALIZE on the finalizer. If you tried the earlier snippets, these tasks already belong to DEMO_ORDERS_PIPELINE, so try this in a fresh schema or after dropping that graph.

1. Suspend the root task. In a root-task graph, the root must be suspended before you modify any of its tasks.
2. Move the graph-level settings off the root. Unset the root's schedule and any overlap, integration, configuration, retry or suspend settings. A task graph object rejects a root that still carries them.
3. Unlink the finalizer. Unset FINALIZE on the finalizer task, because the graph object names its finalizer with FINALIZER instead.
4. Create the graph object with those settings, the root in ROOTS, and the finalizer in FINALIZER. Move shared compute and logging into TASK_DEFAULTS.
5. Commit and resume the graph.

After the move, monitor by graph: TASK_HISTORY with the graph id, and COMPLETE_TASK_GRAPHS with the graph name. Filters on the old root task id no longer return the graph's runs.

Dynamic tables refresh themselves in dependency order, so you don't need a graph for the refreshes themselves. You need one when the refreshes have to line up with work a dynamic table can't do: loads before them, exports and notifications after them, a procedure in the middle.

For that, create the dynamic tables with SCHEDULER = DISABLE. Snowflake then doesn't schedule them, and a task refreshes each one in the order you choose. These refreshes don't cascade up or down the pipeline, and you can't set a target lag on such a table, so the graph owns the order, the timing and the error handling. A task body can mix refreshes with procedure calls:

The version a graph run uses covers task definitions, not the procedures they call, as noted in the versioning section.

Pipeline design for dynamic tables, from target lag to refresh modes, lives in the Comprehensive Guide to Dynamic Tables. For a larger task graph that runs ML Jobs end to end, see End-to-End Task Graph.

If you tried the snippets, everything lives in one database:

Orchestration in Snowflake comes down to one question: who decides when work runs? If a query can express it, let a dynamic table decide. If it's one step, a task on a schedule or a stream does the job. If it's a pipeline of imperative steps, a task graph object gives you one schedule, one retry policy, member defaults, branching on outcomes, and edits you release with a commit instead of a suspend.