Welcome to the Build Data Products and Data Mesh with dbt platform Quickstart guide! Before you begin getting your hands into the technology, let's define what a data mesh is and describe what motivates the topic.

A data mesh is a decentralized data management architecture comprising domain-specific data. Instead of having a single centralized data platform, teams own the processes around their own data.

In a data mesh framework, teams own not only their own data, but also the data pipelines and processes associated with transforming it. A central data engineering team maintains both key data sets and a suite of self-service tools to enable individual data ownership. Domain-specific data teams then exchange data via well-defined and versioned contracts.

Data mesh architecture aims to solve the lingering issues in data systems by adopting the same approach to data systems that software engineering teams take to software systems. It does so by enacting the following four principles:

1. Decentralized domain-oriented data
2. Data as a product
3. Self-service data platform
4. Federated computational governance

In the end, an operational data mesh architecture may look something like this:

The following two conditions are indicators that you may benefit from a data mesh architecture.

The architectural choice to use a data monolith has numerous knock-on effects. Monolithic approaches break down a data processing pipeline into several stages-ingestion, processing, and serving.

A single team often handles all of these stages. This approach can work at first but breaks down with scale. As more and more requests come in, the data engineering team finds itself unable to respond to them promptly. This leads to an ever-growing backlog of feature requests and bug fixes. This slows down the pace of innovation and also leads to the system becoming more brittle over time.

In this approach, data engineering teams often can't gain the full context behind the underlying data in this model. Since they're responsible for maintaining data sets from multiple disparate teams, they often don't fully understand the business rationale behind the data.

This can lead them to make uninformed -- and, sometimes, harmful -- decisions that impact business decision-making. For example, a data engineering team may format data in a way that the sales department doesn't expect. This can lead to broken reports or even lost data.

Monolithic systems rarely have clear contracts or boundaries. This means that data formatting changes upstream can break an untold number of downstream consumers. The result? This can cause teams to avoid making necessary changes for fear of breaking everything. This leads to monolithic systems gradually becoming outdated, brittle, and hard to maintain.

Finally, collaboration also becomes more difficult in a monolithic system. Since no one is familiar with the entire codebase, it takes more people and more time to complete data-related tasks. This affects time to market for new products and features -- which impacts the company’s bottom line.

A data product is a useful information asset, accessed by a clearly defined interface, with an expectation of timeliness and correctness by the consumer. Teams build and maintain data products using processes and technologies that they are responsible for, though they could be supported by other platform-oriented teams.

The properties of a data product are:

Throughout this guide, imagine yourself building a data platform with Snowflake and dbt platform for your organization, starting with two teams:

1. Central data team: a team of experienced data engineers who build foundational data marts in Snowflake organized in a star schema of fact and dimensional tables
2. Finance team: a team that includes data analysts in the finance department who want to own their data and data processes

Each step of the guide will learn how to use features of both Snowflake and dbt platform to achieve building data products.

At the end of this guide, you will have achieved:

Here are the requirements for this Quickstart Guide:

In the following step within this Quickstart Guide, you will be given instructions on how to sign-up for a Snowflake account.

Here are the requirements for this Quickstart Guide:

In the following step within this Quickstart Guide, you will be given instructions on how to sign-up for a dbt platform account. Live webinar attendees should use this workshop signup page[https://cloud.getdbt.com/workshop] and the passcode provided by your instructor. For others, we recommend using Snowflake Partner Connect.

In this step, you will set up Snowflake and dbt platform accounts for use in the rest of the Quickstart guide.

Navigate to signup.snowflake.com and follow the steps. A few things to note:

An email to activate your account will be sent after following the steps in the sign up flow. Click the button in the email "Click to Activate"

Next, specify a username and password for your account.

When prompted to either explore a sample data set or load data into snowflake, select the "Skip for Now" button.

Your Snowflake account is ready!

1. Go to this workshop signup page.
2. Enter your first and last name. Select "Build Data Products at Scale with dbt platform (July 2025)" as the workshop name unless otherwise specified by your live instructor.
3. Use the passcode provided by your live instructor.
4. Magic! Save down your password. You can now skip Option B below and move to section 3 of the quickstart guide.

1. We recommend that you use Snowflake Partner Connect to set up your dbt platform account and project. Using Partner Connect will allow you to create a complete dbt account with your Snowflake connection, managed repository, environments, and credentials with just a few clicks.

2. In the Snowflake UI, click on Data Products in the lefthand sidebar, then Partner Connect which is located within the Data Products section.
   

   

   
   

   Check to make sure your role is set as the ACCOUNTADMIN role. If you're using the classic console, the Partner Connect button will be in the top bar just right of center.

3. Find the dbt tile by typing dbt into the Search Partner Connect search bar. Click on the dbt tile.

   

4. You should now see a popup that says Connect to dbt that contains all of the associated objects created by Partner Connect. Click on the Optional Grant dropdown menu and add Snowflake_Sample_Data in the text box. This will grant your new dbt user role access to the database. Once that’s entered, click Connect. This will create a dedicated dbt user, database, warehouse, and role for your dbt platform trial.

   

5. When you see the popup that says Your partner account has been created, click on Activate.

   

6. You should be redirected to a dbt platform registration page. If your email is already associated with a dbt platform account, you'll see the screen below. Fill out the Account Name and click "Complete Registration". That will then redirect you to the login screen. Fill out the form and make sure to save the password somewhere for login in the future.
   

   

If this is your first time in dbt platform, you'll see the screen below. Add your Account Name and a password for your user.

7. You should now be redirected to your dbt platform account, complete with a connection to your Snowflake account, a deployment and a development environment, as well as a sample job.

   

8. To help you version control your dbt project we have connected it to a managed repository, which means that dbt Labs will be hosting your repository for you. This will give you access to a git workflow without you having to create and host the repository yourself. You will not need to know git for this workshop; dbt platform will help guide you through the workflow. In the future, when you're developing your own project, feel free to use your own repository. This will allow you to play with features like Slim CI builds after this workshop.

9. Now it is best to upgrade this dbt platform account to Enterprise. If you are participating in the Snowflake Summit Hands On Lab session, please use this form to submit your account information so your account can be upgraded for the duration of the Hands On Lab. The required information can be found by navigating to Account Settings. The Account ID should be in the top section "Account Information" - this is what we'll need to upgrade your account.

In this step, you will be setting up Snowflake and dbt platform for two teams: the central data team and the finance team, as shown in the diagram below. You will be using least privileged access principles in order to properly secure the data.

The central data team is well-established in the organization and the average team member is capable of building data pipelines that powers business reporting across various domains: finance, marketing, sales, customer support, and so on. The team uses data management best practices like organizing data in dimensional models for maximum re-usability in various BI and AI/ML applications.

The finance team is an operational team. Finance team members are accustomed to consuming dashboards and building spreadsheets, however more and more demands on data has led to the finance team owning and managing more data to rapidly respond to changing demands. And so, the team has upskilled team members and brought on an analytics engineer to use Snowflake and dbt platform, in order to create data pipelines building off of the foundation project for daily reporting use-cases.

Use this app to quickly provision the Snowflake and dbt platform setup for this Quickstart Guide. To do so, you'll need to first locate the account ID (Settings > Account) and then create a Service Token with Account Admin priveleges:

Come back to this guide when you have successfully followed the steps in the app!

The app creates the following resources for you:

Before we jump into the next section, let's take a moment to setup our developer credentials (click Account Settings and then Credentials) for both the foundational and finance projects. Add in the Snowflake username and password you just created:

When setting up dbt platform for production, there are four recommended security options to configure. They are out-of-scope for this particular Quickstart Guide, but worth mentioning at this point:

With the Snowflake and dbt platform resources provisioned in this step, you have begun laying the first layer of an interoperable and secure data mesh. dbt platform adds a layer of addressability and discoverability to the data platform. In the next step, you will build your first data product.

Here is where you are in the journey towards a data product:

Now it's time for you to add dbt code in the foundational project using the dbt platform IDE. Using the sample TPCH dataset provided by Snowflake, the dbt code will create a fct_orders table representing all of the orders within our organization. The code below has three layers of transformations: raw data sources, staging models, and core business logic. And by using dbt, you automatically have end-to-end data lineage.

Here are the steps:

1. Navigate back to the foundational project via the account / project dropdown in the top-right section of the navbar.
2. Then, navigate to the dbt platform Studio (IDE) via the Studio dropdown in the navbar to verify your setup.
3. If the repo you are working on is empty, click the Initialize dbt project button and commit the changes.
4. Create a new branch.
5. Delete the models/example folder.
6. Navigate to the dbt_project.yml file and rename the project (line 5) from my_new_project to foundational_project.
7. In your dbt_project.yml file, remove lines 39-42 (the my_new_project model reference).
8. In the File Explorer, hover over the models directory and click the ..., then select Create Folder.
9. Create two new folders: staging and marts. If you've done this right, these directories will be inside the models directory.

Now that you've set up the foundational project, let's start building the data assets. Set up the staging layer as follows:

1. Create a new YAML file models/staging/sources.yml.
2. Declare the sources by copying the following into the file and clicking Save.

3. Create a models/staging/stg_customers.sql file to select from the customers table in the tpch source.

4. Create a models/staging/stg_orders.sql file to select from the orders table in the tpch source.

5. Create a models/staging/stg_line_items.sql file to select from the line_items table in the tpch source.

Now set up the marts layer by creating a file models/marts/fct_orders.sql to build a fact table with order details:

Navigate to the Command bar at the very bottom of the IDE and execute dbt build. This will both validate the work you've done thus far and build out the requisite models into your sandbox within Snowflake.

At this point, you may also see what you have built by looking at the Lineage tab in the dbt platform IDE, and as shown in the graphic below.

To finish this step up, go to the Version control section in the IDE to:

1. Click the Commit and Sync button to commit your changes.
2. Merge your changes to the main or production branch.

You now have built the first useful tables in the foundational project, which any user or application can access using SQL. In the next step, you will secure the PII access to privileged users.

Here is where you are in the journey towards a data product:

Congratulations! You built a useful dataset. But, it's not secure. Organizational data privacy policies require Personally Identifiable Information (PII) to be restricted only to a few select people within the organization. Naturally, this restriction is best checked within Snowflake itself, as various applications within the organization access data, so restricting the data at the source helps protect your customers' privacy.

Use Snowsight to execute the following commands to create object tags and a data masking policy.

Now it's time to jump into dbt platform to use a model post-hook to apply the masking policy to the name column of the fct_orders dbt model immediately after the table is built.

Open up the fct_orders.sql file and modify the config block at the top to include the post_hook argument:

You have now secured the PII produced in this data product so it is restricted to privileged users, governed by native Snowflake features. In the next step, you will add model contracts and enforce grants with dbt.

Here is where you are in the journey towards a data product:

Now that you have set up with data masking, it's time to set up proper grants to the foundational project's fct_orders table, add a model contract, and make it accessible to other dbt projects.

By using these configurations within your project, you'll be, in effect, creating a stable set of APIs that your downstream projects are able to reliably consume.

To begin, add the file models/marts/core.yml and add the code below:

This YAML file does the following:

To finish this step up, go to the Version control section in the IDE and:

1. Click the Commit and Sync button to commit your changes.
2. Merge your changes to the main or production branch.

In this step you added a straightforward YAML file, and now fct_orders is a tested data model with documentation, will have the proper grants applied when built, and is publicly available for other dbt platform projects to use. In the next step, you will build the foundational project in production.

Here is where you are in the journey towards a data product:

Before a downstream team can leverage assets from this foundational project, you will need to run a deployment job successfully.

To run your first deployment dbt platform job, you will need to create a new dbt platform job.

1. Click Orchestration and then Jobs.
2. Click Create job and then Deploy job.
3. Fill in details about the job, calling it Daily Job.
4. Select the Generate docs on run option. This will enable dbt platform to pull in metadata from the warehouse to supplement the documentation found in the Catalog section.

5. You may optionally set a schedule for the the job to run on.
6. Save the job
7. Then, click Run now to trigger the job.

In this step, you ran a dbt platform job that deployed a production version of the data to Snowflake and registered the project metadata in dbt platform's metadata storage. This was necessary for the next step, which is to reference the Foundational data from within the Finance dbt platform project.

Here is where you are in the journey towards a data product:

In this penultimate step, you will now set up dbt platform to build the finance team's data product by building off the foundational data product, the result of which is an architecture as shown in the diagram below.

1. First, select the Finance Project from the dbt platform project selector.
2. Now, open up the dbt platform IDE in the Studio toolbar.
3. If you’ve also started with a new git repo, click Initialize dbt project button under the Version control section and commit the changes.
4. Delete the models/example folder
5. Navigate to the dbt_project.yml file and rename the project (line 5) from my_new_project to finance_project.
6. In your dbt_project.yml file, remove lines 39-42 (the my_new_project model reference).
7. In the File Explorer, hover over the project directory, click the ..., then select Create file.
8. Name the file dependencies.yml and add the upstream platform project and click Save.

8. You're now set to add a model that references the fct_orders model created in the separate project by your platform team. In your models directory, create a file models/agg_segment_revenue.sql

Notice the cross-project ref by using two arguments to the function: 1) name of the project, as defined within that upstream project and declared in dependencies.yml, and 2) the name of a public model in that project.

9. Save your file and notice the lineage in the bottom pane.
10. Compile the code and notice that this model is now referencing the production version of the foundational project's fct_orders table.

To finish this step up, go to the Version control section in the IDE to:

11. Click the Commit and Sync button to commit your changes.
12. Merge your changes to the main or production branch.

Now run a production dbt platform Job of the finance project, as you did with the foundational project.

13. Click Orchestration and then Jobs.
14. Click Create job and then Deploy job.
15. Select the Generate docs on run option. This will enable dbt platform to pull in metadata from the warehouse to supplement the documentation found in the Catalog section.
16. Then, click Run now to trigger the job.

With a few lines of code, the finance team now is building directly off of the central data team's work. They are synchronized which ensures data consistency, but autonomous to create their data product to power their own applications. The finance team can also feel confident that the data they are addressing will be reliable, due to the model contract that's in-place with the fct_orders model.

Here is where you are in the journey towards a data product:

You have now completed building projects for this Quickstart Guide. Now it is time to try out dbt Catalog to discover and drill into the dbt assets available within your dbt platform account.

1. Head back to the foundational project.
2. Click Catalog from the upper menu bar.
3. Using the Search bar at the top left of the page, search for orders.

3. Click on fct_orders and scroll through the page, noting Lineage, Build Status, and Description.

4. Click on the Columns tab, and then expand the order_key column to see the Column Level Lineage.

5. Lastly, click on the Project Recommendations tab on the left hand side of the page to see best practices directly in dbt Catalog.

With dbt Catalog, data products built with dbt are automatically published and searchable, allowing for easy discovery and re-usability. Furthermore, you can use dbt Catalog to drill into data product details including documentation, build status, and lineage at the column-level, model-level, or project-level.

You have now completed your data product journey!

During this quickstart guide, you used native features of Snowflake and dbt platform that can be combined to create data products. By using dbt platform, you gained additional governance, automation, and interoperability mechanisms than from Snowflake alone. Overall, these two self-service platforms combined increase the likelihood of success of a data mesh project.

If you are interested in this solution for your organization, contact dbt Labs to get started!