Learn how to set up Snowflake Openflow using Snowpark Container Services (SPCS) in about 25 minutes. You'll create the foundation needed to start ingesting data from external sources using pre-built connectors.

Openflow is Snowflake's managed service for building data pipelines in Snowpark Container Services (SPCS). It provides pre-built connectors that make it easy to ingest data from various sources into Snowflake.

Key Benefits:

Openflow supports 19+ connectors including:

For a complete list with descriptions, see Openflow connectors.

Before starting, ensure you have:

Setting up Openflow involves four main tasks:

Once you complete this 25-minute setup, you'll have a production-ready Openflow environment. Here's what you can do immediately:

✅ Start Ingesting Data

✅ No Code Required

✅ Production-Ready Infrastructure

Before creating a deployment, you need to configure core Snowflake components including the Openflow admin role, required privileges, and network configuration.

This quickstart references the companion GitHub repository which includes Jupyter notebooks and SQL scripts for various Openflow connectors:

Repository: https://github.com/Snowflake-Labs/sfguide-getting-started-with-openflow-spcs

SQL Scripts:

All SQL commands in this quickstart are available as downloadable scripts:

Key Notebooks:

If you downloaded the SQL scripts from the companion repository, here's how to import and execute them in Snowsight:

Using Projects (Recommended):

1. Navigate to Projects: In Snowsight, go to Projects → Worksheets (left sidebar)
2. Create or Open Project: Click "+ Project" to create a new project, or open an existing one
3. Add SQL File: Click "..." (more options) → "Import SQL file"
4. Select Downloaded Script: Choose the .sql file you downloaded (e.g., quickstart_setup_core.sql)
5. Execute Commands:
   • Run All: Click the ▶ Run All button to execute the entire script
   • Step-by-Step: Select individual SQL statements and click ▶ Run for granular control

The OPENFLOW_ADMIN role is the primary administrative role for managing Openflow deployments.

Log into Snowsight and open a SQL worksheet.

Run the following SQL commands to create the Openflow admin role:

This creates the admin role and grants it the necessary permissions to create and manage Openflow deployments.

Check that all core resources were created successfully:

After configuring core Snowflake, create an Openflow deployment. This is the container environment where Openflow will run.

1. Navigate to Openflow: Go to Work with data → Ingestion → Openflow
2. Openflow Interface: You'll see three tabs:
   • Overview - List of available connectors and documentation
   • Runtimes - Manage your runtime environments
   • Deployments - Create and manage Openflow deployments

1. Navigate to Deployments Tab: Click on the Deployments tab
2. Create Deployment: Click Create Deployment button
3. Enter Deployment Name: QUICKSTART_DEPLOYMENT
4. Complete the wizard: Follow the prompts shown in the GIF below

Check that your deployment is running via the Snowsight UI:

1. Navigate to Deployments Tab: Go to Work with data → Ingestion → Openflow → Deployments
2. Check Status: Look for your deployment with status ACTIVE

Expected status: ACTIVE

Create a runtime role that will be used by your Openflow runtime. This role needs access to databases, schemas, and warehouses for data ingestion.

External Access Integrations allow your runtime to connect to external data sources. This quickstart creates one integration with network rules for both Google Drive and PostgreSQL connectors.

Create a runtime associated with the previously created runtime role. A runtime is the execution environment for your Openflow connectors.

Follow these steps to create your runtime:

1. Navigate to Runtimes: Go to Work with data → Ingestion → Openflow → Runtimes tab
2. Click Create Runtime: Click the Create Runtime button in the top right
3. Enter Runtime Name: QUICKSTART_RUNTIME
4. Select Runtime Role: Choose QUICKSTART_ROLE from the dropdown
5. Select External Access Integration: Choose quickstart_access from the dropdown
6. Select Compute Pool: Choose an existing compute pool from the list
7. Click Create: Complete the runtime creation

Check that your runtime is active:

1. Navigate to Runtimes Tab: Go to Work with data → Ingestion → Openflow → Runtimes
2. Check Status: Look for QUICKSTART_RUNTIME with status ACTIVE

Expected status: ACTIVE

Once your runtime is active, you can access the Openflow canvas to add and configure connectors:

Click on the runtime name (QUICKSTART_RUNTIME) to open the canvas where you can add connectors and build data pipelines.

With your Openflow Snowflake Deployments set up, you're ready to configure connectors to ingest data from external sources.

After completing this guide, you have:

You're now ready to configure connectors and build data pipelines!

Now that your infrastructure is ready, here are your next options:

Add and configure connectors for your data sources:

Use the notebooks from the companion repository for detailed connector setup:

Try these common use cases:

Issue: Deployment status stuck in CREATING or shows ERROR

Solutions:

1. Verify Role Privileges:

   -- Check OPENFLOW_ADMIN has required privileges
   SHOW GRANTS TO ROLE OPENFLOW_ADMIN;
   

2. Review Event Table Logs (if enabled):

   -- Query event table for errors
   SELECT * FROM OPENFLOW_CONFIG.EVENTS.openflow_events
   WHERE RECORD:severity = 'ERROR'
   ORDER BY TIMESTAMP DESC
   LIMIT 100;
   

Issue: Connector fails to connect to external services (e.g., Google Drive, databases)

Solutions:

1. Verify External Access Integration:

   -- Check integration exists and is enabled
   SHOW INTEGRATIONS LIKE 'openflow_%';
   DESC INTEGRATION your_integration_name;
   

2. Check Network Rules:

   -- Verify all required hostnames are in network rules
   DESC NETWORK RULE your_network_rule;
   

3. Test Network Connectivity:

   • Ensure the external service endpoint is correct
   • Verify firewall rules allow traffic from Snowflake IP ranges
   • Check service credentials are valid

4. Review Integration Grants:

   -- Verify runtime role has access
   SHOW GRANTS ON INTEGRATION your_integration_name;
   

Issue: "Insufficient privileges" errors when creating resources

Solutions:

1. Verify Current Role:

   -- Check which role you're using
   SELECT CURRENT_ROLE();
   
   -- Switch to appropriate role
   USE ROLE OPENFLOW_ADMIN;
   -- or
   USE ROLE ACCOUNTADMIN;
   

2. Check Role Hierarchy:

   -- Verify role grants
   SHOW GRANTS TO ROLE OPENFLOW_ADMIN;
   SHOW GRANTS TO USER your_username;
   

3. Grant Missing Privileges:

   -- Example: Grant database creation if missing
   USE ROLE ACCOUNTADMIN;
   GRANT CREATE DATABASE ON ACCOUNT TO ROLE OPENFLOW_ADMIN;
   

If you continue experiencing issues:

1. Check Documentation: Openflow Documentation
2. Community Support: Snowflake Community
3. GitHub Issues: Companion Repository Issues

When you're finished with the quickstart or want to remove resources, use Snowsight UI to clean up.

Via Snowsight:

1. Navigate to Openflow: Go to Work with data → Ingestion → Openflow
2. Remove Runtime:
   • Go to Runtimes tab
   • Find QUICKSTART_RUNTIME
   • Click on the runtime and select Delete
3. Remove Deployment:
   • Go to Deployments tab
   • Find QUICKSTART_DEPLOYMENT
   • Click on the deployment and select Delete

After removing deployments and runtimes, clean up the supporting resources:

Congratulations! You've successfully set up Snowflake Openflow using Snowpark Container Services (SPCS). You now have a fully functional data integration platform ready to connect to external data sources.

1. Configure Your First Connector: Use the companion notebooks (EAI_GDRIVE.ipynb or EAI_POSTGRES.ipynb) to set up Google Drive or PostgreSQL
2. Build Data Pipelines: Set up automated data ingestion from your chosen source
3. Add More Connectors: Explore the 19+ available connectors for different data sources
4. Learn Best Practices: Review the official Openflow documentation for tips and patterns

Official Documentation:

Code and Notebooks:

Management and Operations:

We would love your feedback on this QuickStart Guide! Please submit your feedback using the GitHub issues link.