Data for Breakfast Around the World

Drive impact across your organization with data and agentic intelligence.

Snowflake for DevelopersGuidesMonitor Snowflake Postgres Logs with Datadog

Monitor Snowflake Postgres Logs with Datadog

Elizabeth Christensen

Overview

This guide walks through the full end-to-end setup for streaming logs from a Snowflake Postgres instance into Datadog using Snowflake's native event table and Datadog's Snowflake integration.

What you'll build: A pipeline where Snowflake Postgres logs are captured in Snowflake's event table and automatically ingested into Datadog's Log Explorer for monitoring, searching, and alerting.


What You'll Learn

  • How to configure Postgres logging parameters on a Snowflake Postgres instance
  • How Snowflake's event table captures and stores Postgres logs
  • How to set up Snowflake roles, grants, and network policies for Datadog
  • How to configure Datadog's Snowflake integration for log ingestion
  • How to verify logs flow end-to-end from Postgres to Datadog

Prerequisites

  • A Snowflake account with ACCOUNTADMIN access (needed for grants and network policies)
  • A Snowflake Postgres instance in READY state (or ability to create one)
  • A Datadog account with admin access to configure integrations
  • OpenSSL installed locally (for RSA key generation)
  • psql installed locally (for testing connections)

Know your Datadog site. This guide uses placeholder values. Replace them with your site-specific values (US1, US3, US5, EU1, AP1). Your site determines the IP ranges to whitelist and the Datadog URLs.

1. Enable Postgres Logging

Snowflake Postgres supports standard PostgreSQL logging parameters. Before flipping switches, it helps to understand what Postgres can log and why it matters.

A Quick Primer on Postgres Logging

A modern Postgres instance produces comprehensive logs covering nearly every facet of database behavior. While logs are the go-to place for finding critical errors, they're also a key tool for application performance monitoring. Here are the parameters you should know about:

Log severity level (log_min_messages): Controls which server messages are logged based on severity — from DEBUG5 (most verbose) up through INFO, NOTICE, WARNING, ERROR, LOG, FATAL, and PANIC. The default is WARNING, which is a sensible baseline for most environments.

What SQL to log (log_statement):

  • none — Don't log SQL statements (errors and warnings still appear via log_min_messages)
  • ddl — Log data definition changes only (table, column, and index changes). A good default for production.
  • mod — Log all DDL plus data modifications (INSERT, UPDATE, DELETE)
  • all — Log every SQL statement. Useful for development and debugging but generates high log volume.

Slow query logging (log_min_duration_statement): Captures queries that exceed a time threshold — a great way to surface inefficient queries without logging everything. A setting of 1s or 500ms is a common starting point.

Lock wait logging (log_lock_waits): Logs any time a query waits on a lock longer than deadlock_timeout. This is safe for production with virtually no overhead and helps identify contention.

Temp file logging (log_temp_files): Logs when Postgres spills operations to disk instead of keeping them in memory. Set this to your work_mem value to catch operations that exceed available memory — a signal you may need to tune memory settings, add indexes, or rewrite queries.

Connection logging (log_connections, log_disconnections): Logs session connect and disconnect events. Helpful for auditing and spotting connection churn.

Production vs. development trade-off: log_statement=all generates a lot of log data and is generally not recommended for production workloads. For production, ddl combined with log_min_duration_statement for slow queries gives you the important details without the volume. For this guide, we use all so every test query is visible in Datadog.

For a deeper dive on logging configuration, log formats, rotation, auto_explain, and using logs for performance tuning, see Postgres Logging for Performance Optimization.

Snowflake Postgres log line prefix

The default log_line_prefix on Snowflake Postgres is:

[%p][%b][%v][%x] %q[user=%u,db=%d,app=%a] [%h]

This is a printf-style format string that gets prepended to every log line. Each escape sequence maps to a piece of metadata:

EscapeMeaningExample output
%pProcess ID (pid)1592908
%bBackend typeclient backend
%vVirtual transaction ID27/2
%xTransaction ID0
%qNon-session stop point (everything after %q only prints for session processes)(controls conditional output)
%uUsernamesnowflake_admin
%dDatabase namepostgres
%aApplication namepsql
%hClient hostname/IP34.214.158.144

This can be changed with the log_line_prefix setting as needed.

Configure Logging on Your Snowflake Postgres Instance

By default, Snowflake Postgres does not generate logs. You will need to set log_statement to enable them. If you do not have a production instance, you can set this to all for testing log ingestion. For production, review your necessary configurations.

ALTER SYSTEM SET log_statement = 'all';

2. Confirm Logs in the Snowflake Event Table

Snowflake Postgres logs are routed through Snowflake's account-level event table at SNOWFLAKE.TELEMETRY.EVENTS. This table is the bridge between your Postgres instance and Datadog.

After enabling logging (Step 1), run a few queries against your Postgres instance to generate some log data. If you set log_statement to all, here's a sample to run:

CREATE TABLE cookie_monster (
    id SERIAL PRIMARY KEY,
    cookie_type VARCHAR(100),
    flavor VARCHAR(50),
    crunchiness INT CHECK (crunchiness BETWEEN 1 AND 10),
    nom_rating DECIMAL(3,1),
    eaten_at TIMESTAMP DEFAULT NOW()
);

INSERT INTO cookie_monster (cookie_type, flavor, crunchiness, nom_rating)
VALUES
    ('Chocolate Chip', 'chocolate', 8, 9.5),
    ('Snickerdoodle', 'cinnamon', 6, 8.7),
    ('Oatmeal Raisin', 'oat', 7, 7.2),
    ('Double Chocolate', 'dark chocolate', 9, 9.8),
    ('Peanut Butter', 'peanut', 5, 8.1);

SELECT * FROM cookie_monster WHERE crunchiness > 7;
SELECT cookie_type, nom_rating FROM cookie_monster ORDER BY nom_rating DESC;
SELECT AVG(crunchiness) AS avg_crunch, MAX(nom_rating) AS best_nom FROM cookie_monster;

Now verify that logs appear in the event table:

-- Replace the instance ID with your Postgres instance's ID
-- Find it with: SHOW POSTGRES INSTANCES;  (look at the "id" column)
SELECT
    TIMESTAMP,
    RESOURCE_ATTRIBUTES['instance.id']::STRING AS instance_id,
    VALUE['MESSAGE']::STRING AS message
FROM SNOWFLAKE.TELEMETRY.EVENTS
WHERE TIMESTAMP > DATEADD('hour', -1, CURRENT_TIMESTAMP())
  AND RECORD_TYPE = 'LOG'
  AND RESOURCE_ATTRIBUTES['instance.id']::STRING = '<YOUR_INSTANCE_ID>'
ORDER BY TIMESTAMP DESC
LIMIT 20;

Tip: To find your Postgres instance ID, run SHOW POSTGRES INSTANCES; and look at the id column. It's a string like 4jypgsndvzd5ta6ufaryx6owja.

3. Create the Snowflake Role and Grants

Datadog connects to Snowflake as a dedicated role with specific privileges. This role needs access to account usage, monitoring, and the event table.

-- Use ACCOUNTADMIN to create the role and assign grants
USE ROLE ACCOUNTADMIN;

-- Create the Datadog role
CREATE ROLE IF NOT EXISTS DATADOG;
GRANT ROLE DATADOG TO ROLE ACCOUNTADMIN;

-- CRITICAL: Grant IMPORTED PRIVILEGES on the SNOWFLAKE database
-- This is required for Datadog to read the event table and account usage views.
-- Using just USAGE is NOT sufficient -- you must use IMPORTED PRIVILEGES.
GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE TO ROLE DATADOG;

-- Grant monitoring access
GRANT MONITOR USAGE ON ACCOUNT TO ROLE DATADOG;

-- Grant warehouse usage (replace with your warehouse name)
GRANT USAGE ON WAREHOUSE MY_WAREHOUSE TO ROLE DATADOG;
SHOW GRANTS TO ROLE DATADOG;

4. Generate RSA Key Pair

Datadog authenticates to Snowflake using RSA key-pair authentication (not password). You'll generate a key pair, give the public key to Snowflake, and give the private key to Datadog.

Generate the Keys

# Create a directory for the keys
mkdir -p ~/.snowflake/datadog_keys

# Generate an unencrypted RSA private key (PKCS#8 format)
openssl genrsa 2048 | openssl pkcs8 -topk8 -inform PEM -out ~/.snowflake/datadog_keys/datadog_rsa_key.p8 -nocrypt

# Extract the public key
openssl rsa -in ~/.snowflake/datadog_keys/datadog_rsa_key.p8 -pubout -out ~/.snowflake/datadog_keys/datadog_rsa_key.pub

Get the Public Key Value (for Snowflake)

# Print the public key without the header/footer lines (this is what Snowflake expects)
grep -v "BEGIN\|END" ~/.snowflake/datadog_keys/datadog_rsa_key.pub | tr -d '\n'

Copy the output -- you'll paste it into the ALTER USER command in the next step.

5. Create the Snowflake Service User

Create a dedicated service user for Datadog. This user authenticates with the RSA key pair (no password).

Key distinction: DATADOG is the role. DATADOG_USER is the user. When configuring Datadog, you'll enter DATADOG_USER as the username and DATADOG as the role. Mixing these up is a common source of "integration broken" errors.

USE ROLE ACCOUNTADMIN;

-- Create the service user
CREATE USER IF NOT EXISTS DATADOG_USER
  TYPE = SERVICE
  DEFAULT_ROLE = DATADOG
  DEFAULT_WAREHOUSE = MY_WAREHOUSE
  DEFAULT_NAMESPACE = SNOWFLAKE.TELEMETRY;

-- Assign the RSA public key (paste the key value from Step 4)
ALTER USER DATADOG_USER SET RSA_PUBLIC_KEY = '<paste_public_key_here>';

-- Grant the DATADOG role to the user
GRANT ROLE DATADOG TO USER DATADOG_USER;

6. Add Network Permissions for Datadog

Datadog connects to your Snowflake account to poll for data. If your account or Postgres instance has network policies in place, you need to whitelist Datadog's IP ranges so these connections are allowed.

Find Your Datadog Site's IP Ranges

Datadog publishes its IP ranges at a site-specific URL. The ranges you need are under the webhooks section.

Datadog SiteIP Ranges URL
US1https://ip-ranges.datadoghq.com/
US3https://ip-ranges.us3.datadoghq.com/
US5https://ip-ranges.us5.datadoghq.com/
EU1https://ip-ranges.datadoghq.eu/
AP1https://ip-ranges.ap1.datadoghq.com/

Open the URL for your site and look for the webhooks.prefixes_ipv4 array. These are the CIDR ranges to whitelist.

Example (US5 as of writing):

  • 34.149.66.128/26
  • 34.160.40.115/32
  • 35.244.255.175/32

Check this URL regularly. Datadog may add new IP ranges over time. If your integration stops working, check for updated IPs.

Create the Network Rules

You need both ingress (Datadog connecting to Snowflake) and egress (Snowflake responding to Datadog) rules:

USE ROLE ACCOUNTADMIN;

-- Ingress rule: allows Datadog to connect to Snowflake
CREATE OR REPLACE NETWORK RULE DATADOG_INGRESS
  TYPE = IPV4
  MODE = INGRESS
  VALUE_LIST = (
    '34.149.66.128/26',
    '34.160.40.115/32',
    '35.244.255.175/32'
  );

-- Egress rule: allows Snowflake to respond to Datadog
CREATE OR REPLACE NETWORK RULE DATADOG_EGRESS
  TYPE = IPV4
  MODE = EGRESS
  VALUE_LIST = (
    '34.149.66.128/26',
    '34.160.40.115/32',
    '35.244.255.175/32'
  );

-- Create a network policy using both rules
CREATE OR REPLACE NETWORK POLICY DATADOG_NETWORK_POLICY
  ALLOWED_NETWORK_RULE_LIST = ('DATADOG_INGRESS', 'DATADOG_EGRESS');

-- Assign the policy to the Datadog user
ALTER USER DATADOG_USER SET NETWORK_POLICY = 'DATADOG_NETWORK_POLICY';

Scope the policy to the user, not the account. Applying a network policy at the account level could lock out other users. Apply it only to DATADOG_USER.

Note: Datadog may add or change IP ranges over time. When this happens, update your existing network rules rather than recreating the policy.

7. Configure the Datadog Snowflake Integration

Now configure Datadog to connect to your Snowflake account using the role, user, and key you've set up.

  1. Log into your Datadog account
  2. Navigate to Integrations > Integrations (or search for "Snowflake")
  3. Click the Snowflake integration tile
  4. Click + Add Account
  5. Fill in the fields:
  • Account URL: <your-account>.snowflakecomputing.com
  • Username: DATADOG_USER
  1. Under Data Collection, enable the settings you'd like for your account. Make sure that Events Records: is tabbed on, these are the Postgres logs.

  2. Upload the private key file from above.

  3. Click Save

Account URL Format

This can be a common source of errors. Datadog normalizes underscores to hyphens in URLs, which can break the connection for accounts with underscores in their name.

Use one of these formats:

FormatExampleNotes
Connection alias (recommended)myorg-myalias.snowflakecomputing.comMost reliable if you have one set up
Legacy locatormyaccount.us-west-2.snowflakecomputing.comInclude the region
Org-account (regionless)myorg-myaccount.snowflakecomputing.comMay fail if account name has underscores

If your account name contains underscores (e.g., MY_ACCOUNT), Datadog may convert it to MY-ACCOUNT in the URL, causing authentication to fail. Use a connection alias or the legacy locator format to work around this.

To create a connection alias in Snowflake:

-- Via Snowsight: Admin > Accounts > select account > Create Connection
-- Or via SQL:
CREATE CONNECTION MY_ALIAS;
ALTER CONNECTION MY_ALIAS SET COMMENT = 'Alias for Datadog integration';

8. Generate Test Logs and Verify

With everything configured, generate some test traffic against your Postgres instance and verify the logs flow through to Datadog. See the above steps for generating data and checking the event table.

Verify in Datadog

  1. Navigate to Logs > Log Explorer in Datadog
  2. Search for postgres in the search bar to see logs being generated for the Postgres service.
  3. You should see log entries with the SQL queries you ran. Filter by some of your sample query syntax like cookie_monster to make sure you're seeing logs that you expect.

Timing: Datadog polls the event table at the interval you configured. If you set it to 1 hour, it may take up to an hour for new logs to appear. For testing, consider temporarily reducing the interval.

From here you can use Datadog's tools to search, filter, and create alerts around the Postgres logs.

Conclusion and Resources

Congratulations! You've built an end-to-end pipeline that streams Postgres logs from Snowflake into Datadog for monitoring, searching, and alerting.

What You Learned

  • How to configure Postgres logging parameters on a Snowflake Postgres instance
  • How the Snowflake event table captures and stores Postgres logs
  • How to create a dedicated Snowflake role and service user for Datadog with the correct grants
  • How to generate and configure RSA key-pair authentication
  • How to whitelist Datadog's IP ranges with Snowflake network policies
  • How to configure and verify the Datadog Snowflake integration

Resources

Updated 2026-03-31

This content is provided as is, and is not maintained on an ongoing basis. It may be out of date with current Snowflake instances