Skip to content

Expedition. Free, virtual, Nov 3–6.

Technical tracks for practitioners, outcomes for leaders.

Register now

Once a gateway endpoint is shared, the admin problems show up quickly: one team runs up credits on the most expensive model, a new model is called before anyone approved it, and when the bill arrives nobody can say who called what. The previous blog in this series covered the builder experience: locating the governed endpoint and executing initial inference requests through standard SDKs. This second installment focuses on the admin path: defining the gateway specification, granting access, monitoring traffic through trace tables, and implementing quotas to govern consumption.

The objective is to maintain a familiar developer interface while layering on the Snowflake controls required to operate AI applications responsibly.

Configure the gateway specification and route models

Discover the current specification

Before changing anything, inspect the gateway configuration:

SHOW AI GATEWAYS;

The specification column returns the gateway's live YAML. Copy it into version control before making edits. You'll need the complete specification for any subsequent update.

ALTER AI GATEWAY replaces the whole specification

ALTER AI GATEWAY … FROM SPECIFICATION replaces the entire specification. It isn't a patch. If you omit a setting, that setting is removed from the resulting specification. Treat every ALTER statement as a full write.

Always preserve the complete specification when making changes.

Safe update pattern:

  1. Run SHOW AI GATEWAYS; and copy the current specification.
  2. Save the complete YAML in version control or a local file.
  3. Make the smallest intended change.
  4. Write the complete YAML back, including every unchanged setting.

Start with logging on and payload capture off

For a first deployment, enable operational logging but leave request/response capture disabled until your security and privacy review is complete:

ALTER AI GATEWAY SNOWFLAKE FROM SPECIFICATION $$
schema_version: 1
models:
  - name: '*'
logging:
  enabled: true
  enable_client_telemetry: false
  capture_payload:
    request_response: false
$$;

Privacy note: Captured prompts and completions are visible to roles with monitoring access on the gateway. Don't enable request_response: true while secrets, personal data, or regulated data flow through the gateway unless your organization has explicitly approved that handling.

Restrict models with an allowlist

To limit which models are callable through the gateway, replace the models block with a glob pattern:

ALTER AI GATEWAY SNOWFLAKE FROM SPECIFICATION $$
schema_version: 1
models:
  - name: 'claude-*'
logging:
  enabled: true
  enable_client_telemetry: false
  capture_payload:
    request_response: false
$$;

Requests for models outside the allowlist are rejected. In our testing, switching the gateway to claude-* and calling openai-gpt-5.4 returned an authorization error indicating that the model was not in the allowed list, while claude-sonnet-4-6 continued to work normally.

To restore full model access, write the complete specification back with name: '*'. Verify the exact model identifiers available in your account with SHOW AI GATEWAYS before testing restrictions.

Enable telemetry and payload capture later

Once your privacy review clears it, you can enable the broader observability stack:

ALTER AI GATEWAY SNOWFLAKE FROM SPECIFICATION $$
schema_version: 1
models:
  - name: '*'
logging:
  enabled: true
  enable_client_telemetry: true
  capture_payload:
    request_response: true
$$;

This enables three capabilities:

  • Server-side operational traces for inference calls
  • The gateway endpoint for client-side OTLP spans
  • Full request/response payload capture in trace attributes

Treat payload capture as a data-handling decision, not simply an observability setting.

Grant inference and monitoring access with separate roles

Separate the admin role from the calling role

The role that configures the gateway specification and the role that calls the gateway for inference don't need to be the same. In production, separate these responsibilities.

By default, USAGE on the SNOWFLAKE gateway is granted to PUBLIC, so every user can call it. You can confirm this with SHOW GRANTS ON AI GATEWAY SNOWFLAKE. Revoke the PUBLIC grant and give USAGE only to the roles that need it. Revoking it blocks every user and service that relies on the PUBLIC grant, so grant the new role to those users first:

-- Create a role for gateway users
CREATE ROLE IF NOT EXISTS AI_GATEWAY_USER;

-- Grant usage on the default SNOWFLAKE gateway
GRANT USAGE ON AI GATEWAY SNOWFLAKE TO ROLE AI_GATEWAY_USER;

-- Grant the role to users who need inference access
GRANT ROLE AI_GATEWAY_USER TO USER data_scientist_1;
GRANT ROLE AI_GATEWAY_USER TO USER app_service_account;

-- USAGE is granted to PUBLIC by default. Revoke it so only granted roles can call the gateway.
REVOKE USAGE ON AI GATEWAY SNOWFLAKE FROM ROLE PUBLIC;

With USAGE on the gateway, a role can send requests to inference endpoints such as /v1/chat/completions and /v1/messages, provided the user also has the access required for the requested model. Gateway USAGE doesn't grant access to models: each model's access requirements still apply. It also doesn't let the role alter the gateway specification or view trace data; those require separate privileges.

Grant monitoring access

To allow a role to query gateway traces without granting full administrator access:

-- Create a monitoring role
CREATE ROLE IF NOT EXISTS AI_GATEWAY_MONITOR;

-- Grant monitoring access on the gateway
GRANT MONITOR ON AI GATEWAY SNOWFLAKE TO ROLE AI_GATEWAY_MONITOR;

The monitoring privilege allows the role to query AGENT_TRACE_TABLE('SNOWFLAKE') and view captured payloads when request_response: true is enabled. This is a data-access decision: scoping who can see prompt and completion data is as important as deciding whether to capture it.

Query gateway traces with AGENT_TRACE_TABLE

Query gateway traces

Inference activity through the gateway can be inspected with the AGENT_TRACE_TABLE function:

SELECT
  timestamp,
  RECORD:name::STRING AS span_name,
  record_attributes
FROM TABLE(AGENT_TRACE_TABLE('SNOWFLAKE'))
ORDER BY timestamp DESC
LIMIT 20;

A trace represents distributed execution, not necessarily one chat turn. Depending on the workload, AGENT_TRACE_TABLE can include retries, fallbacks, and tool spans in addition to model activity.

If the result is empty, check three things:

  1. Confirm logging: enabled: true in the gateway specification.
  2. Send a new inference request through the gateway.
  3. Confirm the querying role has monitoring access on the gateway.

Useful trace queries

List recent turns, one row per trace. elapsed_duration_ms is wall-clock time from the trace's first span start to its last span end, not the sum of span durations (spans can overlap):

-- Overview: list recent turns (one row per trace)
WITH ev AS (
  SELECT trace:"trace_id"::STRING                                       AS trace_id,
         trace:"span_id"::STRING                                        AS span_id,
         start_timestamp,
         timestamp                                                      AS end_timestamp,
         resource_attributes:"user"::STRING                             AS user_name,
         resource_attributes:"gateway_name"::STRING                     AS gateway,
         record:"name"::STRING                                          AS span_name,
         record:"status":"code"::STRING                                 AS status_code,
         record_attributes:"gen_ai.request.model"::STRING               AS model_requested,
         record_attributes:"gen_ai.response.model"::STRING              AS model_responded,
         record_attributes:"gen_ai.provider.name"::STRING               AS provider,
         record_attributes:"gen_ai.usage.input_tokens"::INT             AS input_tokens,
         record_attributes:"gen_ai.usage.output_tokens"::INT            AS output_tokens,
         record_attributes:"http.status_code"::INT                      AS http_status,
         record_attributes:"gen_ai.conversation.id"::STRING             AS conversation_id
  FROM TABLE(AGENT_TRACE_TABLE('SNOWFLAKE'))
  WHERE scope:"name"::STRING = 'aigateway/tracing'
)
SELECT trace_id,
       MIN(start_timestamp)                  AS started_at,
       MAX(end_timestamp)                    AS ended_at,
       ANY_VALUE(user_name)                  AS user_name,
       ANY_VALUE(conversation_id)            AS conversation_id,
       COUNT(*)                              AS span_count,
       SUM(input_tokens)                     AS total_input_tokens,
       SUM(output_tokens)                    AS total_output_tokens,
       DATEDIFF('millisecond', MIN(start_timestamp), MAX(end_timestamp)) AS elapsed_duration_ms,
       LISTAGG(DISTINCT model_requested, ', ') AS models_used,
       CASE WHEN SUM(CASE WHEN status_code = 'STATUS_CODE_ERROR' THEN 1 ELSE 0 END) > 0
            THEN 'ERROR' ELSE 'OK' END       AS trace_status
FROM ev
GROUP BY trace_id
ORDER BY started_at DESC
LIMIT 100;

Find which models are failing, and how often:

-- Error rate by model
SELECT record_attributes:"gen_ai.request.model"::STRING AS model,
       COUNT(*)                                         AS total_spans,
       SUM(CASE WHEN record:"status":"code"::STRING = 'STATUS_CODE_ERROR' THEN 1 ELSE 0 END) AS errors,
       ROUND(errors / NULLIF(total_spans, 0) * 100, 2) AS error_rate_pct
FROM TABLE(AGENT_TRACE_TABLE('SNOWFLAKE'))
WHERE scope:"name"::STRING = 'aigateway/tracing'
  AND record_attributes:"gen_ai.operation.name"::STRING = 'chat'
GROUP BY 1
ORDER BY total_spans DESC;

See who is calling the gateway, with which models, and how many tokens they use:

-- Per-user activity from traces
SELECT resource_attributes:"user"::STRING                       AS user_name,
       COUNT(DISTINCT trace:"trace_id"::STRING)                 AS traces,
       COUNT(*)                                                 AS spans,
       SUM(record_attributes:"gen_ai.usage.input_tokens"::INT)  AS input_tokens,
       SUM(record_attributes:"gen_ai.usage.output_tokens"::INT) AS output_tokens,
       LISTAGG(DISTINCT record_attributes:"gen_ai.request.model"::STRING, ', ') AS models_used
FROM TABLE(AGENT_TRACE_TABLE('SNOWFLAKE'))
WHERE scope:"name"::STRING = 'aigateway/tracing'
GROUP BY 1
ORDER BY spans DESC;

Accept client OTLP spans

When enable_client_telemetry: true is set in the gateway specification, clients can push their own OpenTelemetry spans to the gateway telemetry endpoint:

${BASE_URL}/telemetry/v1/traces

Use your client's standard OTLP exporter rather than hand-crafting OTLP payloads. For a quick test:

curl --fail-with-body -sS "${BASE_URL}/telemetry/v1/traces" \
  -H "Authorization: Bearer ${PAT}" \
  -H "Content-Type: application/json" \
  -d '{
    "resourceSpans": [{
      "resource": {
        "attributes": [
          {
            "key": "service.name",
            "value": {"stringValue": "my-app"}
          }
        ]
      },
      "scopeSpans": [{
        "scope": {},
        "spans": [{
          "traceId": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
          "spanId": "bbbbbbbbbbbbbbbb",
          "name": "my-custom-span",
          "kind": 1,
          "startTimeUnixNano": "1788150000000000000",
          "endTimeUnixNano": "1788150001000000000"
        }]
      }]
    }]
  }'

Client spans then appear alongside server-generated inference traces when you query AGENT_TRACE_TABLE.

Track AI Gateway credit spend with AI_GATEWAY_USAGE_HISTORY

The account usage view AI_GATEWAY_USAGE_HISTORY is the account-level record of AI credit spend across all AI Gateways in a given account.

Traces show who called what; this view shows what it cost. The query below ranks users by credit spend, which is a useful starting point for setting the per-user quotas in the next section.

-- Credits by user (who is spending the most?)
SELECT user_id,
       COUNT(*)     AS requests,
       SUM(credits) AS total_credits,
       AVG(credits) AS avg_credits_per_request
FROM SNOWFLAKE.ACCOUNT_USAGE.AI_GATEWAY_USAGE_HISTORY
GROUP BY 1
ORDER BY total_credits DESC;

Cap per-user credit spend with gateway quotas

A quota is a Snowflake metering and enforcement control that you associate with an AI Gateway resource. It governs how many Snowflake credits a user can consume through the gateway during a defined period. Usage attribution and enforcement propagate through Snowflake's metering pipeline, so there can be a delay between configuration changes and enforcement taking effect.

Configure the quota in this order: create it, add the gateway, set and verify the user scope, set limits, and enable enforcement last. Turning enforcement on before the scope settles can briefly block users who are outside the scope.

Create a quota admin role

Rather than managing quotas as ACCOUNTADMIN, create a dedicated role that owns the quota. Run the following as ACCOUNTADMIN:

USE ROLE ACCOUNTADMIN;

CREATE DATABASE IF NOT EXISTS QUOTA_DB;
CREATE SCHEMA IF NOT EXISTS QUOTA_DB.QUOTAS;

CREATE ROLE IF NOT EXISTS AI_GATEWAY_QUOTA_ADMIN;
GRANT USAGE ON DATABASE QUOTA_DB TO ROLE AI_GATEWAY_QUOTA_ADMIN;
GRANT USAGE ON SCHEMA QUOTA_DB.QUOTAS TO ROLE AI_GATEWAY_QUOTA_ADMIN;
GRANT DATABASE ROLE SNOWFLAKE.QUOTA_CREATOR TO ROLE AI_GATEWAY_QUOTA_ADMIN;
GRANT CREATE SNOWFLAKE.CORE.QUOTA ON SCHEMA QUOTA_DB.QUOTAS TO ROLE AI_GATEWAY_QUOTA_ADMIN;

-- Grant the role to the person who administers quotas
GRANT ROLE AI_GATEWAY_QUOTA_ADMIN TO USER quota_admin_1;

Create the quota and add the gateway

As the quota admin, create the quota and add the AI Gateway domain. Each account has a single gateway, named SNOWFLAKE, so the domain covers it:

USE ROLE AI_GATEWAY_QUOTA_ADMIN;

CREATE SNOWFLAKE.CORE.QUOTA IF NOT EXISTS
  QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA();

CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!ADD_SHARED_RESOURCE('AI GATEWAY');

Narrow scope with tags (optional)

A new quota applies to all users by default. To target specific groups, tag users and scope the quota by tag. Create the tag, tag users, and grant APPLYBUDGET on the tag to the quota admin role as ACCOUNTADMIN (or the tag owner):

USE ROLE ACCOUNTADMIN;

CREATE DATABASE IF NOT EXISTS TAGS_DB;
CREATE SCHEMA IF NOT EXISTS TAGS_DB.TAGS;

CREATE TAG IF NOT EXISTS TAGS_DB.TAGS.DEPT
  ALLOWED_VALUES 'engineering', 'analytics', 'data_science';

-- Required for tag-based quota scoping
GRANT APPLYBUDGET ON TAG TAGS_DB.TAGS.DEPT TO ROLE AI_GATEWAY_QUOTA_ADMIN;

-- Tag a user
ALTER USER data_scientist_1
  SET TAG TAGS_DB.TAGS.DEPT = 'data_science';

Then, as the quota admin, scope the quota to users with the selected tag value. SET_USER_TAGS takes an array of [tag_reference, value] pairs and an operator (UNION or INTERSECTION), which is required even for a single tag:

USE ROLE AI_GATEWAY_QUOTA_ADMIN;

CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!SET_USER_TAGS(
  [
    [(SELECT SYSTEM$REFERENCE('TAG', 'TAGS_DB.TAGS.DEPT', 'SESSION', 'APPLYBUDGET')), 'data_science']
  ],
  'UNION'
);

To apply the quota to all users instead:

CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!SET_USER_TAGS([], 'ALL_USERS');

The role that calls SET_USER_TAGS must have the APPLYBUDGET privilege on each tag it references.

Verify the scope

Check the scope right away with GET_QUOTA_SCOPE, which returns the monitored domains and the configured tags and operator:

CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!GET_QUOTA_SCOPE();

To see which users are resolved into the scope, call GET_USERS. It relies on tag references that can lag by up to about 2 hours, so recently tagged users might not appear yet:

CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!GET_USERS();

Set a per-user daily limit

The following sets a tiny test budget: one Snowflake credit per user per day. This isn't a recommended production default; it's a test value for verifying that enforcement works.

-- Test budget: 1 credit per user per day
CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!SET_PER_USER_LIMIT(1, 'DAILY');

Enable enforcement last

If you changed the scope, wait about 10 minutes for it to take effect, then turn on block enforcement:

-- Arguments: enable enforcement, notify users by email when they're blocked
CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!SET_BLOCK_ENFORCEMENT_ENABLED(TRUE, TRUE);

The first argument turns enforcement on or off. The second controls whether blocked users receive an email; pass FALSE to block without notifying them. When you disable enforcement, users this quota blocked are unblocked automatically within about 5–10 minutes.

Verify configuration with GET_CONFIG

Always verify that your quota settings persisted correctly:

CALL QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA!GET_CONFIG();

Expected result:

PER_USER_LIMIT_DAILY = 1 and BLOCK_ENFORCEMENT_ENABLED = true.

If PER_USER_LIMIT_DAILY is NULL or BLOCK_ENFORCEMENT_ENABLED is false, the settings didn't persist. Re-run the configuration calls and check for errors.

Allow time for quota changes to propagate

Allow approximately 5–10 minutes after creating or modifying a quota before testing enforcement. This applies to:

  • Initial quota creation
  • Changes to per-user limits
  • Changes to enforcement settings
  • Changes to tag scoping or gateway associations

Tag changes can take longer to show up in GET_USERS (up to about 2 hours), as noted above.

A quota isn't a synchronous hard stop. Requests may continue briefly while a new limit or scope propagates through Snowflake's metering pipeline.

What enforcement looks like

When a user reaches the limit, the gateway can block new requests. The response identifies quota exhaustion; the exact status, error code, and message can vary by account and release, so test enforcement in your account and treat the response you observe there as the source of truth.

Illustrative example only (your code and message may differ):

{
  "code": "391936",
  "error_code": "391936",
  "message": "Your access to this AI Gateway is blocked as you have exceeded the usage quota limit defined by your administrator."
}

Key details to validate during testing:

  • The response indicates that access is blocked because the usage quota was exceeded.
  • Whether the message tells the user when access will be restored.
  • The block can persist until the quota resets or an administrator changes the quota configuration.

Clean up test quotas

After testing, drop the quota and the test objects:

-- Drop the quota (as the quota admin)
USE ROLE AI_GATEWAY_QUOTA_ADMIN;
DROP SNOWFLAKE.CORE.QUOTA IF EXISTS
  QUOTA_DB.QUOTAS.MY_GATEWAY_QUOTA;

-- These commands drop QUOTA_DB, TAGS_DB, and the quota admin role. Run them only if you created them for this test
USE ROLE ACCOUNTADMIN;
DROP DATABASE IF EXISTS QUOTA_DB;
DROP DATABASE IF EXISTS TAGS_DB;
DROP ROLE IF EXISTS AI_GATEWAY_QUOTA_ADMIN;

Why gateway quotas instead of account budgets or a proxy

Budgets can track and alert on spend, including shared-resource spend attributed to teams or cost centers by user tags, but they don't provide the built-in per-user blocking that quotas do. A self-hosted LLM proxy can enforce per-key budgets and log every call, but that's another service to run and secure, and its logs and spend data live outside Snowflake. Gateway quotas enforce per-user limits where requests are served, and they use the same users, roles, and tags you already govern in Snowflake. The tradeoff is the propagation delay described above: quotas aren't a synchronous hard stop. You can use both: a budget for account-wide alerts, and quotas for per-user enforcement.

Production checklist for administering an AI Gateway

Before rolling out an AI Gateway to production:

  • Use a dedicated service identity and least-privileged inference role.
  • Separate gateway configuration, monitoring, and quota-creation responsibilities.
  • Run SHOW AI GATEWAYS and use the returned specification and model identifiers as the source of truth.
  • Remember that ALTER AI GATEWAY … FROM SPECIFICATION replaces the complete specification.
  • Enable logging first and leave payload capture off until data-handling requirements are approved.
  • Add request IDs and client service names to make traces easier to correlate.
  • Create a small test quota before broad rollout and allow time for propagation.

Start with logging and a test quota, then widen access

Cortex AI Gateway lets developers keep using the OpenAI and Anthropic SDKs they already know, while administrators govern that traffic with standard Snowflake controls. In this post, you restricted which models the gateway serves with a specification allowlist and used separate roles for inference and monitoring. You also queried AGENT_TRACE_TABLE for recent turns, error rates, and per-user activity, tracked credit spend with AI_GATEWAY_USAGE_HISTORY, and capped per-user consumption with quotas.

Start small: turn on logging with payload capture off, apply a test quota, and confirm that enforcement works as expected. Then open up access once your security, privacy, and cost requirements are approved. Because the gateway runs inside Snowflake, the same governance you use for your data also covers your AI applications.

Updated Nov 3, 2026

This content is provided as is, and is not maintained on an ongoing basis. It may be out of date with current Snowflake instances