Every time you change a semantic view or agent spec, you want to know whether the new version is better before it serves live traffic. This guide shows you how to build a five-stage GitHub Actions pipeline that answers that question automatically.

The pipeline validates your YAML, deploys the semantic view and a candidate agent version, runs two quality gates (one for the semantic view, one for the agent), and promotes the candidate only when both pass. If either gate fails, the promotion job is skipped and the pipeline turns red.

This tutorial demonstrates gating agent-version promotion in a dedicated demo environment. The semantic view is replaced and the dashboard is deployed before evaluation. Keeping an existing agent default selected does not roll back those shared objects, so this is not an isolated production-release strategy.

Fork or clone sfc-gh-cnantasenamat/semantic-view-eval-cicd to your GitHub account, then push it to a new public repo. The repo needs at least one commit on a main branch before you can connect a git-backed Snowsight Workspace in a later step.

The repo has this layout:

Use a dedicated tutorial environment. The setup script replaces demo tables and evaluation datasets, so do not run it against resources you need to preserve. In a Snowsight worksheet, connected as ACCOUNTADMIN (or a role with equivalent privilege), run the full contents of sql/setup.sql.

The script creates:

After the script completes you should see:

Before setting up the CI/CD pipeline, open the companion notebook to explore the demo data and walk through each step the pipeline will automate, running each one manually in your own Snowflake session.

Download 01_Explore_and_Deploy.ipynb from the separate guide repository. It is not included in the application clone. In Snowsight, navigate to Projects → Notebooks and import the downloaded file:

The notebook covers:

1. Explore the three synthetic tables and key growth metrics
2. Deploy GROWTH_ANALYTICS manually using SQL
3. Query the semantic view with natural language via Cortex Analyst
4. Deploy GROWTH_AGENT manually
5. Chat with the agent, including testing boundary enforcement with an out-of-scope question
6. Run an eval manually with EXECUTE_AI_EVALUATION

The final cell bridges to the pipeline: once you have seen each step run manually, the CI/CD pipeline automates all of it on every push to main.

The pipeline authenticates to Snowflake using OIDC workload identity federation. No private keys or passwords are stored anywhere. GitHub Actions issues a short-lived token per run; Snowflake validates the token's issuer and subject claims and creates a session.

First, work out the subject claim GitHub issues for your repository and branch. For a repository under a personal GitHub account on github.com, it uses GitHub's default format:

Register it in Snowsight as ACCOUNTADMIN:

Replace <you> and <your-repo> with your GitHub username and repository name.

Only one secret is needed:

No private key or username secret is required. The OIDC token carries the identity, and the username is hardcoded in the workflow file.

The workflow sets permissions: id-token: write at the top level. This is required for the GitHub runner to request an OIDC token. Without it, snowflake-actions cannot generate the token and authentication will fail silently.

The repo includes .snowflake/config.toml with only warehouse and role defaults, without an account, user, or authenticator. It contains no credentials. The workflow injects the account, user, and authenticator via environment variables at runtime, which take precedence over anything in the file.

If your account uses a warehouse other than COMPUTE_WH, update the WAREHOUSE env var in .github/workflows/deploy.yml and the warehouse field in GROWTH_AGENT.agent.yaml.

Before triggering the pipeline, here is what each key file does.

cortex_project/cortex-project.yaml is the project manifest. It lists every artifact (semantic view, agent, eval configs) with its file path, type, and Snowflake target object name. The deploy and eval scripts read this file to know what to deploy and where.

cortex_project/GROWTH_ANALYTICS.osi.yaml is authored in the Open Semantic Interchange (OSI) format, a vendor-neutral schema maintained by the Apache Ossie project. The validate script (scripts/validate.py) uses the Apache Ossie Pydantic models to check the file against the spec before any Snowflake call is made. The deploy script converts the OSI YAML to Snowflake native format using scripts/osi_to_sv.py and deploys it with SYSTEM$CREATE_SEMANTIC_VIEW_FROM_YAML.

The semantic view has 45 metrics across three tables, 8 verified queries (VQRs) that anchor the sql_correctness eval, and a SNOWFLAKE custom_extensions block that maps each metric to its source table.

cortex_project/GROWTH_AGENT.agent.yaml defines the agent model, instructions, and tools. The single tool (growth_data) is a cortex_analyst_text_to_sql tool that points at the deployed semantic view. The deploy script either creates the agent (first run) or calls MODIFY LIVE VERSION + COMMIT (all subsequent runs), which saves a new named version on the shelf without touching the currently live default.

cortex_project/GROWTH_ANALYTICS_APP.py is a Streamlit-in-Snowflake app that queries SIGNUPS directly and renders three charts: signups by channel (bar), revenue by plan type (pie), and a channel × month heatmap. The deploy script uploads the .py file to STREAMLIT_STAGE and runs CREATE OR REPLACE STREAMLIT in the same job that deploys the semantic view and agent, so the dashboard is always in sync with the latest data model.

cortex_project/growth_analytics_sv.eval.yaml drives Cortex Analyst eval (sql_correctness) against the 8 VQRs.

cortex_project/growth_agent_eval.eval.yaml points the agent eval at the GROWTH_AGENT_EVAL dataset (10 questions with expected tool calls and answers).

evals/thresholds.yaml sets the promotion floors:

Both gates must pass before the promote job runs. These are demonstration thresholds, not production quality targets. The historical semantic-view baseline was 0.375 against the 0.35 floor. Inspect failed questions and choose thresholds appropriate for your use case rather than interpreting a green pipeline as high answer accuracy.

Any push to main that touches cortex_project/, scripts/, evals/, sql/, or the workflow file triggers the five-job pipeline. To trigger it now without changing any files, use the workflow dispatch button in the GitHub Actions tab.

After eval_sv and eval complete, open Snowsight and navigate to AI & ML → Evaluations to see the full score breakdown per question. You can also query the results directly:

After a successful run, inspect the agent versions:

Chat with the live default:

After the pipeline completes successfully, open the second companion notebook to inspect what the pipeline produced.

Download 02_Inspect_Pipeline_Results.ipynb from the separate guide repository. In Snowsight, navigate to Projects → Notebooks and import the downloaded file:

The notebook covers:

1. Inspect agent versions on the shelf (SHOW VERSIONS IN AGENT)
2. Query SV eval scores and visualize them against the promotion thresholds
3. Query agent eval scores (answer_correctness, logical_consistency, tool_selection_accuracy) and visualize
4. Compare scores across multiple pipeline runs
5. Chat with the promoted default version

This pipeline blocks bad changes automatically. You can verify this by introducing a deliberate regression.

Open cortex_project/GROWTH_AGENT.agent.yaml and replace the orchestration instruction with one that tells the agent to never use tools:

Commit and push to main. Watch the pipeline:

1. validate: passes (the YAML is syntactically valid)
2. deploy_candidate: passes (a new version is committed to the shelf)
3. eval_sv: passes (the semantic view did not change)
4. eval: fails if answer_correctness or tool_selection_accuracy drops below 0.70
5. promote: skipped when an evaluation gate fails; the broken version stays on the shelf and the previous default remains selected

Revert the commit or push back the correct instruction:

Push to main. The pipeline runs again, evals pass, and promote flips the default to the restored version.

Git-backed Snowsight Workspaces let you edit the same YAML files through a browser UI and have the CI/CD pipeline pick up your changes automatically. No local tooling required.

1. In Snowsight, navigate to Projects → Workspaces.
2. Click + Workspace → From Git repository.
3. Paste the HTTPS URL of your repo. Select or create an API integration and authenticate with OAuth or a personal access token.
4. The workspace opens with the full repo tree.

A git-backed workspace is private to your user. Collaborators each connect their own workspace to the same repo and coordinate through git branches and pull requests, not through GRANT on the workspace.

1. Open cortex_project/GROWTH_AGENT.agent.yaml or cortex_project/GROWTH_ANALYTICS.osi.yaml in the workspace editor.
2. Make a change, for example, add a new sample question to the agent or adjust a metric description in the OSI YAML.
3. In the left sidebar, click Changes, write a commit message, and click Commit.
4. Click Push to send the commit to main.
5. GitHub Actions picks up the push and the pipeline runs.

Save any notebook results, evaluation evidence, and source changes you want to retain before removing tutorial resources.

1. Disable the GitHub Actions workflow (Actions » Deploy and evaluate Cortex project » ⋯ » Disable workflow, or gh workflow disable deploy.yml) and cancel any active runs, so a push can't recreate resources while you clean up.

2. Drop the objects setup.sql created. Run this in Snowsight as ACCOUNTADMIN. Dropping the database also removes the demo tables, semantic view, agent, Streamlit app, stages, procedures and evaluation datasets inside it.

   USE ROLE ACCOUNTADMIN;
   
   DROP DATABASE IF EXISTS SV_EVAL_CICD;
   DROP USER IF EXISTS SV_EVAL_CICD_USER;
   DROP ROLE IF EXISTS SV_EVAL_CICD_ROLE;
   -- Only if COMPUTE_WH was created by setup.sql (it uses CREATE WAREHOUSE IF NOT EXISTS):
   -- DROP WAREHOUSE IF EXISTS COMPUTE_WH;
   

3. Delete the imported notebooks and the git-backed workspace in Snowsight, then remove the repository secret:

   gh secret delete SNOWFLAKE_ACCOUNT
   

You've built a five-stage eval-gated CI/CD pipeline for Snowflake Semantic Views and Cortex Agents. Pushes to the configured paths on main validate the OSI semantic view against the Apache Ossie spec, deploy a candidate agent version, run two quality gates, and promote the candidate when both pass. Before adapting this demo for production, isolate candidate semantic views and dashboards as well as agent versions.

Documentation:

Notebooks:

Additional Reading: