Running Python and Spark on Snowflake without a stored procedure
You do most of your data work in Python and Spark: pipelines, training runs, feature-engineering scripts, batch jobs on a schedule. The code lives in a project on your laptop, in Git, with a requirements.txt or pyproject.toml, an entrypoint or two, and maybe a handful of CLI arguments. That's the shape of the artifact you actually ship.
Getting that code to run inside Snowflake, right next to the data it's processing, has usually meant giving up the shape. You take your main.py and rewrite it as a Python stored procedure: hoist the logic into a function that matches a specific handler signature, hand-declare each package, wrap the whole thing in CREATE OR REPLACE PROCEDURE. The project you had (modular, imported, tested with pytest, versioned in Git) becomes a serialized stub of itself. Every change means another edit-paste-recompile cycle between your IDE and a SQL editor.
Where stored procedures fail
The wrapper problem isn't specific to Snowflake. Every managed data platform has this shape. If your compute lives close to the data and speaks SQL, and your code lives on your laptop and speaks Python, something has to bridge them. Historically the bridge has been a stored-procedure handler: a function signature the platform recognizes, with the actual code jammed inside it.
The list is the same across platforms:
You lose your project structure. A stored procedure is a single logical unit, so anything that spans multiple files gets flattened or awkwardly imported through session variables.
You lose your dependency file. Packages get redeclared in the CREATE statement, so your pyproject.toml is no longer the source of truth.
You lose local testability. The handler signature is a platform contract; the CLI your engineers used to run against is gone. And you lose the deployment story your CI/CD already knows how to do: git push to pytest to dbt run to docker build becomes git push to a hand-written SQL migration that recreates the procedure.
Teams end up doing one of three things. Some run Python outside the platform entirely, on Airflow or a container or a laptop, and reach into the warehouse over a driver. Governance suffers and data moves over the wire. Some accept the wrapper and let their main.py diverge slowly from the stored-procedure version. Some replace enough Python with dbt to shrink what needs wrapping. None of these approaches let you run the code you have as-is.
The path from stored procedures to Code Bundles
Snowflake stored procedures started as SQL scripts wrapped in a callable object. When Python became a viable language for data work, stored procedures picked up a Python handler, letting you run Python inside a warehouse instead of shipping the data out to another runtime. That was a real change. For the first time, a Python function could SELECT a table, call a UDF, and land the result in another table, all inside a single database session, without a driver call from a laptop.
Snowpark Python built on that. The session object gave Python code a native handle to Snowflake data with a DataFrame API and query pushdown. Combined with the Python stored procedure, you could write code that looked like what you'd write in a notebook and have it run next to the data.
The shape of the artifact stayed the same, though. A stored procedure is still a single logical object with a fixed handler signature and a hand-declared package list. To run a real project (a directory of .py files with a pyproject.toml, a CLI, and tests) you still had to unpack it into wrapper form, or split it across multiple stored procedures, or move the whole thing out of the warehouse and connect back over a driver.
Code Bundles are the next step in the same arc: same "run Python where the data lives" ambition, without the shape trade-off.
What a Code Bundle is
A Code Bundle is a first-class, schema-level Snowflake object that holds a project's files and runs them on Snowflake compute.
The mental model is small. You have your files plus a single code_bundle.yml, and that one file declares how the project runs: language, compute target, runtime version, dependencies, secrets, environment variables. Snowflake reads the spec, provisions the compute, injects a Snowpark session so your code has credential-free access to your data, and runs your entrypoint. Your main.py stays main.py. The dependency file stays where it is. You run the code by reference.
Because a Code Bundle is a real Snowflake object, everything you already do with Snowflake objects works: grants and role-based access, scheduling via Tasks, replication, catalog visibility, orchestration from the Snowflake CLI in a CI/CD pipeline. Full run history, logs, and status are tracked for you.

See it run end to end
Say you have a nightly aggregation job. You develop it in whatever IDE you like (Snowflake Workspaces, VS Code, JupyterLab), then describe how it runs in a code_bundle.yml:
bundle: type: custom compute_type: warehouse language: python compute_options: runtime_version: '3.11' properties: requirements_file: requirements.txt
Create the bundle from your project files on a stage, then run it by naming the entrypoint:
CREATE CODE BUNDLE sales_pipeline FROM @my_stage/project_files; EXECUTE CODE BUNDLE sales_pipeline ENTRYPOINT = 'jobs/aggregate.py';
The Snowflake CLI has an equivalent shape. It packages your project directory and creates the bundle in one shot:
snow bundle create sales_pipeline \ --source . --exclude ".venv/**" \ --database MY_DB --schema MY_SCHEMA snow bundle execute sales_pipeline --entrypoint jobs/aggregate.py
Put it on a schedule with a Task, passing arguments the same way you would on the command line:
CREATE OR REPLACE TASK nightly_agg WAREHOUSE = my_wh SCHEDULE = 'USING CRON 0 2 * * * UTC' AS EXECUTE CODE BUNDLE sales_pipeline ENTRYPOINT = 'jobs/aggregate.py' ARGUMENTS = ('--source-table', 'RAW_SALES', '--days-back', '7'); ALTER TASK nightly_agg RESUME;
That's the whole loop, from developing the job to describing it to deploying and scheduling it. The handler signature and the copy-paste between your IDE and a SQL editor are gone.
Bundles reach external APIs by attaching secrets and external access integrations to the spec.
Choose where it runs
A Code Bundle can run in three places. Set the target in code_bundle.yml and Snowflake handles the rest.
Run on a warehouse
compute_type: warehouse. Your Python runs on the same virtual warehouses you already use for SQL. You skip the container image and the compute pool ceremony entirely. This is often the fastest path to production for data-processing and ETL jobs where the heavy lifting happens in SQL and UDFs and the Python is orchestration around them.
Run on a compute pool
compute_type: compute_pool. Your job runs in Snowflake's container environment, backed by Snowpark Container Services. You get a full Python runtime with the flexibility to install packages from any source, pick CPU or GPU machine types, and run heavy computation directly in the Python process. Popular ML frameworks come preinstalled, and distributed data-loading and training APIs use every core and GPU on the node. This is the target for model training and fine-tuning, hyperparameter search, and batch inference at scale.
Compute-pool execution is the capability that shipped as Notebook Projects. It runs exactly as it always has (more on that below).
Submit a Spark job (new)
type: spark. Snowflake runs Spark applications in Scala, Java, or Python as native batch jobs on Snowflake compute. Spark support is built on Snowpark Connect, which runs the Spark DataFrame API directly on the Snowflake engine, so existing PySpark code moves over with minimal changes and you don't stand up or tune a Spark cluster.
Query pushdown into Snowflake still applies. The governance and access controls that apply to every other Snowflake operation apply to Spark jobs too. Submit a job through SQL or the REST API, then track, monitor, and cancel it by its job ID.
A Spark code_bundle.yml looks like this:
bundle: type: spark compute_type: warehouse language: python compute_options: runtime_version: '1.42.0'
runtime_version pins the Snowpark Connect for Spark client version; omit it to run on the latest available version.
And submission is the same shape as a warehouse job:
CREATE CODE BUNDLE funnel_analysis FROM @my_stage/spark_project; EXECUTE CODE BUNDLE funnel_analysis ENTRYPOINT = 'spark_analytics.py' ARGUMENTS = ('--events-table', 'RAW_EVENTS', '--funnel-output', 'FUNNEL_METRICS', '--pairs-output', 'PRODUCT_PAIRS');
For orchestration from Airflow, CI/CD pipelines, or a custom submitter, submit the Spark job through the REST API. It submits jobs from a stage path rather than from a stored bundle, which suits callers that aren't running SQL.
Generate a bearer token from key-pair authentication, then POST the job spec:
SNOWFLAKE_TOKEN=$(snow connection generate-jwt) curl -X POST \ "https://<account>.snowflakecomputing.com/api/v2/code-bundle-executions?asyncExec=true" \ -H "Authorization: Bearer ${SNOWFLAKE_TOKEN}" \ -H "X-Snowflake-Authorization-Token-Type: KEYPAIR_JWT" \ -H "Content-Type: application/json" \ -H "X-Snowflake-Role: MY_ROLE" \ -H "X-Snowflake-Warehouse: MY_WH" \ -H "X-Snowflake-Database: MY_DB" \ -H "X-Snowflake-Schema: MY_SCHEMA" \ -d '{ "from_location": "@MY_STAGE/spark_project", "entrypoint": "spark_analytics.py", "arguments": ["--events-table", "RAW_EVENTS", "--funnel-output", "FUNNEL_METRICS", "--pairs-output", "PRODUCT_PAIRS"], "specification": { "bundle": { "type": "spark", "compute_type": "warehouse", "language": "python", "compute_options": { "runtime_version": "1.42.0" } } } }'
An async submission returns 202 Accepted with a job_id you can poll for status:
{ "code": "392604", "message": "Request execution in progress.", "job_id": "01c51743-c819-4261-0000-5349586311aa" }
Check status by GETting /api/v2/code-bundle-executions/<job_id> with the same auth headers. This is the pattern Spark engineers already use on other platforms: POST a job definition, get an ID back, poll for status.
When to reach for a Code Bundle
Snowflake has several ways to run logic against your data. Code Bundles fill the gap between them.
| Reach for a... | When... |
|---|---|
| Code Bundle | You have an imperative Python or Spark job (an ETL script, a model-training run, a batch-inference pipeline) with a defined start, a defined end, and a success-or-failure result you want to deploy as-is and schedule. |
| Stored procedure or UDF | The logic is small and SQL-adjacent, and you want to call it inline from a query. |
| dbt Project | The work is a set of SQL transformations you model, test, and materialize with dbt. |
| Service | The workload stays running and serves requests, like an app or an API that keeps handling traffic between deploys. |
Most teams end up using several of these together. dbt Projects handle the SQL model layer, Code Bundles handle the Python and Spark jobs that run beside them, and stored procedures and UDFs stay embedded in the queries that call them.
Operate it in production
Beyond the create/execute/schedule loop, three capabilities in this release let you match execution to the workload as it evolves: per-run spec overrides, asynchronous submission, and run history in the catalog.
Override the spec at runtime
A code_bundle.yml sets the default, but a single project often contains files that need different runtimes. Override the stored spec inline at execution time with WITH SPECIFICATION:
EXECUTE CODE BUNDLE FROM '@my_stage/spark_project' ENTRYPOINT = 'spark_analytics.py' ARGUMENTS = ('--events-table', 'RAW_EVENTS', '--funnel-output', 'FUNNEL_METRICS', '--pairs-output', 'PRODUCT_PAIRS') WITH SPECIFICATION $$ bundle: type: spark compute_type: warehouse language: python compute_options: runtime_version: '1.42.0' $$;
The override replaces the stored spec entirely. Every required field has to be repeated, even if it hasn't changed. One bundle can hold a Python sessionizer and a Spark analytics job side by side, each executed with its own spec.
Submit asynchronously
The REST API and the Snowflake CLI both support asynchronous submission. From the CLI:
snow bundle execute my_pipeline \ --entrypoint jobs/aggregate.py \ --async \ --database MY_DB --schema MY_SCHEMA --warehouse MY_WH
An async submission returns a job ID immediately. Poll status by ID:
snow bundle status <job_id>
Review run history
Every execution shows up in SNOWFLAKE.INFORMATION_SCHEMA.CODE_BUNDLE_HISTORY. Query it like any other table function:
SELECT ENTRYPOINT, STATUS, BUNDLE_TYPE, COMPUTE_TYPE, START_TIME, END_TIME, ERROR_MESSAGE FROM TABLE(SNOWFLAKE.INFORMATION_SCHEMA.CODE_BUNDLE_HISTORY( BUNDLE_NAME => 'MY_PIPELINE', RESULT_LIMIT => 20 )) ORDER BY START_TIME DESC;
Failures land in ERROR_MESSAGE alongside the query ID, so a run that failed in a Task at 2am is diagnosable the next morning without SSHing into anything.
What this means for Notebook Projects
Code Bundles are Notebook Projects renamed. The name reflects what the feature has grown into: a home for any packaged code, notebooks included. If you're using Notebook Projects today, there's nothing to migrate. Your existing objects, your CREATE and EXECUTE NOTEBOOK PROJECT statements, and your scheduled tasks all keep working. Those objects show up as Code Bundles in the catalog.
Resources
- Official Quickstart: https://www.snowflake.com/en/developers/guides/run-python-and-spark-jobs-with-snowflake-code-bundles/
- Docs: https://docs.snowflake.com/en/developer-guide/code-bundles/code-bundles
- Spark on Code Bundles: https://docs.snowflake.com/en/developer-guide/code-bundles/spark-code-bundles
- Snowpark Connect: https://docs.snowflake.com/en/developer-guide/snowpark-connect/snowpark-connect-overview
- Companion repo: https://github.com/Snowflake-Labs/sfguide-run-python-and-spark-jobs-with-code-bundles
This content is provided as is, and is not maintained on an ongoing basis. It may be out of date with current Snowflake instances