Improving Model Loading for Batch Inference with the Snowflake Model Registry

Suppose you run fraud and anti-money laundering (AML) screening for a payment processor that handles tens of millions of card transactions a day. Alongside the real-time checks at each card swipe, the deeper analysis runs in Snowflake as batch inference. Every hour, a multiclass XGBoost model of about 5 GB scores the settled transactions against more than 40 fraud and risk typologies. You have logged the model in the Snowflake Model Registry and call it from SQL with SELECT MODEL(ML.REGISTRY.AML_FRAUD_CLASSIFIER)!PREDICT(...) FROM settled_transactions, and the results feed the case queues from which your compliance analysts work.

To score those rows quickly, Snowflake runs the model method with multi-process execution. When the query starts, Snowflake creates a secure sandbox on each warehouse node, loads the required packages into it, starts a parent Python process and then forks a child process for each degree of parallelism. In the common case, eight separate child processes, called Python workers, run the model method concurrently on a node. The exact count can vary, and this post uses eight as the example.

That parallelism is what keeps your hourly scoring fast, but by default model loading also runs independently in each Python worker. Before scoring its first batch of transactions, each of the eight Python workers reads, decompresses and deserializes its own copy of the 5 GB model. Model state alone then occupies roughly 40 GB on a single node before the first prediction is made.

For your fraud pipeline, and for any batch inference workload in the Snowflake Model Registry, those repeated copies add time before the first prediction and memory use that grows with every Python worker. For a model this size, they can also decide how large a warehouse the job needs.

The model_init_once option on log_model changes that equation. Enabling this option moves model loading into the parent process before the Python worker processes are created, allowing each node to load the model once per query execution and share it across Python workers. Your model and its inference methods continue to execute as before, while model memory on that node drops from roughly 40 GB toward a single 5 GB shared copy, often allowing workloads to run on a smaller warehouse at lower cost. As of publication, model_init_once is in public preview.

When you register a model in the Snowflake Model Registry, each inference method (such as predict) is backed by a Snowflake Python function. Whether you invoke that method from Python with mv.run or from SQL, Snowflake executes it in your session's virtual warehouse using Snowpark's parallel Python runtime, the same execution engine that powers Python UDFs.

Snowflake distributes a query across the nodes of a virtual warehouse, and within each node, Snowpark launches multiple Python worker processes so the function can use multiple CPU cores concurrently. When input data is skewed, Snowflake can dynamically redistribute rows across those processes, including processes on other nodes, so each Python worker receives a balanced share of the work. Automated Optimizations for Python UDFs with Snowpark's Parallel Execution describes how that redistribution works and when Snowflake applies it.

Each of those Python workers needs the model in memory before it can score its first batch. As the Model Registry documentation notes: "By default, a model that runs in a warehouse loads separately into each worker process."

Python's global interpreter lock (GIL) allows only one thread at a time to execute Python bytecode within an interpreter. Separate Python worker processes each get an independent interpreter, which allows Snowflake to execute handlers concurrently across compute resources. The Python UDTFs documentation describes this directly: when the GIL "prevents a multi-tasking approach from scaling across all CPU cores, you can execute concurrent tasks using separate worker processes, rather than threads."

Python worker process isolation is a deliberate design choice. Each process has its own heap and interpreter state, preventing one Python worker's mutable state from affecting another. As Designing Python UDFs states: "Write UDF handlers that are single-threaded. Snowflake will handle partitioning the data and scaling the UDF across the virtual warehouse compute resources."

Because each Python worker starts with its own isolated address space, model loading also runs independently in every worker. For the fraud model, each node repeats the same file read, decompression and deserialization eight times before all Python workers are ready to score rows, on top of holding eight copies in memory.

Caching inside the Python worker does not help. functools.lru_cache, cachetools and an if model is None guard all deduplicate within one process. Each Python worker starts with an empty cache, so each Python worker still loads the model once. 

To share a single in-memory model across isolated Python workers without giving up process isolation, initialization has to happen before the worker processes are forked. Linux fork() creates a child process by mapping its virtual address space to the parent's physical memory pages under a copy-on-write (CoW) scheme. Parent and child initially share the same physical pages, mapped read-only, and the kernel allocates a private copy of a page only when a process writes to it.

Trained model weights fit this arrangement naturally: They are loaded once and read repeatedly during inference. If the model is resident in memory before fork(), all Python workers initially inherit mappings to the same physical pages. The deserialization cost is paid once per node and amortized across all Python workers.

The model_init_once option builds on this copy-on-write behavior. When you log a model with options={"model_init_once": True}, the Model Registry configures each warehouse inference function to load the model once in the parent process before the fork, using the same prefork initialization mechanism that Snowpark provides for Python UDFs:

Inference is unchanged. Invoking a method through mv.run or SQL still executes in a Python worker, behind the same isolation boundary as before; only the model load moves earlier. Module-level imports and setup for that function also run in the parent process.

The scope of sharing is limited to a single query execution of that model method on one node. On a multi-node warehouse, each node runs its own parent process and loads the model once for its Python workers.

model_init_once helps most when every Python worker needs the same large, read-only model state:

• Large models: Batch prediction with large model artifacts, such as the fraud classifier above.
• Ensemble inference: Custom models that combine multiple model artifacts in a single inference method.
• Reference datasets: Custom models that load extensive reference datasets or static lookup structures into memory.
• One-time setup: Computationally heavy model startup steps intended to execute once per query per node.

The savings rest on one rule: Pages from the prefork model load stay shared only as long as no process writes to them. Setting aside the baseline memory each Python worker needs, loading a model of size M in each of W Python workers takes about W × M of memory. Loading it once before fork() keeps that close to M, as long as the model stays read-only. For the 5 GB fraud model across eight Python workers, that is roughly 5 GB instead of 40 GB.

Every run loaded a synthetic matrix reflecting the dimensions of our fraud classifier. In scenarios where each Python worker instantiated a dedicated copy, scoring millions of transactions quickly exhausted node memory. Shifting to prefork loading while restricting access to read-only operations enabled identical queries to execute smoothly on the same virtual warehouse. Conversely, introducing in-place modifications within individual Python workers triggered copy-on-write page duplication, depleting available memory once more. These empirical runs highlight how copy-on-write memory sharing preserves compute resources, rather than establishing a hard ceiling for artifact size.

Ensure your model's inference methods treat the loaded model as strictly read-only: Avoid in-place tensor modifications, object reconstruction or state duplication across Python worker processes. Python's garbage collector and reference counting can still write to object headers, which copies those pages into the Python worker. The extra memory stays small for models made mostly of contiguous native buffers such as NumPy arrays, because the data buffers themselves stay shared. Models built from many small Python objects copy more pages.

With model_init_once, everything that runs while the model loads now runs before the fork: deserializing the model, a custom model's constructor and the libraries they import. fork() copies memory but not execution context, and several things that work correctly in an un-forked process do not survive the fork in a useful state.

Mutable state: Inference code that modifies any object inherited from the parent process triggers a private copy of the affected pages. Keep the loaded model logically read-only for the lifetime of the query: model loading should load and store weights, and inference should not mutate shared objects.

Locks and mutexes: Locks held at fork time are inherited in their acquired state by every Python worker. Any Python worker that later tries to acquire such a lock will wait indefinitely. Avoid holding locks when model loading finishes.

Thread state: fork() produces one thread in each child: the thread that called fork(). Any thread pool constructed before the fork is absent in the Python workers, but the data structures those threads held, including any locks in an acquired state, are inherited. A Python worker that tries to acquire one of those locks will wait indefinitely. Some inference runtimes construct intra-operation thread pools when a session object is built. Model loading should load the weights without leaving a prebuilt session behind; each Python worker should construct its own session on first use.

File descriptors: Open file descriptors are duplicated into all Python workers, and the duplicates share a single kernel file offset. Independent reads through what each Python worker believes is its own cursor will interleave, and each Python worker receives a partial result with no error. Files opened while the model loads should be closed before loading finishes. SnowflakeFile is not supported in code that runs during the prefork load.

Network connections and database sessions: Multiple processes sharing a socket produce interleaved protocol traffic and corrupted session state. Connections should not remain open across the fork.

Module import side effects: Modules that register atexit handlers, spawn background threads or mutate process-global state during import now do so in the parent process. All Python workers inherit the result. Whether that is correct depends on what the import side effect was designed to accomplish.

In practice, model-loading code that reads an artifact, deserializes it and stores a reference to the result usually satisfies these constraints. Some third-party inference runtimes are not read-only or thread-safe after loading, so test yours before relying on the shared copy. The more elaborate a runtime's session setup, the more carefully the boundary between weight loading and session construction needs to be drawn.

The following constraints apply as of publication:

• Requires snowflake-ml-python >= 1.33.0.
• Applies to WAREHOUSE target-platform deployments only; it has no effect on Snowpark Container Services deployments.

Python worker process isolation enables the Snowflake Model Registry to run batch inference in parallel in a warehouse, but it does not have to mean loading a separate copy of the same read-only model in every Python worker. By moving the model load before fork(), model_init_once lets each node load the model once and share its memory pages across Python workers. The result can be lower memory usage, less repeated initialization and a faster path to the first prediction without changing your model or how you call it. For the hourly fraud scoring job, that means one 5 GB copy of the model on each node instead of eight.

The key is to keep the loaded model read-only and model loading fork-safe. Start with a model you already run in a warehouse, log a new version with options={"model_init_once": True}, and compare Python sandbox max memory usage and total query time in the Query Profile before and after the change. If the reduced footprint is consistent for your workload, evaluate whether it can run reliably on a smaller warehouse. See Loading large models once in a warehouse in the Model Registry documentation to get started.