Migrating to Snowpipe Streaming High Performance Architecture
Real-time data is no longer just about speed; it’s about massive throughput, predictable scale and data that lands ready to be queried. AI agents and intelligent applications need to continuously reason on fresh data and near real-time context, and that has inspired us to reimagine our high-performance streaming architecture. Customers are already leveraging the newest Snowpipe Streaming innovations to unlock critical use cases. While Snowpipe Streaming Classic has been the engine for low-latency ingestion directly into Snowflake tables, the next generation of our streaming architecture enables these heavier and more complex AI workloads as they grow from thousands of events per second to millions.
This post is about two things: explaining the immense value of the new Snowpipe Streaming High Performance architecture and showing you exactly how easy it is to migrate. We built this architecture to unlock up to 10-GB/s throughput per table in supported configurations, reduce end-to-end latency and simplify how you manage streaming at scale — all while maintaining a straightforward pricing structure that instills confidence even as workloads grow.
Appian, a leader in process automation, migrated to the High Performance architecture to replace a rigid, code-heavy ingestion pipeline. Previously, schema changes required complex code updates to handle DDLs manually. With Snowpipe Streaming High Performance, Appian decoupled table structure from their ingestion logic, allowing them to manage pipes via Terraform transparently to their Java application. According to Appian, their custom “Snowblower” engine now ingests nearly half a petabyte of log data per month across 27 regions, with low-latency performance in their environment.
"The migration from Snowpipe Streaming Classic to High Performance was seamless, and the simplified SDK and infrastructure have significantly improved our overall experience. We are currently handling half a petabyte of data per month in our largest environment and have yet to hit a limit on throughput."
Aboubacar Toure
Here is why you should move to Snowpipe Streaming High Performance and how to do it.
Why upgrade? The High Performance advantage
Performance at scale
Snowpipe Streaming High Performance moves the heavy lifting of file parsing and validation from the client to the server. By decoupling the ingestion path using a Pipe object, we’ve unlocked massive scalability. You can now achieve up to 10 GB/s per table and typically 5-10-second end-to-end latency in supported workloads. Thanks to better data parsing optimizations, our internal testing showed that optimized file layouts can result in up to 50% faster downstream query performance, as compared to Snowpipe Streaming Classic, depending on workload and data characteristics.
Unlocking low-latency serving use cases with Interactive Tables
Performance isn't just about ingestion speed; it's about query latency at scale. By combining Snowpipe Streaming High Performance with Snowflake Interactive Tables, you can now power high-concurrency applications directly from Snowflake. This allows you to avoid off-loading "hot" data to external operational data stores.
While standard tables excel at massive analytical scans, Interactive Tables are optimized for low-latency, high-concurrency analytics, often achieving sub-second response times in supported workloads. You can stream data directly into them with near real-time freshness. This enables you to build real-time personalization engines, customer-facing dashboards and embedded analytics applications that serve thousands of concurrent users. This unified architecture simplifies your stack by reducing the cost and complexity of maintaining separate serving layers just for speed.
Predictable, flat-rate pricing
One of the biggest friction points with Classic was the cost model tied to client connection time, a variable that could scale unpredictably if you had many idle clients. Snowpipe Streaming High Performance simplifies this. You now pay a flat rate of 0.0037 credits per uncompressed gigabyte. You pay for the data you move, not the time your clients stay connected.
Multilanguage capability and efficiency
We’ve broken the Java-only barrier. Snowpipe Streaming High Performance launches with first-class SDKs for Java and Python, along with a REST API. Under the hood, these SDKs are rebuilt on a high-performance Rust core. This doesn't just make them fast; it makes them light, and some customers report up to 30% reductions in client-side CPU and memory utilization, depending on workload and implementation.
New capabilities: Power without complexity
Snowpipe Streaming High Performance introduces features that were previously difficult to implement in Classic.
Preclustering: Sort data in flight
Streaming data often arrives unsorted, leading to costly background maintenance or slow queries until autoclustering kicks in. With Snowpipe Streaming High Performance, you can enable CLUSTER_AT_INGEST_TIME = TRUE. This sorts and clusters your data in flight before it is written to storage, meaning data lands in a highly optimized format for immediate query performance.
Complex data types
Modern streaming isn't just simple text. Snowpipe Streaming High Performance adds robust support for complex data shapes, including native support for GEOMETRY and GEOGRAPHY types. We also added support for column values up to 128 MB, which is essential for heavy JSON blobs or complex variant data that exceeded the limits of Classic.
Stateless In-Flight Transformations
Snowpipe Streaming High Performance introduces the Pipe object to enable stateless in-flight transformations directly within the ingestion path. You can now apply standard COPY command logic, such as data type casting, JSON extraction, and column reordering, before the data lands in your target table.
This approach can reduce both latency and infrastructure costs. By transforming data as it enters, you eliminate the need for intermediate staging tables and the compute resources required for post-load processing. Data lands in your final table ready for immediate analysis, removing the lag associated with secondary transformation jobs.
Native schema evolution
In Classic, a change in your source data schema often meant a broken pipeline. Snowpipe Streaming High Performance supports ENABLE_SCHEMA_EVOLUTION = TRUE natively. New columns are detected and added automatically, and NOT NULL constraints are dropped on the fly, keeping your pipeline running even as your data evolves.
The default pipe
To make migration easier, we introduced the default pipe. You don’t need to manually create a Pipe object to start using Snowpipe Streaming High Performance. If your code targets a table but points to the High Performance API, Snowflake automatically handles the routing. This means if your Classic code wrote to Table_A, your Snowpipe Streaming High Performance code can simply write to Table_A (via the default pipe) without extra setup.
Migration guide: The path to High Performance
Migrating is designed to be a straightforward, zero-data-loss process.
Step 1: Ensure prerequisites
Ensure your Snowflake account is in a supported region. As of November 2025, Snowpipe Streaming High Performance is generally available in AWS, Azure and GCP commercial deployments.
For more information on getting started, including SDK references and code examples, see the documentation.
Step 2: Update your code
Update your application dependencies to the latest version of the snowflake-streaming SDK.
There are two key API changes to note. First, the builder now requires schema context. Instead of just builder(channelName), you will use builder(channelName, dbName, schemaName). Second, the ingestion method changes from insertRow(...) to appendRow(...).
Step 3: Perform the zero-data-loss cutover
To switch from Classic to Snowpipe Streaming High Performance without dropping or duplicating records, you simply hand off the offset token.
Pause your Classic ingestion application.
Retrieve the last committed offset token from the Classic client using getLatestCommittedOffsetToken().
Initialize the Snowpipe Streaming High Performance channel using that token.
import com.snowflake.ingest.streaming.SnowflakeStreamingIngestClient;
import com.snowflake.ingest.streaming.SnowflakeStreamingIngestClientFactory;
import com.snowflake.ingest.streaming.SnowflakeStreamingIngestChannel;
import java.util.Properties;
import java.util.Map;
import java.util.UUID;
// 1. Configure your client properties
Properties props = new Properties();
props.put("user", "my_user");
props.put("private_key", "MII...");
props.put("role", "my_role");
props.put("warehouse", "my_warehouse");
// 2. Create the client
SnowflakeStreamingIngestClient client = SnowflakeStreamingIngestClientFactory.builder(
"MY_CLIENT_NAME",
"MY_DATABASE",
"MY_SCHEMA",
"MY_PIPE" // TABLENAME-STREAMING can be used by default for any table
).setProperties(props).build();
// 3. Open a channel
// We use getChannel() to unwrap the response object
SnowflakeStreamingIngestChannel channel = client.openChannel("MY_CHANNEL_NAME", "0").getChannel();
// 4. Ingest a row
Map<String, Object> row = Map.of("c1", 123, "c2", "test_data");
// appendRow adds the row to the buffer
// "001" is the offset token tracking this specific record
channel.appendRow(row, "001");
Java example showing new api flow
Step 4: Verify
Once your Snowpipe Streaming High Performance application is running, use the getChannelStatus endpoint or query your landing table to confirm that the last_committed_offset_token is advancing and that your throughput has stabilized.
Conclusion
Snowpipe Streaming High Performance is more than a version bump; it is the architecture that will support your real-time workloads for the next decade. With 10-GB/s performance, native schema evolution and flattened pricing, there has never been a better time to upgrade. By simply updating your client SDK, opening a new channel, and switching your ingestion flow, you unlock a massive leap in throughput and efficiency without minimal downtime.
But the real value of this upgrade extends far beyond ingestion metrics. In the era of Generative AI and Large Language Models, latency is paramount. AI agents and predictive models are only as smart as the data feeding them; if your data is stale, your insights are irrelevant. Snowpipe Streaming High Performance helps make your architecture AI-ready, delivering the fresh, low-latency context your models need to provide accurate, actionable results in the moment.
The future of data is streaming, and the fuel for that future is real-time. Make sure your platform is ready to handle it.
Ready to start?
Authors
