At our recent Snowday event, we announced a wave of Snowflake product innovations for easier application development, new AI and LLM capabilities, better cost management and more. If you missed the event or need a refresh of what was presented, watch any Snowday session on demand. Let’s dive into all new releases in September, October and November.
Architecture Flexibility
Iceberg Tables – public preview
While many customers value the simplicity of fully managed storage and a single, multi-language and multi-cluster compute engine to power a variety of workloads, some customers prefer to manage their own storage using open formats, which is why we’ve added support for Apache Iceberg Tables. Iceberg Tables bring the easy management and great performance of Snowflake to data stored externally in an open source format. Iceberg Tables also make it easier and cheaper to onboard without requiring upfront ingestion. To give customers flexibility for how they fit Snowflake into their data architecture, Iceberg Tables can be configured to use either Snowflake or an external service such as AWS Glue as the table’s catalog to track metadata, with an easy, one-line SQL command to convert the table’s catalog to Snowflake in a metadata-only operation. Learn more about Iceberg Tables here.
Snowflake Horizon
Seamless data governance with a new user experience in Snowsight – general availability
Users can effortlessly explore objects of interest and perform essential actions—all without SQL. Whether assigning tags, implementing masking policies, or applying row access policies, the process is easy and simple. The intuitive dashboard provides seamless navigation to desired databases and schemas, offering detailed reporting on tags and policies and the ability to take immediate action to apply them. Learn more here.
More comprehensive security with additional platform protections
To further secure data, Snowflake is improving the manageability, troubleshooting and granular controls for security admins with the following features:
- Network rules and network isolation for S3 stages (in public preview) for better management of network restrictions to Snowflake to enhance network security
- Multiple SAML IDP support, authentication policy and identifier first flow (in public preview soon) to improve authentication
- Database roles (generally available) to make access management easier while enhancing secured and flexible data sharing
Learn more here.
More granular access management and governance on shared data with database roles – general availability
Database roles are a new Snowflake object type within a database. Privileges on securable objects within the same database can be assigned to database roles. Database roles provide ease of access management for database owners within a single account, while also allowing providers to have centralized governance over shared data when serving multiple consumers. It further reduces the attack surface on shared data by limiting the scope of access granted to roles and users on the consumer account. Learn more here.
Intuitive UI for Replication and Client Redirect in Snowsight – public preview
You can now easily create and monitor replication groups, failover groups and connection objects in Snowsight without writing any SQL queries. Through the UI, you can select what objects you want to replicate, which target account you want to replicate to, and the schedule in which the replication refresh should occur. The monitoring experience also provides rich insights, making it easy to quickly identify and troubleshoot any errors, ensuring your Recovery Point Objectives are always met. Learn more here.
Cost Management
More effectively control and limit your spend with budgets – public preview
Budgets are now available across AWS, Azure, GCP and for VPS. This feature allows customers to set spending limits and receive notifications for Snowflake credit usage for either their entire Snowflake account or for a custom group of resources within an account. Learn more.
Easy access to billing usage statements – general availability
With this release, Snowflake customers can now use Snowsight to view and download their billing usage statements, starting with the July 2023 statement. The retention period will be 1 year. Learn more here.
Data Pipelines
Improved processing for streaming data with Dynamic Tables – public preview
Streaming and CDC data can be challenging to handle. That’s why we built Dynamic Tables, currently in public preview. Dynamic Tables is a new table type that drastically simplifies continuous data pipelines for transforming both batch and streaming data, declaratively. In recent releases, we’ve rolled out a number of improvements to make it even more robust for customers.
Dynamic Tables can be refreshed on creation by default, including those with a downstream lag. Sharing capabilities have been added, allowing for easier collaboration. Snowflake has also enabled support for granting privileges on all or future Dynamic Tables, including bulk grants. Learn more here.
Ingesting data is even easier with improved Snowpipe auto-ingest and schematization – general availability
Several recent releases will make streaming and batch ingestion easier for Snowflake customers.
Snowpipe now supports cross-platform auto-ingest for Snowflake accounts hosted on any supported cloud platform. Customers can trigger automated Snowpipe data loads using S3 event messages, GCS Pub/Sub event messages, and Azure Event Grid messages. Learn more.
Onboarding data is easier than ever with the latest schema detection and schema evolution capabilities, both of which are also now generally available. The schema detection feature for JSON and CSV uses the INFER_SCHEMA function to automatically detect the schema in a set of staged data files and retrieve the column definitions. The INFER_SCHEMA function applies to the following file formats: Apache Parquet, Apache Avro, ORC, JSON and CSV. Learn more.
In addition, the structure of tables in Snowflake can now evolve automatically to support the structure of new data received from the data sources. Snowflake allows adding new columns or dropping the NOT NULL constraint from columns missing in new data files. Learn more.
Observability
Improved pipeline observability with task graph – public preview
Snowflake has improved capabilities for data engineers to more easily monitor and debug data pipelines. With task graph, you can review and run history to identify critical failing tasks that prevent a graph from completing, long-running tasks, inefficient task graphs, and other monitoring and debugging cases.
Snowflake Alerts for better monitoring and debugging – general availability
A Snowflake Alert is a schema-level object that you can use to send a notification or perform an action when data in Snowflake meets certain conditions. For example, you can set up a Snowflake Alert to send a notification or perform an action when:
- The warehouse credit usage increases by a specified percentage of your current quota.
- The resource consumption for your pipelines, tasks, materialized views, etc., increases beyond a specified amount.
- Your data fails to comply with a particular business rule that you have set up.
Learn more here.
Snowpark Python Updates
Snowpark external access – public preview
Snowpark external access, now in public preview on AWS and Azure regions, provides flexibility to reach public internet endpoints from Snowpark without any additional infrastructure setup. Users can now easily connect to external network locations from their Snowpark code (UDFs/UDTFs and Stored Procedures) while maintaining high security and governance over their data. Learn more here.
Python 3.11 support in Snowpark – public preview
Users will now have the option to upgrade to Python 3.11 for creating user-defined functions and stored procedures, and be able to take advantage of the latest Python enhancements and compatible third-party packages in Snowpark. Learn more here.
Python package policies – general availability
Using Python package policies enables you to set allowlists and blocklists for third-party Python packages from Anaconda at the account level. This lets you meet stricter auditing and security requirements and gives you more fine-grained control over which packages are available or blocked in your environment. Learn more here.
Vectorized Python UDTFs – general availability
Vectorized Python UDTFs allow seamless partition-by-partition processing by operating on partitions as pandas DataFrames and returning results as pandas DataFrames or lists of pandas Series or arrays. Vectorized Python UDTFs allow for easy integration with libraries that operate on pandas DataFrames or pandas arrays. Learn more here.
Snowflake Native Apps
Snowflake Native SDK for Connectors – public preview
Building native connectivity with Snowflake just got easier! With the Snowflake Native SDK for Connectors, developers receive a set of Snowflake Native Connector templates and quickstarts that show how to build a Snowflake Native App that ingests data from an external data source into Snowflake. Just clone the template repository, modify the boilerplate code, and start creating your own Snowflake Native Connectors! Learn more here.
Cross-Cloud Auto-Fulfillment support for sharing Snowflake Native Apps – public preview
We are excited to introduce Cross-Cloud Auto-Fulfillment in this release, enabling providers to seamlessly share Snowflake Native Apps across supported regions. Detailed information can be found at Configuring Cross-Cloud Auto-Fulfillment.
Additionally, this update brings enhancements to the application package upgrade and versioning process, including cross-region DROP APPLICATION PACKAGE support and an updated APPLICATION_STATE view. Learn more at Versioning, Patching, and Upgrading an Application.
Collaboration and Monetization
Enable more users to set up Cross-Cloud Auto-Fulfillment – general availability
Users with the ACCOUNTADMIN role can delegate privileges to non-admin roles to allow other users to set up Cross-Cloud Auto-Fulfillment and share listings with other accounts in other cloud regions. Learn more here.
Snowflake On Demand customers can now purchase Snowflake Marketplace listings – general availability
We are pleased to announce that Snowflake On Demand customers can now purchase products on Snowflake Marketplace. Customers with trial accounts will still need to convert to paid accounts in order to purchase products on Snowflake Marketplace. Learn more here.
Offer subscription-based pricing plans for your products on Snowflake Marketplace – general availability
Subscription-based pricing plans are now available, allowing you to bill your customers upfront on a recurring basis for access to your product. Learn more here.
Offer a limited trial of your product on Snowflake Marketplace – public preview
As a provider on Snowflake Marketplace, you can now publish a listing that provides prospective buyers with access to a limited trial of your offering. You can configure either a sample of your offering as a free trial, or limit the time a prospect can access your full offering. Once the trial has ended, prospects can complete a form to request full access. Learn more here.
Performance Improvements
Snowflake improves performance for metadata APIs
Snowflake recently rolled out performance improvements to some of our metadata APIs (including show tables, show schemas and show databases) that reduced average query times by nearly 30% when returning large result sets. We also greatly reduced the prevalence of longer running metadata API queries by up to 5%. Learn more here.
Advanced Analytics
H3 functions for GEOGRAPHY objects – public preview
The H3 grid divides the world into hexagonal cells of equal size, simplifying data analysis across various domains. Snowflake’s native H3 functions in SQL provide an efficient way to work with the H3 grid, specifically for GEOGRAPHY objects in Snowflake. Besides enabling analytical, machine learning, and visualization use cases, they also help enhance the performance of geospatial queries. Learn more.
Structured Types – public preview
Snowflake now supports Typed Array, Typed Object, and Typed Map in Iceberg, enabling the support of nested types. It works by defining structured typed columns, and ingesting and accessing corresponding data from these columns. Learn more.
View query history in worksheets – public preview
When you view query history for a worksheet, you can now review the queries run in a Snowsight worksheet, as well as the query results. Learn more.
Snowflake Marketplace
Snowflake customers can tap into Snowflake Marketplace for access to more than 2,300 live, up-to-date, and ready-to-query third-party data sets, data services and Snowflake Native Apps all in one place (as of October 31, 2023). To learn more about providers who posted new listings in the categories economy, health, government, environment and more in September, October and November, visit Snowflake Marketplace Listings.
—-
Forward-Looking Statements
This post contains express and implied forward-looking statements, including statements regarding (i) Snowflake’s business strategy, (ii) Snowflake’s products, services, and technology offerings, including those that are under development or not generally available, (iii) market growth, trends, and competitive considerations, and (iv) the integration, interoperability, and availability of Snowflake’s products with and on third-party platforms. These forward-looking statements are subject to a number of risks, uncertainties, and assumptions, including those described under the heading “Risk Factors” and elsewhere in the Quarterly Reports on Form 10-Q and Annual Reports of Form 10-K that Snowflake files with the Securities and Exchange Commission. In light of these risks, uncertainties, and assumptions, actual results could differ materially and adversely from those anticipated or implied in the forward-looking statements. As a result, you should not rely on any forward-looking statements as predictions of future events.
© 2023 Snowflake Inc. All rights reserved. Snowflake, the Snowflake logo, and all other Snowflake product, feature, and service names mentioned herein are registered trademarks or trademarks of Snowflake Inc. in the United States and other countries. All other brand names or logos mentioned or used herein are for identification purposes only and may be the trademarks of their respective holder(s). Snowflake may not be associated with, or be sponsored or endorsed by, any such holder(s).