In August 2020, Snowflake announced several new features, all in preview, that make its cloud data platform easier to use, more powerful for sharing data, and more usable via Snowflake-supported languages. These innovations mean you can bring more workloads, more users, and more data to Snowflake, helping your organization solve your most demanding analytics challenges.
Multi-Cloud, Cross-Cloud, and Pattern-Matching Support in Snowpipe
First introduced in 2017, Snowpipe is Snowflake’s continuous data-ingestion service built for near real-time analytics. When it became generally available, customers were using Snowpipe to load petabytes of data into Snowflake, and its adoption has only increased since then. In August, Snowflake previewed the following features to make Snowpipe more robust:
- Automated continuous data loading for Google Cloud Storage stages: Aligning with the spirit of near-zero maintenance as a core tenet of Snowflake, this feature enables automated continuous data loads for Google Cloud Storage (GCS) without requiring administrative intervention. Using event notifications, Snowpipe automatically loads new data from cloud storage into Snowflake, making it more readily available to analytics teams for faster time to insight.
- Continuous data loading to Snowflake on AWS: Many Snowflake customers require a multi-cloud approach, whether it’s to avoid vendor lock-in or to accommodate different platforms resulting from mergers and acquisitions. Often organizations look for ways to use Snowflake’s cloud data platform across the leading cloud providers Snowflake supports, namely Amazon, Google, and Microsoft. Since data regularly is in native cloud storage offered by these providers, Snowflake now enables users to get data into Snowflake on AWS regardless of the cloud provider that stores the data. Now, users can automate continuous data loads into Snowflake accounts hosted on AWS from data files in either GCS or Microsoft Azure.
- Support for pattern matching: Event-driven architectures are commonplace today. As certain types of events occur (such as fraud detection or real-time customer service requests), using data to take the right action is critical. Pattern matching is ideal to trigger automated Snowpipe data loads based on event notifications. Depending on the event, data can be routed in different ways to different parts of the organization to make sure they have the most up-to-date data for the workload they want to optimize.
Enhancements to Go Snowflake Driver
Snowflake users often use the Go programming language when they work on Snowflake’s platform. In August, Snowflake further enhanced its Go support for a broader range of workloads and data by offering the following features in preview.
- Go Snowflake Driver 1.3.7 support for multi-statement queries: The Go Snowflake Driver Version 1.3.7 and later enables users to submit a string containing multiple SQL statements to be executed in a single call to the Go driver. Developers now can issue a single SQL statement that returns multiple result sets.This greatly simplifies the developer experience, reducing the amount of code that is needed. (Preview)
- Go Snowflake Driver 1.3.4 support for the Arrow data transfer format: Snowflake first introduced support for the Apache Arrow format in February 2020. The Arrow format provides high performance and supports conversion between a larger number of combinations of Golang and SQL data types than the JSON format provides.
Secure Data Sharing for External Tables
For many organizations with a data lake, a lot of data is in files located in cloud storage such as Amazon S3, GCS, or Microsoft Azure. Unfortunately, sharing this data with other users or third-party stakeholders requires making copies or sharing data using FTP, which increases maintenance overhead and potentially introduces governance problems. But this secure data sharing enhancement, now in preview, enables users to share data directly from their data lake. When using Snowflake, data in the data lake is shared securely and seamlessly, without the additional overhead required by traditional sharing approaches.
Snowsight Usability Updates
Snowsight, the Snowflake user interface, is where analysts can build queries and interact with Snowflake’s cloud data platform. Snowflake continually works to extend Snowsight functionality to include the capabilities required for all Snowflake-supported workloads.
- User refresh of schema metadata: Previously in Snowsight, the schema metadata was refreshed automatically every 24 hours. When a new table or view was created in a database, users with the right roles and permissions could query the object; however, the schema browser would not list the new object immediately, and the new object could not be searched, meaning important updates or new data could be missed. Now users have the option to refresh the schema at a time of their choosing so they have more up-to-date information about important database changes without having to wait for them to get populated.
- Execution of multiple SQL statements sequentially in a single worksheet: Analysts use worksheets within Snowsight to build and run SQL statements on Snowflake. Previously, analysts had to either build complex scheduling routines or execute SQL statements manually. With this enhancement, however, users can execute multiple SQL statements sequentially within Snowsight. The result is greater productivity and better performance.
For a complete list of features, enhancements, and changes introduced in August, visit the Snowflake documentation Release Notes page here.
This post contains express and implied forward-looking statements, including those relating to features and functionality that are in preview, as well as the benefits of those features and functionality, which are not historical facts and are instead based on our current expectations, estimates and beliefs. The accuracy of such statements involves risks and uncertainties and depends upon future events, including those that may be beyond our control, and actual results may differ materially and adversely from those anticipated or implied by such statements. Any forward-looking statements included in this post speak only as of the date hereof and, except as required by law, we assume no obligation to update or otherwise revise any of such forward-looking statements to reflect subsequent events or circumstances.