Snowpipe is Now Generally Available
Dec 20 | 2 Min Read
Author: Jeannie Liou | Contributing Authors: Dinesh Kulkarni
Snowflake News, Snowflake Technology
Real-time analytics are a key-enabler for making faster, data-driven decisions. After all, decisions made in haste with stale data or no data at all can backfire, leaving organizations at a competitive disadvantage.
In response to the increasing demand for real-time analytics, in late 2017 we introduced Snowpipe, our continuous, automated, and cost-effective service that loads all of your data efficiently and without any manual effort. We’re happy to share that Snowpipe is now generally available (GA) for broader use across regions on both AWS and Azure.
Making Rapid Decision-Making a Reality
Data-generating applications are emerging daily, creating large and continuous streams of both structured and semi-structured data. To support rapid decision-making, organizations need to analyze data as quickly as possible without impacting other workloads or requiring server management.
Today’s on-premises and cloud-based solutions batch load data at off-peak times, leading to delays in gaining access to valuable data. With Snowpipe, customers always control the tradeoff between cost efficiency and latency. By selecting the batch size that best meets their needs, customers use a simple but flexible mechanism for ingesting data through small batches aggregated into a single file. What’s more, newly ingested data is immediately available to all Snowflake users.
Data for the Masses
Since Snowpipe’s initial release, customers have collectively loaded over 10 million files and, in some cases, up to 20TB of data per week. In the aggregate, Snowflake customers have loaded and logged over 2.5 PB of data and 2 billion files.
For example, Snowflake customer, Capital One — one of the largest banks in the US — has been using Snowpipe over the past year to streamline loading masses of data with great success. “Our cyber division has leveraged Snowpipe to load many trillions of records with minimal overhead,” Serban Tanasa, Sr. Manager of Cybersecurity said. “And it’s all with a low cost and superb near-live streaming performance.”
Try Snowpipe today
Give Snowpipe a spin today and let us know your feedback. You can find the documentation and information on how to get started here.