Are all databases in the cloud, built for the cloud?
September 14, 2018
Author: Saqib Mustafa
“Crap on-premises, move it to the cloud; it is still crap, just in the cloud”
Traveling around the country for the CloudAnalyticsCityTour.com, we see a fundamental shift in the world of analytics and the impact cloud has had on analytics. As more data sources come online (IoT, Mobile, Applications), the cloud economies of scale allow companies to bring in larger data sets and analyze them effectively. Increasingly, all major legacy data warehouse and big data platforms are moving to the cloud, to make sure they can accommodate customer workloads. However, listening to data management and analytics users in NYC and Boston, there is a genuine sense of frustration that existing platforms, whether on-premises or moved to the cloud, are not serving their purposes. Even in the cloud, these legacy platforms still require the customer to either sign up, typically in advance, for excess capacity to cater for peak performance or concurrency requirements, or they must risk sacrificing performance and business value to offset costs. Plus, none of the platforms offer the simplicity of a true, modern, built for the cloud SaaS-based solution. So you actually end up needing the same expensive resources that you did on-premises and just connecting to them in the cloud. Hence the quote: “Crap on-premises, move it to the cloud; it is still crap, just in the cloud.”
Additionally, the picture has only gotten more crowded, and confusing, with the introduction of open source platforms. As David Menninger (@dmenninger) puts it: “Hadoop is like a free puppy”. Whether in the cloud or on-premises, these open source “free puppies” need all sorts of care and feeding. They end up using even more resources than more traditional alternatives. According to a recent study by Ventana Research (link, registration required), only 1 in 6 organizations have Hadoop skills while 6 out of 10 have SQL skills. As Michal Klos, Engineering Director, Data Platform, Localytics hinted in a recent talk at the Boston edition of the CloudAnalyticsCityTour.com: they developed a data warehouse on their own in Presto for 1 year and eventually gave away that puppy because of all the additional effort it would take to get any real business value.
Then what is the alternative?
The true alternative has to be a modern cloud data warehouse that is simple to manage, offers performance and is able to meet your concurrency needs, on demand. It has to be built for the cloud, to make sure you can bring in all your diverse data and effectively scale when you need to scale, rather than requiring you to overbook capacity upfront. Around the country, companies are begging for a solution that helps them better analyze data and speed their data pipeline, rather than being stuck managing the entire environment in yet another data center. Plus, it has to fit inside the ecosystem of modern ETL/ELT and BI partners that help with moving the data from source to analysts.
Want to hear how your peers are benefitting from the shift to cloud analytics? See how data driven companies, like Snag-a-job, are able to analyze 1.5 million impressions a day and, at the same time, 18 months of historical records to understand where their customers can improve their product. Come join us for the CloudAnalyticsCityTour.com when it visits DC (October 4th) and Philly (October 6th) to hear thought leaders from AWS, Looker, Informatica, Tableau, Snowflake, and the likes of David Menninger, about how legacy platforms and free puppies keep giving us more headaches. Or join us at the TDWI San Diego conference where IAC Publishing Labs (Ask.com, About.com Investopedia.com) will talk about how they won the TDWI Best Practice award for Emerging Technologies and Methods by using Snowflake, the data warehouse built for the cloud, to turn the BI group from a cost center to a valued part of the organization. You can also visit our booth at the TDWI exhibitors hall.
As always, keep an eye on this blog site and our Snowflake-related Twitter feeds (@SnowflakeDB, @drkoalz, and @kentgraziano) for more “Cool Things About Snowflake”, and for updates on all the latest action and activities here at Snowflake Computing.