From Zero to Data Warehouse
September 14, 2018
Author: Jon Bock
It’s amazing what can happen when you have the courage and ability to start over.
When I joined Snowflake, the founders had just moved from the proverbial garage phase (in this case, an apartment that didn’t even have a garage) into a real office. Our Snowflake e-mail had just been configured, we had an internet connection that seemed to go down every time a big enough truck rumbled by, and barely a line of code had been written.
But what Snowflake had was a clear vision and an equally clear set of guiding principles. In my career I’ve found that the common thread among the most rewarding experiences has been that they shared the audacity to go after a big problem combined with the discipline and perseverance to go about it in the right way. The founders of Snowflake had the ambitious vision of transforming data warehousing, a huge product category that has delivered immense value historically, but has failed to keep pace with the changing world around it. Equally important, Snowflake had a set of guiding principles for how to get there:
- Think big. We did not set out to build a niche product that was one iteration improved from existing offerings because we had seen first-hand how frustrated customers had become with their fundamental limitations. We set out with the vision to reshape a multi-billion dollar market–data warehousing, and saw that to do so we had to build a completely new solution.
- Stay true to your vision. Throughout my time at Snowflake we’ve constantly faced product choices that could have compromised our vision. Most of those involved making our solution more like existing solutions, for example giving users more knobs to tweak and more they could manage. However, we’ve consistently held true to creating a service that didn’t copy existing systems just for the sake of copying them because we saw that customers needed to eliminate the pains of current offerings, not replicate those pains in a new architecture.
- Be willing to adapt. Even with a clear vision, the future has a lot of variables in it that will continue to change. We kept reexamining our design as we proceeded and as we iterated with our earliest customers, adapting and prioritizing to make certain that we were not solving a problem that existed only in the past rather than solving customers’ current and future problems.
- Partner with your customers. We started engaging customers early on, engaging them as our design partners. Even before we had a prototype, we spent time with customers learning about their data pipelines, data sources, and analytics.
Starting from nothing meant that it took a huge amount of work just to get to the database equivalent of “Hello, world”—a simple SELECT * statement. But once the core foundation was in place, it set the stage for rapid innovation. It’s a lot like constructing a skyscraper—it takes a huge amount of time just to build the foundation up to ground level, but once you get that far it literally seems to grow overnight. Since completing the foundation, we’ve been able to rapidly innovate in so many areas, delivering customer-requested functionality in weeks that would have taken years to do in an established product.
It’s been an exciting journey so far, but there’s so much more that we plan to do. We’re excited to share what we’ve built so far, and just as excited to work with you on the future innovations that you need.