There are moments in life when you hear yourself saying something with disbelief. For some, it’s the “I sound like my mother” realization. That might or might not have happened to me but I’ll never admit it. Another scenario which did just happen is that I heard myself say that I was excited about migration strategies.
Yep, how a company migrates an existing business process from existing infrastructure to the Snowflake Data Cloud is really interesting to me. Why? Because it’s a complex challenge. For Snowflake, it’s moving a legacy database, data pipelines, and a consumption layer with reporting and analytics tools—and the reports and dashboards themselves. And, that requires consideration, not just of the technology but of people and processes as well. It also helped that the story that sparked my interest was told by one of my favorite customers (oops, I don’t really pick favorites).
Different perspectives, different paths
For years I’d been of the mind that the move to a new architecture shouldn’t be a “big bang.” Several years ago the Chief Data Officer of an outdoor clothing company told me that they had designed a new data architecture, but it wouldn’t go live until the data lake was fully populated. They wanted to get it right, and didn’t want the platform used until it was ready. It had already been several years.
Another CDO recounted how they had taken a more grassroots, phased approach. Rather than telling the business it was building a common framework (and making the business wait for it), the team worked closely with each of the business units to deliver needed insights but did so in a way that would fit together with a common back-end infrastructure and visualization tools. Over time, the team created a new “enterprise information store” with common dashboarding capabilities shared across all brands. The big reveal was a surprise for business stakeholders, but not a “big bang” from a migration perspective. No one had waited years for the insights needed. Value was delivered throughout the incremental approach. I was sold, becoming a true incrementalist … until recently.
Last month I participated in the Snowflake Data Cloud World Tour event in Toronto. It was a fabulous event with compelling presentations and even more captivating side conversations. During a break, I found myself with data leaders from a Canadian retailer and an insurance provider, and the topic turned to migration strategies. Yep.
I then shared a discussion I’d recently had with another customer who was struggling. They were slogging through a migration to their desired destination architecture, and planning for a big bang. It reminded me of the clothing company. Heads nodded. However, when I brought up the alternative, incremental approach, one of my breakmates pushed back. In her experience, the best practice was a big bang. Just do it, and do it quickly. In their case, the migration strategy was a wholesale “lift and shift.” No immediate re-architecture, just a move to the new infrastructure. Her organization was coming from a legacy, on-premises platform that was insufficient to meet their current needs and expensive to maintain. The big bang “lift and shift” meant they didn’t have to keep two systems running simultaneously. And, saying goodbye and good riddance to the old system was a priority.
Yet, the story wasn’t over. Although moved to the new platform, not all the existing reports and dashboards delivered value. Herein lies the challenge with a “lift and shift.” You end up dragging around some extra baggage.
What was the solution? For my breakmate, it was turning it all off. Yes, that’s right. All 80,000 migrated reports and dashboards were shut down. And, yes, as you can imagine there was an outcry, but not as loud as one might have expected. Not all reports had a loyal following. Only about 3,000 were turned back on. All’s well that ends well. And, fortunately, this particular Canadian data leader has nerves of steel.
Picking the right cloud migration strategy
The bottom line is that migration is a challenge, and different approaches must be taken into consideration. There are two decisions to make: migration strategy itself, and the “cutover” approach.
The “cutover” approach is described as either a “big bang” or a “phased approach”:
- Big bang: A migration from the legacy to the new platform in one operation, at a single point in time, with benefits delivered at that time.
- Phased approach: A platform migration broken into manageable pieces and executed over a period of time, with benefits accruing as each phase is completed.
Next, the migration strategies reflect what you want to move and into what destination. If you don’t want to make immediate changes, you’d lift the existing data and shift it into the new platform. If you’d prefer to make changes up front, you’d “fix” or redesign before making the move. And, you can use either “cutover” approach that works best. There are always pros and cons.
The figure below illustrates three common migration strategies:
Ultimately, no one size fits all. The right approach depends on the organization, the existing platform, and the company’s appetite for change. A big bang lift and shift might seem less than optimal. You migrate everything in one go, garbage and all. But my discussion in Canada taught me that sometimes you might want to go that route. The key is to deliver value as quickly as possible, whether it’s incremental value from a new platform in a phased approach or the dramatic shot that killed the costly, legacy platform.
Getting started with your migration
According to Snowflake’s Professional Services team, whatever your migration strategy, you’ll want to follow some basic steps:
- Try it: Identify an initial use case to migrate that provides business value and tests out migrating from your current platform to Snowflake.
- Scope it: Based on the learnings from the proof of concept/pilot, identify what is in and out of scope for migrating from your current platform to Snowflake.
- Plan it: Create a migration plan with costs, timelines, tasks, and resources needed to migrate from your current platform to Snowflake.
And, ensure that you have all the input you need to make your migration decisions. The questionnaire below, used in Snowflake’s Migration Readiness Assessment, is a great place to start.
|Question About the Migration
|What is driving your migration (for example, legacy platform costs)?
|Is there a specific timeline that needs to be met?
|What legacy system(s) will be migrated to Snowflake?
|Which approach to migration is preferred (Lift/Shift, Lift/Fix/Shift, complete redesign)?
|Which cutover strategy is planned (“big bang” or phased)?
|Who is doing the migration work? (in-house, partner, Snowflake, combined approach)
|Do you have a preferred systems integrator you like to work with?
|What data sources are used to populate the legacy system?
|Which tools are currently used for data ingestion?
|Which tools are currently used for reporting and analytics?
|How many different business units / users access the legacy system?