Snowflake considers the security of customer data to be critically important, so Snowflake uses sophisticated technologies to ensure the security and safety of data, including using a hardware security module (HSM) to store cryptographic keys used for data encryption in a tamper-resistant hardware device. In this blog post, we explain how we seamlessly migrated from AWS CloudHSM Classic to AWS CloudHSM without interrupting the Snowflake service.
AWS CloudHSM versus AWS CloudHSM Classic
One of the services offered by Amazon Web Services (AWS) is AWS CloudHSM, which allows you to securely store encryption keys and perform cryptographic operations on the HSM hardware device. AWS CloudHSM is an important building block of Snowflake’s security infrastructure, ensuring the security and integrity of customer data.
AWS upgraded their CloudHSM offering and rebranded the original service as AWS CloudHSM Classic. There are notable improvements when you upgrade from AWS CloudHSM Classic to AWS CloudHSM. We believe the following are the main improvements:
- Managed service: AWS CloudHSM is a managed service while AWS CloudHSM Classic is an unmanaged service. Because AWS CloudHSM Classic is an unmanaged service, we had to (a) provision and patch the HSM devices ourselves, (b) set up and maintain the high-availability configuration ourselves, and (c) perform backups ourselves. Because AWS CloudHSM is a managed service, it handles all of this now; Snowflake’s operations team no longer has to handle all this.
- Security: AWS CloudHSM complies with FIPS 140-2 Level 3 whereas AWS CloudHSM Classic complies with FIPS 140-2 Level 2 only. FIPS 140-2 Level 3 includes additional mechanisms to detect and prevent attacks, making AWS CloudHSM more secure.
- Performance: We ran performance experiments comparing AWS CloudHSM and AWS CloudHSM Classic. We saw up to 20x performance improvements with AWS CloudHSM.
In addition, there are cost savings and better compatibility with other AWS services and user interfaces. In summary, it is beneficial to migrate from AWS CloudHSM Classic to AWS CloudHSM.
Migrating a service such as the Snowflake data warehouse from AWS CloudHSM Classic to AWS CloudHSM is not straightforward. One important property of an HSM such as AWS CloudHSM Classic is that you cannot export keys stored in the HSM. Exporting a key from AWS CloudHSM Classic and importing it into AWS CloudHSM is impossible—by design. Therefore, you must create a new master key in AWS CloudHSM and migrate all wrapped encryption material. This might be a lot of work. However, Snowflake’s encryption key hierarchy keeps the amount of migration work small and manageable.
Snowflake Encryption Key Hierarchy
Snowflake’s encryption key management uses an encryption key hierarchy. An encryption key hierarchy has several layers of keys where each layer of keys (the parent keys) encrypts the layer below (the child keys). When a key encrypts another key, security experts refer to that as “wrapping.” In other words, a parent key in a key hierarchy wraps all of its child keys.
Snowflake’s encryption key hierarchy consists of four levels of keys: the root key, account master keys (AMKs), table master keys, and file keys (Figure 1).
Figure 1: Snowflake’s encryption key hierarchy rooted in AWS CloudHSM
Each AMK corresponds to one customer account in Snowflake. Each table master key corresponds to one table in a database. Because of automatic key rotation, one customer account usually has more than one AMK; one table usually has more than one table master key.
A parent key in the key hierarchy wraps all of its child keys. The root key wraps AMKs. AMKs wrap table master keys. Table master keys are used to derive file keys. The root key at the top of the hierarchy is stored in AWS CloudHSM in Snowflake’s AWS deployments. Before migrating to AWS CloudHSM, the root key is stored in AWS CloudHSM Classic.
Migrating from AWS CloudHSM Classic to AWS CloudHSM
We use a migration process where we run AWS CloudHSM and AWS CloudHSM Classic in parallel and make use of the encryption key hierarchy. Here are the steps we take to migrate from AWS CloudHSM to AWS CloudHSM Classic:
- Connect the Snowflake data warehouse in parallel to AWS CloudHSM Classic and to AWS CloudHSM.
- In each AMK’s metadata, state which root key wraps the AMK. AMKs are wrapped either by the root key in AWS CloudHSM Classic or by the new root key in AWS CloudHSM.
- Run an upgrade function in the background that «re-wraps» AMKs. The function unwraps an AMK in AWS CloudHSM Classic and re-wraps the AMK with AWS CloudHSM. The function then transactionally replaces the re-wrapped AMK in the encryption key hierarchy along with updated metadata stating that the AMK was wrapped by the new root key.
The benefit of having an encryption key hierarchy is tremendous: you need to re-wrap only AMKs, not table master keys or file keys. Those lower-level keys remain completely untouched. Hence, the number of keys to re-wrap is comparatively small. In fact, it is orders of magnitudes smaller than if you had to re-wrap all encryption keys. In practice, this function runs in minutes, in the background, without impacting customer workloads.
Improving Data Security as a Service
Snowflake provides best-in-class key management as part of its service: a hierarchical key model rooted in AWS CloudHSM. As we have shown, Snowflake upgrades the underlying components of its service when better options become available. Upgrades are entirely transparent to customers and require no configuration, management, or downtime. That’s data security as a service. Snowflake customers benefit from Snowflake’s managed data warehouse service, the same way Snowflake now benefits from the managed AWS CloudHSM service.