Advanced, Built-In Data Governance and Data Discovery with Snowflake Horizon Catalog
Silos complicate effective governance and discovery
With the advent of generative AI and large language models (LLMs), enterprises are racing to unlock as much business value as possible from their data assets, including apps and models. Unfortunately, these data assets are often locked away in silos across multiple cloud service providers and solutions, as well as across different partner, customer and vendor ecosystems. As the number of silos increases, discovery and democratized access becomes really hard, and managing the associated security risks and costs of moving data assets between different systems with different governance models becomes extremely complex.
Simplify governance and discovery at scale with Snowflake Horizon Catalog
A core goal of Snowflake from the beginning has been to break down data silos and make all data available in a simple, consistent manner. However, making it easy to access data is not enough: It must be easy to find data, easy to collaborate and easy to govern all of an organization's data. These insights inspired us to build Snowflake Horizon Catalog.
With the Horizon Catalog, we sought to build a catalog that reflects our relentless pursuit of simplicity and uniformity. At the core of the Horizon Catalog is a unified, global repository for the Snowflake AI Data Cloud, which includes structured, semi-structured and unstructured data; models; Snowflake Notebooks; Streamlit; Snowflake Native Apps; and listings. On top of this repository, we built a suite of capabilities around the following five themes:
Security: The Horizon Catalog is built on a powerful security foundation of strong network security, robust identity management and authentication — coupled with continuous risk monitoring and protection through the Trust Center. In addition, centralized role-based access control (RBAC) across clouds helps you ensure that data is protected from the time it lands to the time it is consumed.
Compliance: The Horizon Catalog provides tools for customers to ensure that access to data is controlled and audited, that data is high-integrity and that data is used for appropriate purposes through capabilities such as auto-classification for sensitive data detection; granular authorization policies to protect data, such as through tag-based masking and dynamic data masking; and Data Quality Monitoring and Data Lineage Visualization.
Privacy: Organizations often limit data use due to privacy considerations. Horizon Catalog's advanced privacy features, such as Synthetic Data Generation, Differential Privacy Policies and Snowflake Data Clean Rooms, enable data analytics and collaboration while preserving privacy, opening up new possibilities.
Discovery: Users must be able to find the data, apps and models they need, understand them and seamlessly access them. The Horizon Catalog provides seamless discovery of content (data, apps and models) operating on data regardless of format or location.
- Collaboration: The Horizon Catalog enables secure collaboration with clients and suppliers without moving data, both within and outside the organization. Customers can also enrich and unlock their data with third-party data sets, apps and AI products from Snowflake Marketplace.
A central principle that motivated us as we built the Horizon Catalog is ease of use at scale: Even a small team should be able to secure and curate the most complex data estates through a single set of tools. The Horizon Catalog is built to be easy to use because it is:
Multi-cloud and global: Your data estate might extend across clouds and regions, which includes where your partners, customers or vendors might store data. The Horizon Catalog helps you manage your data estate with a single governance and security model that works consistently across regions and all three major clouds — AWS, Azure and GCP — as well as sovereign clouds for regulated markets.
Integrated by design: You don’t want to spend valuable time and resources building custom integrations that are often brittle and unreliable. Horizon Catalog’s governance features, discovery surfaces and request workflows are integrated by design, helping you ensure consistency of enforcement and reliability. For example, the Horizon Catalog’s Internal Marketplace is fully integrated with governance and access controls to provide a simple self-service mechanism for technical and business users alike. Meanwhile, the Horizon Catalog Trust Center provides a single, unified pane of glass that streamlines cross-cloud security monitoring to discover security risks with recommendations to address them.
Powered by Snowflake Cortex AI: The Horizon Catalog leverages Snowflake Cortex AI to automate and supercharge what used to be manual tasks. To automate data curation, the Horizon Catalog integrates with Cortex AI to intelligently generate object and column descriptions. You can use AI-powered natural language search to discover the data that you need. Finally, you can use AI to understand and analyze your data through Snowflake Copilot, Cortex Analyst and Cortex Knowledge Extensions — all with an intuitive natural language interface.
Interoperable: You want a platform that gives you the flexibility to work with open table formats to minimize lock-in and that gives you the freedom to choose the right tool for the right workload. The Horizon Catalog integrates with Apache Polaris (an open source catalog for Apache Iceberg Tables) as well as the Snowflake Open Catalog (a managed service powered by Apache Polaris) to seamlessly manage not only objects within Snowflake but also open Iceberg objects outside of Snowflake. This integration provides enterprises with unprecedented levels of choice, flexibility and control over their data by extending Horizon Catalog’s governance and discovery capabilities to Iceberg Tables created by any other compatible engine in Apache Polaris and the Snowflake Open Catalog. The Snowflake Horizon Catalog partner ecosystem further extends Horizon Catalog’s capabilities beyond Snowflake and Iceberg.
With the Horizon Catalog, data governors, stewards, security admins and chief information security officers can understand, protect and audit accounts and assets across regions and clouds while allowing data teams to discover, access and share data, services and apps globally, without ETL. The Horizon Catalog builds on Snowflake’s existing leadership in data sharing — with thousands of customers sharing data within and across organizations and procuring data or apps on Snowflake Marketplace — and in data governance and security, with the RBAC, classification, tagging and fine-grained policies that are trusted by thousands of customers. At the same time, the Horizon Catalog extends these leading, native governance and discovery capabilities to open table formats for enhanced interoperability across metadata management systems, query engines and cloud infrastructures.
Learn more
In this blog post, we introduced why we built the Horizon Catalog and how its ease of use helps you effectively manage your modern enterprise data platform. In subsequent blog posts we will provide a deeper dive into how capabilities across the five themes can help you achieve your goals.
To learn more about the Snowflake Horizon Catalog, check out these resources:
Watch demos on the Snowflake Horizon Catalog YouTube playlist
Try out some key features with this quickstart
Visit our docs
Learn more about some of Horizon Catalog’s recent announcements:
Authors
