Snowflake Launches Kafka Connector
Nov 20, 2019 | 2 Min Read
Author: Isaac Kunen
To reap the value of data, you need to collect it, store it, and analyze it. Apache Kafka is the system of choice for the reliable collection, transmission, and delivery of event data. Snowflake, the data platform built for the cloud, is the premier system for storing and analyzing that data.
In a modern data pipeline, Snowflake and Kafka are a natural fit.
Working in conjunction with Confluent, we’ve made that pairing even easier with the general availability of the Snowflake Connector for Kafka, which makes it simple to configure a Kafka Connect cluster to deliver JSON and Avro events into a Snowflake table.
Once events are in Snowflake, you can use tools such as Snowflake’s materialized views and data pipeline features to further process your data, integrate it with other business data, and refine it for use in your analyses.
The Kafka Connector is available from the Maven Repository and can be installed to a Kafka Connect cluster. Instances of the connector can be created through a simple JSON configuration or the Confluent Control Center. Once it is configured for a set of topics, the connector creates and manages stages, pipes, and files on your behalf to reliably ingest messages into Snowflake.
There is no direct charge for the use of the connector, which is open source and freely available under an Apache 2.0 license. The connector makes use of tables, stages, files, and pipes, which are all charged at normal rates.
For more details, refer to the Snowflake for Kafka Connector documentation.