Data Engineering School

Length: 20 Minutes Learn how to bring all your data together with Snowflake Openflow to build with limitless interoperability and deployment flexibility, no matter if it's structured, unstructured, batch or streaming data. Make your integration pipelines ready for AI and the future.

Length: 28 Minutes With Snowflake's integration service, Snowflake is providing enterprises with open, extensible and secure data integration capabilities that enable real-time, scalable and bi-directional data movement. Powered by Apache NiFi, this service offers customers the option to use a self-contained data plane in Snowpark Container Services, as well as the ability to run BYOC data planes in the customer’s cloud VPC. We will dive into the platform architecture in this technical session.

Length: 20 Minutes Businesses need real-time insights powered by transactional data. CDC streaming has emerged as a critical technology to ensure that operational databases seamlessly feed analytics and AI applications without disrupting core business operations. Watch this session as we unveil Snowflake's new OLTP database CDC streaming connectors, purpose-built to bridge transactional data with Snowflake in real time.

Length: 22 Minutes Discover how Snowflake revolutionizes streaming data integration from popular streaming platforms including Apache Kafka and Amazon Kinesis. Learn best practices about streaming ingestion in Snowflake, and experience seamless ingestion, powerful in-flight transformations and custom flows that turn real-time data into immediate business value.

Length: 26 Minutes Learn how to efficiently scale and manage data engineering pipelines with Snowflake's latest native capabilities for transformations and orchestration with SQL, Python, and dbt Projects on Snowflake. This session contains new product and feature overviews, best practices, and live demos.

Length: 21 Minutes Learn best practices around monitoring (alerts); backup/restore (cloning, replications, practicing failover scenarios, using finalize constraints); scheduler configurations (intuition around target lag, scheduling tick overview, target lag vs scheduling refresh); debugging best practices (walk through some debugging workflows); running thousands of DTs and much more!

Length: 12 Minutes Watch this session to learn how to successfully move data engineering workloads to Snowflake. We will share best practices and review reference architectures that can ensure success while migrating to Snowflake. Topics include: Migrating PySpark code, migrating notebooks to stored procedures, data movement: Iceberg tables/Snowflake native tables, code testing, CI/CD approaches, performance and optimization considerations, and data validation considerations and anti-patterns/considerations.

Length: 9 Minutes Modern data engineering leverages Python to build robust, scalable, end-to-end workflows. In this talk, we will cover how Snowflake offers you a flexible development environment for developing Python data pipelines, performing transformation at scale, orchestrating and deploying your pipelines at scale. Topics we’ll cover include: Ingest: Data source APIs, Snowflake file-to-read and ingest data of any format when files arrive, with sources outside Snowflake, Develop: Packaging (artifact repo), Python runtimes, IDE (Notebook, vscode), Transform: Snowpark pandas, UDFs, UDAFs, and Deploy: Tasks, Notebook scheduling.

Length: 5 Minutes Explore the cutting-edge Snowflake Tasks features that are transforming how organizations orchestrate their data pipelines. You will learn practical insights into using Tasks for effective data pipeline orchestration. Get inspired by Blue Yonder’s success story that showcases how using Tasks has led to significant cost savings and data pipeline performance optimizations. Learn how these innovative solutions can help you elevate your data game, drive business transformation, and revolutionize your approach to stay ahead in an increasingly data-driven world!

Length: 8 Minutes Data engineers often face the challenge of creating complex data transformation pipelines while balancing infrastructure overhead, debugging inefficiencies and cost control. Watch to learn best practices for building scalable and cost-efficient data transformation pipelines with built-in telemetry enhancing monitoring and debugging capabilities. By leveraging automation and managed services in Snowflake, engineers can focus on delivering insights rather than maintaining infrastructure. Join us to discover how to simplify data pipeline management, optimize costs and improve reliability in modern data ecosystems.

Length: 11 Minutes Learn how to leverage build, test and deploy your work with Snowflake's latest developer tooling. Join us for new product and feature overviews, best practices and live demos.

Length: 15 Minutes In today's data-driven landscape, visibility into workloads is essential for ensuring performance, reliability and efficiency. Snowflake Trail, Snowflake’s suite of observability tools, allows you to monitor, analyze and optimize diverse data workloads. In this session, we will explore key observability capabilities from auto-instrumentation to real-time monitoring. Through a real-world example, you'll see how Snowflake Trail enables deep observability, helping you enhance performance, troubleshoot issues and drive operational excellence.

Length: 20 Minutes Data-driven success needs fast analytics, but manual pipeline deployments slow things down. See how organizations transform data operations using software practices in Snowflake engineering. Using terminal-first methods, version-controlled SQL and automated checks, teams cut deployment time while keeping reliability. Learn to implement Git-based changes, data-aware CI/CD, and infrastructure-as-code to speed up insights.

Length: 13 Minutes Rethink how you build open, connected, and governed data lakehouses: integrate any Iceberg REST compatible catalog to Snowflake to securely read from and write to any Iceberg table with Catalog Linked Databases. Unlock insights and AI from semi-structured data with support for VARIANT data types. And enjoy enterprise-grade security with Snowflake's managed service for Apache Polaris tm, Snowflake Open Catalog.

Length: 20 minutes Apache Iceberg provides an open storage standard that can democratize your data stored in disparate data lakes by providing freedom and interoperability to use various data processing engines. Explore Snowflake’s latest advancements for data engineering on Snowflake Iceberg Tables. We’ll dive into newly launched features that enhance interoperability as well as bring Snowflake’s ease of use to your Iceberg data lakes.