Introducing Polaris Catalog

An open source catalog for Apache Iceberg

CUSTOMER STORIES

IGS Energy Uses AI and ML to Reduce Forecasting Complexity and Improve Anomaly Detection

With Snowflake, IGS Energy uses data to solve AI/ML use cases — from more cost-effective forecasting models to more accurate anomaly detection —  to realize its mission of a sustainable future for all.

KEY RESULTS:

75%

Cost savings by moving the training of customer-level forecasting models in Databricks to a unified model in Snowflake

A female engineer wearing a hard hat and high-visibility jacket is using a laptop in an industrial setting.
isg energy logo
Industry
Manufacturing
Location
Dublin, Ohio

Empowering earth-friendly and wallet-friendly customer choices

IGS Energy is committed to building a greener future. With a mission to make reliable, affordable, clean energy available to everyone, IGS offers various sustainable energy solutions, including renewable electricity, carbon-neutral natural gas and solar. As a retail energy provider in the Midwest, IGS serves more than 1 million U.S. customers — ranging from businesses to individual households — with electricity and natural gas. 

  

“Data is very important for our mission because it helps us make our offerings more reliable and more competitive for customers,” says Dan Shah, Manager of Data Science at IGS Energy. When its legacy on-prem system could no longer keep up with the vast amount of data it was processing daily, IGS sought to implement a scalable and robust data platform. With Snowflake now at the backbone of its data infrastructure, IGS is solving a variety of business goals with AI/ML, ranging from demand forecasting to anomaly detection.

“Our anomaly detection model helps predict when rooftop solar arrays are underperforming, helping us deliver a better customer experience."

Dan Shah
Manager of Data Science, IGS Energy
Story Highlights
  • AI/ML forecasting with 75% lower costs: Moving from hundreds of thousands of individual forecasting models in Databricks to one unified model in Snowflake helped IGS achieve 75% cost savings in training — without sacrificing accuracy. 

  • Better AI/ML anomaly detection to increase customer satisfaction and reduce manual work: By more precisely pinpointing issues with customers’ solar panels, IGS saves resources while helping customers derive more value from their green investment. 

  • Faster time to value with Streamlit: Thanks to Streamlit’s ease of use and intuitive user interface, IGS’ data science team gets stakeholder input quickly, allowing the team to iterate and demonstrate value more quickly.

A scalable data platform that supports daily commodity purchasing decisions

While data is vital to IGS’ mission, the company previously relied on legacy, on-prem data infrastructure that limited the team’s ability to accomplish daily work with accuracy and efficiency. “We had a monolith of workflows dependent on one another,” Shah says. “If something broke in the chain it was hard to decouple. Additionally, it was hard to improve the process..”

These challenges were especially problematic when it came to demand forecasting — an essential part of IGS’ business that requires utmost accuracy. “We make purchasing decisions for power every day, and those decisions require a forecast,” Shah says. These transactions range from long-term decisions around buying and selling power to trades in the “day-ahead energy market,” which lets market participants buy or sell wholesale electricity one day before the operating day to avoid price volatility.

IGS’ ML-powered forecasting models require massive amounts of data, including information on weather, historical energy consumption, account details and contract dates. Since wholesale electricity prices vary by time of day, the company must forecast hourly electricity consumption for the duration of a customer’s contract, which could span many years. The data produced in this process could add up to 40 to 50 billion rows and exceed a terabyte. “If something breaks upstream while forecasting, the process doesn’t work, and we’re at risk of losing quite a bit of money,” Shah says.

IGS turned to Snowflake for a scalable, flexible solution. “The Data Cloud seemed to be the most cost-effective, quick way to store, query and use this huge volume of data,” Shah says. “Snowflake has been an important part of our vision to make data more useful, interpretable and actionable.”

A unified forecasting model for 75% cost savings

With Snowflake, Shah’s team built the “next generation” of its demand forecasting model. “Demand forecasting is typically more accurate at the most granular level,” Shah says. “But we used one predictive model per account — and with many customer accounts, this came with a lot of overhead.” IGS transitioned from training a model per customer to training a single, unified model in Snowflake — a shift that has reduced complexity and maintained high accuracy while slashing training costs by 75%.

“Previously, the process to train all these models and generate predictions took a half hour,” Shah says. “The unified model on Snowflake is super quick; we’re talking minutes to generate forecasts for hundreds of thousands of customers. This speed and simplicity will help unlock additional capabilities for the business like simulation and scenario forecasting.”

Shah sees the potential benefits of their Snowflake-powered forecasting model extending far beyond IGS. “We think this approach could work outside of the energy industry, such as retail or supply chain,” says Shah. “It could be a blueprint for any industry where forecasting is a challenge because of the number of forecasts that need to be created.”

A better anomaly detection model for happier customers — and data scientists

In addition to delivering renewable energy via the utility grid, IGS also offers rooftop solar panels. Once the panels are installed, IGS monitors their energy generation to ensure proper performance.

Previously, IGS detected any performance anomalies manually, exporting historical generation averages from its on-prem SQL server to an Excel sheet, then comparing these to production. But this manual process was time-intensive and not as accurate. “Catching these anomalies is essential to our customers having a satisfactory experience with solar panels,” Shah says. “But before Snowflake, there was a risk of false negatives, which meant we’d miss anomalies, as well as a risk of false positives, which meant we’d send a truck but find no issues.”

Shah’s team turned to Streamlit to test their idea of an ML-powered model that uses weather data and solar array specifications to predict underperformance. Thanks to Streamlit’s ease of use, Shah’s team created an initial anomaly detection application for internal stakeholders in just two weeks for testing. “Streamlit helped our data scientists feel more free to do some engineering tasks, develop models with Snowpark and build things for the business side,” Shah says. “It’s an easy tool for them to do data science work and not have to worry about working outside of their ecosystem.”

“We can more easily build predictive models and mock up data products all in the Snowflake ecosystem — whether in Streamlit or eventually in Snowpark Container Services — because the data is all there.”

Dan Shah
Manager of Data Science, IGS Energy

A data-driven future, supercharged by Snowflake

As Shah and his team look to the future, they’re continuing to evaluate new, innovative ways to drive value with data. And Snowflake is a key partner in their journey — whether it’s harnessing the value of AI and ML or generating new sales leads with Snowflake Marketplace. “The benefit of Snowflake Marketplace is huge,” Shah says. “It abstracts away ELT [extract, load and transform] of data and schema maintenance, allowing us to easily access and use a wide variety of high-quality data sets — all in our Snowflake environment, joined with our data.”  

Moving forward, IGS also plans to explore using geospatial and demographic data to identify prospective buildings that would be strong candidates for renewable energy solutions.

windmills and solar panels in front of a sunset sky

“We are big Snowflake fans here. Snowflake is a powerful, reliable platform, and we are looking forward to new machine learning capabilities, version control, notebooks and what’s coming next."

Dan Shah
Manager of Data Science, IGS Energy

Start your 30-DayFree Trial

Try Snowflake free for 30 days and experience the AI Data Cloud that helps eliminate the complexity, cost and constraints inherent with other solutions.