PLEASE NOTE: This post was originally published in May 2023. It has been updated to include the customer’s perspective in video format.
When it comes to building customer loyalty, an ill-timed email or message can easily backfire. That’s why companies rely on customer engagement platforms (CEPs) to improve digital marketing campaigns and provide customers with a frictionless experience when ordering online and in store.
As the Director of Data Science at Paytronix, a CEP used by more than 1,800 brands in the restaurant and convenience store industries, Jesse Marshall leads a small team of data scientists and data engineers tasked with collecting, organizing, and deriving insight from data from many sources and in disparate formats.
How does data science relate to customer experience? Marshall offers this example: “Say you visited Peet’s this morning at 9 a.m. and then you received an email from them at 9:30 a.m. saying, ‘Hey, we haven’t seen you in a while. Come back in!’ You would think that Peet’s doesn’t know you at all because you were just there 30 minutes ago,” explained Marshall.
To prevent such a scenario from happening to customers, Paytronix needed data coming into Snowflake to be transformed in real time. With real-time predictive modeling, Paytronix can help its customers build and protect customer loyalty by creating tailored 1:1 messaging based on up-to-date data. In order to do this, Paytronix has to have a lot of data all in one place, and be able to glean useful insights from all of it in real time. The company relies on data science initiatives, powered by Snowpark for Snowflake and Coalesce, to make it happen.
A Moneyball approach to data science
More than 1,800 brands in the restaurant and convenience store industries, including national brands such as Panera Bread, Jimmy John’s, and Peet’s Coffee, use the Paytronix platform for their customer loyalty and rewards programs, omnichannel messaging, online ordering, and customer payments. In order to best support them all, Marshall wanted to apply the Moneyball approach to his team’s data science projects: experiment a lot, fail often, fail fast, and move on to the next one, even if that meant a 50% or 75% failure rate.
But that was impossible to do when the team relied on a mix of handwritten, custom-coded Scala and PySpark jobs for data transformation, a key part of the data preparation process. “I was frustrated with how long it took from requesting a certain table or a pipeline to it getting built,” Marshall said. “Every time you had a change, it was like starting from scratch with the pipeline again.”
Real-time predictive modeling with Snowpark by Snowflake and Coalesce
Marshall’s team started using Snowpark because it allows data to stay in Snowflake so the Paytronix platform can read models in real time, rather than once a day in batch.
The data team had spent six months attempting to manually convert PySpark scripts that were run on EMR to PySpark scripts that could be run in Snowpark, making barely any progress. “They are complex transformations, and it’s extremely hard to test the old way and to validate data,” Marshall explained.
An industry colleague recommended the Coalesce data transformation solution. Built exclusively for Snowflake, Coalesce is the only transformation platform that automates data transformations by collecting and managing metadata at the column and table level. Thanks to Coalesce’s column-aware architecture, users can understand their data at a finer level of granularity with more context on data lineage, and visibility into the evolution of data as it moves through pipelines.
Most important, the ease of use of the Coalesce interface—a flexible combination of a GUI and code-driven capabilities—allowed the two newest members of the Paytronix data team to complete 90% of the work on the platform’s most complex transformation project in less than a month.
“That’s been an outstanding success in my mind—the time saved,” Marshall said. “Being able to write your transformations, do your logic, and then run right at the node level is incredible.”
Embracing a single source of truth for all data
It wasn’t just Marshall’s team that benefited from the efficiency of Snowflake and Coalesce. Frustrated by the time it originally took the engineering team to manually build and maintain pipelines, the analysts on the larger strategy and analytics team at Paytronix had resorted to creating persistent derived tables themselves using the company’s BI tool, essentially turning it into a rogue ETL solution. Over time, this resulted in multiple sources of data being created and rampant confusion around metrics.
With Coalesce, Marshall’s team shortened the length of development time and the time it took to push requested changes, so other teams no longer create rogue ETL pipelines in their quest for fresh data. Everyone now goes back to a single source of truth and is able to see data lineage at the column level with Coalesce, democratizing data access while rebuilding trust in the data at the same time.
With time previously spent on break-fix support and manual coding now freed up, Marshall’s team is able to focus on building new features and predictive models that will power the Paytronix platform and bring value to Paytronix and its customers.
“In my mind, the really exciting part is the next phase,” Marshall said. “And that’s building new IP, new features, and new predictive models to help our clients offer the best possible experience to each and every one of their customers.”