Luminate Amps Entertainment Analytics with 300% Faster Data Processing and Richer Insights

When was the last time you streamed a movie, show or song? Was it yesterday at the gym when you played your favorite pump-up playlist? Or while relaxing with an episode (or three) of your favorite bingeable series? Whenever it was, whatever you were doing, that play mattered. For musicians, storytellers and the companies that produce their work, that data helps identify market share and powers chart rankings — all of which can impact artist royalties, industry awards and pop culture trends. Luminate tracks all of it.

The company provides essential information across music, film and television from hundreds of verified sources. This includes powering the longstanding Billboard music charts but also providing streaming viewership data for TV and movie rankings on Variety’s streaming originals charts. Major record labels, production companies, tech leaders, studios, networks, talent agencies, financial institutions and more rely on Luminate for independent data about the industry so their own customers can make data-driven decisions. 

Chief Data Officer Glenn Walker oversees all the activity that goes into tracking and modeling data across all of Luminate’s products. This includes the data science team responsible for data modeling and spearheading AI/ML initiatives to enhance Luminate’s offerings and extract deeper insights; the music data operations team responsible for ensuring data accuracy and the creation of the Billboard charts; the film and TV data team that collects and maintains industry-leading intelligence metadata; and the data partnerships team responsible for building and maintaining data relationships that power Luminate’s products. 

As its data landscape became more complex, Luminate was in search of solutions to make its data more accessible to customers and partners, while also expanding data sets for more robust analysis. With an on-premises legacy technology that was “woefully outdated,” Walker and his team turned to Snowflake.

Data is Luminate’s entire business — it receives more than 3.5 terabytes of data every day, in different formats, from different sources. The studios, record labels, networks and artists they serve all need that data to understand what’s happening in the competitive landscape. Walker and the team knew that the faster and more seamlessly they could deliver data to customers, partners and the entertainment industry at large, the easier it would be to maintain trust and deliver results. 

To modernize its tech stack, Luminate moved from on-premises solutions like Spark and SQL Server to the cloud. Then, the team worked to select a best-in-class cloud data platform that would allow for exceptional speed and easy, secure data sharing with customers — even when handling sensitive data requiring extra privacy. 

Snowflake hit all the right notes. Direct data sharing (and now private Marketplace listings) enable seamless collaboration between Luminate and external organizations, including some of its pre-existing clients. With Snowflake Secure Data Sharing, customers can now get data in a few minutes rather than having to wait a few days.

With Snowflake at the core of its data lake architecture, Luminate can now automatically scale storage and compute independently on a single platform. “We have years of data that we were never able to glean deeper insights from before Snowflake. Essentially, we were just building rank lists,” Walker says. Now Luminate can view the data holistically and combine it with additional data sources — providing truly unique, cross-industry insights that go beyond simple viewership and consumption numbers. 

For example, if a very popular streaming show includes a certain song in a pivotal episode, Luminate is the only company that can directly and truly measure the song’s performance in the market, model viewership for that show and then analyze the correlative effect of people consuming the song based on its inclusion in the show. 

In other words, Snowflake has unlocked enormous potential for identifying cross-marketing opportunities. “We’ll be able to correlate between people who are fans of show X who are also fans of artist Y and brand Z,” says Walker. “We couldn’t have done this before — and this breakthrough is thanks to having our unified data on Snowflake.”

Since Luminate has moved all its data onto Snowflake, it can process data 334% faster than before — which translates into accelerated delivery of data products that can help change the entertainment landscape itself. Having a centralized place for data and a unified process made it easier to develop Luminate’s groundbreaking Streaming Viewership Model (SVM), which launched in beta in 2023. 

SVM tackles the tricky challenge of tracking streaming consumption. Streaming services don’t provide transparent first-party data about show consumption, making it hard for third parties to rank top shows based on viewer behavior, let alone do any more intricate analysis. So SVP of Data Science Julian Pan and his team figured out how to combine structured data (from metadata and ACR and panel consumption providers) with unstructured data (like press releases detailing a streamer’s monthly top shows) to develop a complete picture that reliably models streaming viewership consumption. “We heavily depend on dbt on top of Snowflake, Snowpark and Snowpark ML,” Pan says. “The ability to apply all the Python code and models directly on top of data is especially beneficial.”

On the music side, Luminate is the undisputed market leader when it comes to data. Bringing in all significant first-party data from physical retailers and digital service providers, Luminate is able to provide their customers a unique view of the music industry, everything from song consumption to album sales to major label market share. The new data platform built on Snowflake will further enhance the company's market leading position. 

As a data leader on the cutting edge of entertainment analytics, Luminate has been experimenting with Snowflake’s AI features. Walker and Pan’s teams hope to use Cortex Analyst, Cortex Search and other tools to incorporate them into the tech stack — perhaps even develop an AI companion for customers to query their data with. “To be able to build directly on top of data and combine that with Snowflake’s scalability can shorten the development cycle for new AI features, which would help us to continue delivering more value to customers,” Pan says. 

Luminate plans to continue improving products for the artists, labels, studios and production companies it serves. This includes further combining the first-party data it has access to with other sources to better enrich its offerings. Walker is confident the business can do it with Snowflake as part of its entourage.

“Not many companies of Snowflake’s size have incredibly responsive people you can contact at any level of the business,” Walker says, recalling how much individualized support the Snowflake team has provided over the years with training, architectural consultation and other guidance. “Snowflake has many tech differentiators. But the main differentiator is really the human relationship.”