Bright Data helps organizations tap into the power of the world’s largest database: the internet. At first, it offered public data scraping infrastructure. Now, with support from Snowflake’s Data Cloud, Bright Data delivers seamless public data sets directly to customers, faster than ever before.
“Every company has a huge database at its disposal. The internet is, arguably, the largest database out there, and it houses all the world’s information at this point. But it’s really hard and becoming increasingly harder to find and extract that data in a way that machines can process later,” explained Or Lenchner, CEO of Bright Data.
This is the core problem Bright Data, a market leading public web data collection platform , is trying to solve: how can every organization in the world access publicly available web data at scale and speed?
To deliver on its mission, Bright Data offers solutions for scraping public data, and more recently compiles its own public data sets that customers can purchase directly. But delivering these demands requires a powerful, robust data platform that can process and provide quality data on time, every time—even when trillions of requests come in each year. Here’s how Bright Data found that data platform, and empowered its customers with all new products, in Snowflake’s Data Cloud.
The power of public internet data
“We serve commercial enterprises in more than 30 industries. We also support 650 organizations through our Bright Initiative—a pro-bono program designed to help entities that use data for the public good,” explained Lenchner. “From retailers looking at how competitors price goods, to leading banks conducting market research, to civil organizations monitoring human trafficking and universities or journalists researching gender equality and climate change efforts around the world—we help all kinds of companies and organizations turn public data into meaningful actions.”
Much of this data is time sensitive. “If the data isn’t available right away, it’s almost useless for many of our customers,” said Lenchner. “In some cases, even 15 minutes can mean the difference between relevant data and outdated information.” But delivering so much data at speed isn’t easy, especially when you process trillions of requests a year for over 15,000 global customers.
Bright Data shifts from build to buy and transforms data delivery
Bright Data has always pushed the boundaries to find the ideal solutions for its business—and its customers. “It’s that tension between build and buy,” recalled Lenchner. “If you buy you can get best of breed, but we often choose to build instead of getting something perfectly tailored to our needs and use cases.”
But over time, Lenchner and the Bright Data team realized that the right data platform partner could help it refocus on doing what it does best. “Snowflake is undoubtedly the number one data platform,” said Lenchner. “That was clear to us when we looked at the PoC we ran, and when we talked with Snowflake’s experts. By letting Snowflake do what it does best, we could then make sure we stayed focused on what we do best: extracting public data for our customers.”
After making some internal changes to its infrastructure, Bright Data was able to deploy a Snowflake environment in just a few short weeks. And since then, it’s delivered new levels of performance and new products to help customers get public data faster and with less friction.
Faster data delivery and even new opportunities
Accelerating data delivery is important. But it’s not just about doing things faster. “Every business has different needs,” explained Lenchner. “Some need data within ten minutes. Some need it within days. In the past we used to have to tie up engineering resources to consider if we could meet potentially demanding SLAs. With Snowflake, we don’t need to take that time; we know we can always deliver. And that helps us go after new customers we couldn’t consider before.”
Now that the speed of data is a non-issue, Bright Data can also offer data sets to customers through a self-serve platform. “With our application running on Snowflake, customers can easily query subsets of our data through our UI,” said Lenchner. “They can find subsets they need in real time and pay for them on the spot, all without ever talking to an engineer or salesperson. It reduces the friction for them.”
And customers are clearly happy with the arrangement. Bright Data is now selling more data sets than ever before.
More flexibility for Bright Data—and its customers
With a powerful data platform comes all-new levels of flexibility and reliability for everyone in the Bright Data ecosystem.
For the business, its growing BI team now benefits from a trusted source of data. “The BI team now trusts the data much more,” said Lenchner. “It’s higher quality and there’s fewer errors than you’d see with other databases.”
And with Snowflake’s Marketplace, customers also get more flexibility in how they consume and purchase Bright Data’s products. Snowflake customers can directly purchase and consume data sets from Bright Data directly, without needing to clone or move data between Bright Data and their own systems. “It’s helped us from a discovery point of view, but it also offers a big operational benefit for our customers,” said Lenchner. “They can click three buttons and have all the data they need.”
From infrastructure to data. And now, onwards to complete insights
Over time, Bright data has transformed from delivering infrastructure businesses can use to collect public data, to offering its own data sets to customers. Snowflake has played an important role in that transformation, but the Snowflake and Bright Data partnership is just getting started.
“We’ve shifted from a technical infrastructure solution to a suite of data products that almost anyone can use,” explained Lenchner. “And now we’re going one level up the chain again to offer insights from that data.”
Bright Data recently acquired analytics provider Market Beyond to help launch its Bright Insights division. It’s already rolled out its insights solution to ecommerce customers, but is looking to offer it to other industries soon.
“From data to insights, it’s all growing,” said Lenchner. “And it’s all on Snowflake. Snowflake has just been awesome throughout. It’s an important partner for us, of course. But Snowflake is also a company we look to as an example for how we want to be in the future.”