Challenges and New Opportunities in Data Analytics
Author: Jon Bock
Fall is conference season in the industry, and this fall there has been no shortage of discussions and insights about data analytics at events both big and small. The Cloud Analytics City Tour has been a highlight here at Snowflake, but we’ve also seen the analytics conversation front and center at big conferences like Dreamforce.
The Challenges of Data Analytics
Our Cloud Analytics City Tour, now entering its home stretch, has brought together a diverse set of attendees, with small entrepreneurs sharing the room with people from some of the most established companies around. That diverse audience and the thought leaders who participated as speakers have provided some great discussion and insights.
For one, it’s clear that data analytics in the cloud has quickly become a topic of mainstream interest to organizations of all stripes and sizes. In fact, the conversation has moved on from “should we consider data analytics in the cloud at all” to “how do we figure out what to do in the cloud and how”?
That shift was reflected in some of the key themes and insights we’ve been hearing on the City Tour. Among those themes and insights:
- The challenges are more than just technology. We heard repeatedly that one of the biggest challenges in cloud analytics is getting organizational buy-in. Even though acceptance of cloud has grown, getting people to do things differently still takes a lot of work.
- Data integration and analytics now need to be a continuous process. The batch, scheduled approach to making updated data and analytics available no longer meets the needs people have today. Continuous data integration is becoming vital as organizations look to drive agile, data-driven decision-making throughout their organizations.
- Finding great analytics people remains hard. The “people issue” – finding the right talent for analyzing data, is now even more urgent. However, it’s still hard to solve even as a greater number of people become data savvy.
- Data quality still matters. While the technology to manage large and disparate sets of data is far more accessible in part because of the cloud, the quality of the data is still a challenge – how do you verify and normalize the data as quickly as your system can deliver and parse it?
Bringing Data Analytics to All
The importance of data analytics was also front and center at other conferences. At Dreamforce, the former Salesforce CRM conference that has now evolved into a much broader event encompassing wide-ranging business and technical topics, data-driven decision making for competitive advantage was a key theme. However, the conversation at Dreamforce has evolved from last year’s spotlight on the importance of using “big data” to a focus this year on how the nature of this data is changing, and on how to practically use more of the new types of data in everyday decision-making without being overwhelmed by its complexity.
What was most interesting about this discussion was that there were clearly two camps: increasingly sophisticated organizations with access to the skills and resources to be able to apply the latest data analytics approaches, and organizations that do not have in place or within reach the skills and resources to enable data-driven decision-making for greater insight. Those deep-pocketed enterprises who are rebuilding their entire infrastructures with the help of consultants like Accenture are leap-frogging into new productive use cases and revolutionary advances in deep learning.
The result is that well-funded start-ups who can attract highly skilled resources (and who can start from scratch) and those deep-pocketed enterprises who are rebuilding their entire infrastructures with the help of consultants like Accenture threaten to leapfrog the millions of organizations stuck in the middle who may know what they want to do with data and analytics, but don’t know how to get there. To add to the complexity, not only the technical infrastructure but the mindset within the organization and across departments needs to change.
For organizations across that spectrum, new solutions have emerged. Salesforce’s announcement of Einstein, a data analysis solution for data in Salesforce systems, is one example. But even more importantly, cloud analytics and systems designed to support it are making analytics accessible to more than just the well-resourced 1% of organizations.
As we have learned from the nimble companies that have gone from startup to billion-dollar unicorn in the last five years, thinking and operating in the cloud is the ultimate enabler. For more established companies hindered by legacy systems, changing the technology is now the easy part with solutions such as Snowflake available. But the rewards in overcoming these cultural and process barriers are invaluable to any organization that doesn’t want to be left behind in this next wave data revolution.
To connect with like-minded revolutionaries and learn more about how to move your organization’s data sophistication to the next level, join us at one of our next Data Analytics forums, including this week’s event in San Francisco as well as upcoming events in Chicago and Los Angeles. The best learning happens in person, and we hope you have or will take advantage of our Cloud Analytics City Tour as a great forum for intelligent discussions and meaningful insight.