By 2025, research firm IDC projects that global data creation will grow to 180 zettabytes annually, from 64 currently.
The reason for this increase, as steep and continuous as it is, is not mysterious. There are specific changes to the digital world that explain it. First, the growth of AI as a business differentiator has increased the creation of data. Second, the growth of edge technologies has done the same thing. By 2030, 30 billion connected devices will be in operation. Finally, there is the realization that data, which used to be considered a waste product, is now recognized as a resource. The efficient use of data can give a business a leg up on its competitors.
Put together the increase in data and the realization that data is raw material, and you can see why the ability to hire and retain data scientists has become paramount. But it’s also difficult to do, or appears so in any case.
A guide to hiring data scientists
We interviewed five data scientists and managers here at Snowflake in order to create a blueprint you can use to determine if you need data expertise and, if so, what kind.
One of the common beliefs about data scientists is that big companies are ahead of smaller ones in hiring and keeping them. But not so fast. Big companies can more easily offer higher pay and prestige. But that is quite often not enough for data scientists, especially for those who are driven by the discipline.
How can a company that has yet to achieve juggernaut status compete against the bigger fish? There are two types of strategy you can employ: offer exciting work or reduce your need to reach outside.
Questions to ask before you hire
Before you run headfirst into the hiring scrum, it’s important you make sure you know why you need what you think you need.
“You first figure out what data you have and its quality,” said Julian Forero, lead Product Marketing Manager for Snowflake’s data science workload. “For any company, one of the first things is to answer this question: Do you need a data scientist yet?”
If you do, will that data scientist be able to focus on building ML models, or will they be hung up on data management? Forero calls these “existential questions in the data analytics curve.”
Here are two questions to answer prior to enlisting the help of a data scientist:
- What is the state of your data?
- What do you need a data expert to do?
It might be helpful to comb through a checklist like this one by Harpal Singh and this one by Talia Borodin. If you are unable to answer these questions to your own satisfaction, perhaps a data architect can help you.
Once you know the answers to these two core questions, you’re ready to make your case to the data economy.
Challenge accepted!
If you lead the data science team of an enormous company, you will probably be able to offer, in addition to a recognized name for the candidate to decorate their resume, a generous payment package. But according to Forero, that is not top of the list for most data specialists.
“For a data scientist, doing interesting work is at the top of their list,” he said. That comes in three parts:
- How cutting edge is the data science you’re doing?
- How are you enabling data scientists to spend more time on building models versus just preparing data?
- Finally, how interesting to the world at large is the work that you’re asking them to do?
If you can provide more of the center of the loaf than the heel, if you can offer work they and their peers recognize as cutting edge, and if you can provide them work that has a real effect on the world, you’ve pulled up even with, or even passed, some of your bigger competitors.
The anxiety of influence
One of the things that data scientists absolutely love, according to Bhaskar Saha, Data Science Manager at Snowflake, is to see their work as influential.
That influence can take place on three levels: within their own company, within their industry, and to the world as a whole. If your company is not a behemoth, but has enough heft or importance that its competitors will feel its hip check, a data scientist may see their work influence the trajectory of business as a whole and because of that, it may affect the whole world.
“At Snowflake, we are not only competing with folks like Google, Amazon, or Microsoft to provide data solutions, but in some ways we are pushing them [forward] as well,” said Saha.
“If we set standards and they improve upon the standards, and then we improve upon them, it benefits the end customer overall. We are improving the level of competition in the marketplace.”
“One of the things about data culture that attracts really good talent is when data is not on the sidelines,” said Kristen Werner, Snowflake’s Director of Data Science and Engineering. “People who want a seat at the table are going to migrate toward companies where there’s a data-first mentality from the top down.”
Automation-augmented analysts
Some companies will undoubtedly recognize they aren’t ready to dive into the competition for these in-demand professionals, at least not yet.
Forero suggests that technology may offer other options. “There is automated machine learning. This could either help me enable someone that’s not a data scientist to do some data science, or it could help the data scientist you already have to be better or faster at their job,” he said. There is a lot of automation happening around data, and that automation may provide you with a different way around your competitors.
“Rather than go and hire a team of a hundred data scientists at a high cost to the business that you can’t retain, to whom you can’t give interesting enough projects to keep them, look inwardly at your current resources, such as the business analysts and the data analysts that have been managing data insights with BI and reporting, and have had a job that’s been data-centric for years,” suggested Paul Winsor, Head of Industry GTM, Retail and CPG, EMEA and APAC at Snowflake.
“With the kind of automated approach that current technology provides, we can lift them up and say ‘go and train yourself on these applications and move our decisioning to the predicted space and let the technology do the heavy lifting,’” he said.
Additional tips on finding a data scientist
Data is no longer an elective—it’s a requisite. However, before you go dashing headlong into the data scientist marketplace, you have to understand what your data offers you, what state it’s in, and what your needs are. Frankly, you are unlikely to buy your way to data excellence. You have to use your bean.
Data science is an area in which even a small company can compete. If your org is brave and creative, there’s a route for you to the expertise you need. If your company has employees who wish to learn and are comfortable with automation, there’s a route for you, too.
To dig into the subject and review additional strategies to compete in this data-intensive world, consider the following tactics from CIOs and other experts.
- Partner with associations or with local nonprofits and universities, where you can find and help train candidates.
- Expand your global search, not just in common places like India but lesser known areas such as eastern Europe. This approach obviously may involve opening new offices or building more remote teams.
- Hire candidates who may lack the exact profile you were expecting—look for business problem-solving as a primary quality.
- Hire or elevate a CDO to show your commitment to this discipline.
- Require that vendors and implementation partners train your internal staff as you go along.
- Conduct an internal or external data science competition to identify new candidates and raise your organization’s profile in the space.
- Improve your interviewing process, starting with an objective, quantitative review.