Have you ever taken a certification exam and wondered how it was made? Who wrote the questions, who scores the exam, who decided on the content? Valid, reliable, and fair professional certification exams follow international psychometric guidelines, and a fascinating cyclical process:

It all starts with demand from the industry for qualified individuals who can demonstrate competency with a defined set of skills. To help us identify this skill set, we bring together a diverse set of professionals who are knowledgeable and working in the industry. These are our certification subject matter experts (SMEs). From there we follow these next steps:

Step 1 - Job task analysis: SMEs provide their opinions about all aspects of the job role: who is the target audience, what tools and environments is the target audience familiar with, what is in-scope and out-of-scope for the role, and, most important, what knowledge, skills, and abilities demonstrate competency? The result is an outline for our exam.

Step 2 - Question writing: SMEs are trained in the science of writing great questions. During the training the SMEs review sample questions that align with best practices as well as sample questions that do not align with best practices. Once the SME feels comfortable with the question-creation guidelines, they receive an independent writing assignment. For each question they write, the SME must provide a rationale and documentation support as validation. 

Step 3 - Question review and approval: The questions are presented to a larger group of SMEs for review and approval. The group will see each question first with no key in order to ensure the experience of the test taker is simulated, and then they send the question back if revisions are needed. The SME group must approve each question before it will appear on an exam to collect statistical data before the question is scored live on an exam. 

Step 4 - Beta exam: A beta exam allows Snowflake the ability to pretest a new exam before issuing scores to ensure each question meets our statistical thresholds. Snowflake beta exams are longer than our live exams and allow candidates to provide feedback before a score is issued. 

Step 5 - Standard setting: The exam passing score is set based on the beta exam’s statistical analysis and input from our SME group. The group is asked to give opinions answering the question “How many minimally qualified practitioners would know the answer to this question?” The results provide a difficulty rating that is used to arrive at a minimum raw passing score. 

Step 6 - Live exam: The questions that made it through the beta process and the selected passing score are applied to the live exam, so that scores can be issued on demand. During each administration of the exam, Snowflake inserts a small section of new exam content to continue the process of collecting statistics. This allows us the ability to refresh exam forms, keep the questions current, and ensure the statistical thresholds are continually being met.   

Step 7 - Content refresh: As the cycle continues, new questions are written reflecting Snowflake’s latest features and services, and the exam outline is reviewed annually to ensure that the defined skills are relevant and the passing standard is appropriate for the industry.  Technology is constantly changing so the outline, questions, and passing standard need continuous review to ensure our exam stays in alignment. 

A certification exam is not a static object—it is constantly being monitored for performance, candidate feedback, and technical accuracy, and then frequently updated as needed. A great certification exam gives candidates the opportunity to demonstrate their competence in a way that is measurable, defensible, and meaningful to all stakeholders.

Interested in getting SnowPro certified? Click here. 

Or do you want to apply to be a SME and do a deep dive into the certification programs? Apply here.