An engaging hiring conversation can leave you feeling that you understand a candidate’s personality and how they would work. To use that impression in a hiring decision, identify which answers support which job capabilities. Feeling convinced by a conversation does not establish that you have accurately predicted later performance.
A research reanalysis reported a larger relationship with job performance for interviews with predetermined job-related questions and scoring criteria than for interviews driven by free-ranging questions and overall impressions. The former approach is called a structured interview. Hiring managers who are unsure about an impression should examine which capability a question assessed and whether the answer provided evidence they could score. (Sackett and colleagues’ reanalysis)
Random answers still produced a sense of understanding
In Dana and colleagues’ 2013 experiment, 76 undergraduates at Carnegie Mellon University in the United States conducted 20-minute interviews to predict other students’ semester grades. One group received answers unrelated to the interviewee’s actual views during the second half. Usefulness ratings did not differ significantly from ratings for truthful answers. (Dana and colleagues’ experiment)
These results concern student grade predictions; hiring accuracy requires separate evidence. Collection dates were unreported. The finding illustrates the need to check subjective confidence against the information obtained.
The practical implication is to distinguish a coherent story about a candidate from evidence of job capability. If calm conversation suggests to you that someone handles incidents well, ask what they actually decided and did in an incident. Evaluate a record of behavior rather than the plausibility of your impression.
One way to examine interview usefulness is to study the relationship between interview ratings and actual job performance. A meta-analysis combines results from multiple studies. Sackett and colleagues’ 2022 paper revisited correction methods that had inflated earlier estimates of selection validity. It reanalyzed existing research rather than measuring the outcomes of one new hiring program.
Their 2023 follow-up presents corrected average correlation estimates for job performance:
| Interview format | Average correlation with job performance |
|---|
| Structured | 0.42 |
| Unstructured | 0.19 |
A correlation describes how higher assessment scores relate to higher performance. The value 0.42 does not mean a 42% hiring success rate, and dividing the coefficients does not mean hiring succeeds about twice as often. Results vary across studies and interview designs; the average cannot simply become your company’s expected accuracy. (Sackett and colleagues’ 2023 paper)
These findings give you a reason to reconsider interview assessment. If candidates are compared mainly through overall impressions, prepare questions and criteria tied to the job. The figures above describe relationships across research, rather than an experimentally measured improvement from changing your own interview process.
Do identical questions make an interview structured?
A structured interview uses common questions and evaluation criteria for candidates applying for the same role. The U.S. Office of Personnel Management, or OPM, also describes standardizing question order and rating scales. A shared question list still leaves assessment undefined if scores depend on conversational fluency or an interviewer’s preferences. (OPM’s structured interview overview)
Build questions from successful job behavior
Questions should examine decisions and actions that the role requires. OPM describes using job analysis to develop questions about past behavior or proposed responses to job-like situations. (OPM on question design)
For a customer support role, move beyond a general label such as communication skills. Ask about an experience in which the candidate identified a request, investigated necessary information, and selected a response. Distinguish team outcomes from the candidate’s own contribution. Sales totals or case counts alone reveal little about the decisions that person made.
Decide what a scorable answer contains
The following is an illustrative rubric for a question about customer support experience. It is an original design example showing how questions and assessment can align, rather than a reproduction of an OPM scale.
| Assessment | Evidence in the answer |
|---|
| Insufficient evidence | Describes the problem but leaves personal actions or reasons unclear |
| Required evidence present | Specifies information investigated, action selected, reasons, and outcome |
| Additional capability demonstrated | Also explains how the outcome was checked and used in later responses or process improvement |
This rubric assesses decisions and actions described in the experience. Adding conversational polish or shared hobbies changes what the score represents. Define follow-up questions for missing information in advance so candidates receive comparable opportunities to provide evidence.
Candidates without job experience may have little material for questions restricted to past professional work. Consider job-like scenarios or decisions from study and other activities, with the questions designed around the capabilities the role requires.
Why interviewers should score before discussing candidates
Recording impressions only after a panel discussion loses the connection between each interviewer’s judgment and the answers they heard. OPM’s September 2008 practical guide recommends recording and evaluating responses independently, followed by discussion of scores and supporting evidence. The guide describes U.S. federal interview design. (OPM’s practical guide)
A concrete application is to record ratings and supporting answers before the discussion. When scores differ, examine whether interviewers applied different standards to the same answer or recorded different facts. Use those discrepancies to revise scoring criteria or note-taking, rather than relying only on an average score.
After hiring, review whether the assessed capabilities matched the work actually assigned. A small number of hires provides a limited basis for estimating accuracy. Use missing evidence and unclear scoring reasons to improve later interview design.
Align the job description with the work assessed in interviews
Assess interview impressions by asking whether you obtained evidence of required job behavior. Shared questions, scoring criteria, and interviewer records make the basis for comparing candidates explicit.
Describe that work clearly on the recruitment page too. Publishing actual responsibilities and expected contributions lets applicants compare their experience with the role before applying.
SpreadSite is a no-code website builder that lets you create, publish, and update a website from Google Sheets using ready-made templates.
To create a recruitment website explaining responsibilities and current vacancies, give SpreadSite a try!
Build a website with SpreadSite →
