Selection: structure beats instinct
Introduced| English | Meaning |
|---|---|
| structured interview | every candidate asked the same scored questions in the same order |
| predictor validity | how well a selection measure predicts later job performance |
| selection ratio | The number of people hired divided by the number of applicants. |
Cheaper at selection than at correction
- Nimbus's first structured hire: 40 applicants → 18 phone screens → 9 trial days → 4 offers → 3 hires. Each arrow is a rate; each gate needs a criterion.
- Structure beats instinct in every meta-analysis: by about a third in the classic 1998 figures, and more than twice over in the 2022 re-estimate.
The validity ladder
- Work samples ≈ 0.54, structured interviews ≈ 0.51, cognitive tests ≈ 0.49, unstructured ≈ 0.38 — the "I know in five minutes" illusion.
- Predictor validity 预测效度 is a correlation with job performance; reliability (consistency) comes first.
How valid is this predictor?
Rank selection methods by their meta-analytic validity and spot the legal-but-useless ones.
Of 40 applicants, 3 are hired. The yield, in % (one decimal)?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
$3/40 = 7.5\%$ - and every intermediate arrow is a rate with a named criterion.
Interviewer confidence is a good proxy for interview validity.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Unstructured interviews feel most insightful and score weakest (~0.38). The five-minute illusion is why the sheet exists.
Match each method to its validity.
Difficulty 2/5 · 140 coins + a small bonus on first completion today
The ladder is meta-analytic and stable - design the funnel around it.
The base-rate arithmetic
- Base rate of strong performers 0.20; a test of validity 0.4 lifts the top-quartile cut to ~0.35 — mediocre test plus cut still beats instinct.
- The funnel's economics: trial day £90 × 9 = £810 against a mis-hire season of chaos (30–50% of salary). The funnel is an investment with a number.
Base rate 0.20; top-quartile cut with r = 0.4 lifts the strong-performer share to about?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
About 0.35 - the selection theorem in one line: even a mediocre test plus a cut beats instinct. Better tests, better lift.
Trial day 90 pounds a candidate, 9 candidates. Total spend, in pounds?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
£810 - against a mis-hire at 30-50% of salary. The funnel is cheap; the correction is not.
Structure the interview
- Old: "Are you good under pressure?" (everyone says yes). A structured interview 结构化面试: "Tell me about a Saturday that went wrong — what did you do first?", scored 1–5 on anchored scales by two independent raters.
- Validity 0.38 → 0.51 is bought with a question sheet, not a personality.
The validity jump from 0.38 to 0.51 is bought with:
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Structure is the whole trick - fixed job-relevant questions and scoring rubrics. The sheet outperforms the personality of the interviewer.
Training: stock-outs 3 -> 1 a month, each worth 40 of margin; cost 180. The annual ROI gain, in pounds?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
Saved: $2 \times 40 \times 12 = £960$; minus £180 cost = £780 net - evaluated on the BEHAVIOUR moved, not the room's warmth.
A test that gives different results week to week can still be valid. True or false - and why?
Difficulty 1/5 · 100 coins + a small bonus on first completion today
Consistency is the floor: without reliable scoring, any validity claim is noise wearing a correlation.
The validity numbers were revised
- A 2022 re-analysis (Sackett and colleagues) corrected the classic figures less generously. Structured interviews now lead at about 0.42; work samples 0.33, cognitive tests 0.31, unstructured interviews 0.19.
- The ranking moved, but the lesson grew stronger: structure now predicts $0.42/0.19 = \mathbf{2.2}$ times as well as instinct. Quote a validity with its source and year.
Structured interview validity 0.42 against unstructured 0.19 (2022 figures): ratio (one decimal)?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
$0.42/0.19 = 2.21$.
Worked example: a strict cut on a modest test
- Nimbus hires 3 of 40 applicants, a selection ratio 选拔比率 of 7.5%. With a base rate of 0.20 and a test of validity 0.4, about 47% of those hired are strong performers.
- With a 25% cut the share is about 36%, and with 50% only 29%. A modest test becomes valuable when there are many more applicants than places.
3 hires from 40 applicants: selection ratio (one decimal, %)?
Difficulty 2/5 · 140 coins + a small bonus on first completion today
$3/40 = 0.075$.
The same test raises the share of strong hires more when the selection ratio is smaller.
Difficulty 1/5 · 100 coins + a small bonus on first completion today
At validity 0.4 and base rate 0.20: about 29% at a 50% cut, 36% at 25%, 47% at 7.5%.
Three selection slips
- The five-minute illusion. Gut is the weakest predictor wearing the strongest confidence.
- Validity without a criterion. Correlate against measured performance or it is not validity.
- Training evaluated by warm rooms. ROI = measured behaviour change.
Recap
- Structure every gate; work samples lead validity.
- Base rate × validity prices the cut.
- Development ROI = the behaviour moved, measured.