The Off-List Reference Check
A reference-check procedure engineered to surface the negatives, by going past the provided referees and scoring the candidate out of ten.
- Difficulty
- Easy
- Time to result
- ~days to results
- Steps
- 5
- Confidence
- 86%
Sam Parr treats reference checks as the single biggest improvement to his hiring, and runs them to extract bad news rather than confirmation. He always asks for three references, then calls all three mainly to ask who else the person worked with, because the goal is to reach people the candidate did not nominate, while being discreet enough not to blow up the applicant's spot with a current employer. His scoring question is fixed: rate them one to ten. The answer is always an eight or nine, which is the setup for the real question, what is missing that would make them a perfect ten. That is when the negatives arrive. He then sorts the negatives into ones he can live with, such as certain kinds of mistakes, and ones he cannot, such as consistently missing deadlines. Shaan Puri adds two complementary questions: whether this person is one of the two or three people in any company who carry the team on their back, listening for the conviction in the yes, and a disarming framing that he is already leaning towards hiring but wants to know what they are not good at because everybody has something.
Origin
Sam's habit hardened after a single blunt reply about a management candidate saved him from a bad hire, prompting him to thank the referee profusely and systematise the approach.
Core principles
- 01Most reference checks will not give you honesty
- 02The referees you were given are a doorway, not the destination
- 03Ask for the score, then ask what is missing
- 04Sort negatives into tolerable and intolerable in advance
- 05Listen to how the yes is said, not just that it was said
How to run it
- 1
Collect three references, then go past them
Ask for three names as standard, then call each one and ask who else this person worked with. The provided three are a route to the unprompted referees, not the evidence itself.
Watch out Be careful with current employers. Do not blow up the applicant's spot while chasing off-list names.
- 2
Run the ten-point question
Ask what the person is out of ten. The answer is reliably eight or nine. Then ask how they could be a perfect ten and what is missing. This is the question that produces the negatives.
Pro tip Do not react to the eight or nine. It is only the setup.
- 3
Ask the force-of-will question
Shaan's version: at every company two or three people carry the team on their back, and without them things would be rough. Ask who those people were, or whether this candidate counts as one, and listen to how much conviction is behind the answer.
Pro tip A yes can be said two different ways. You are grading the delivery.
- 4
Disarm before asking for weaknesses
Say you are already leaning towards hiring so the referee does not feel their answer will sink the candidate, then say that everybody has things they are not good at and you simply do not know this person's yet, so you can have the full picture before pulling the trigger.
- 5
Sort the negatives against your own tolerances
Take each flaw and decide whether you can live with it. Sam can put up with certain categories of mistake but will not tolerate someone who misses deadlines all the time. The check only pays off if you have decided your thresholds.
In the wild
Sam messaged a contact asking whether a candidate would be decent at managing a small team. The reply was that they did not think the person could manage anyone, let alone themselves.
→ Sam did not hire the person, and went out of his way to thank the referee and send them a gift for the rare honesty.
Common mistakes
Only calling the three names you were given
Candidate-selected referees are pre-filtered for positivity. Treating their answers as evidence rather than as a routing step means the check tells you nothing.
Asking open questions that invite politeness
Without the score-then-gap structure, referees default to vague praise. The one-to-ten question followed by 'what is missing' is what forces a specific, usable negative.
Hearing a flaw and rationalising it away
The negatives are only useful if you have pre-decided which ones are disqualifying. Otherwise excitement about the candidate absorbs the warning.
Is it for you?
Best for
Founders making senior or high-trust hires where a bad fit is expensive to unwind
Not ideal for
High-volume entry-level hiring where the per-candidate call cost cannot be justified
From the transcript
“reference checks are so important and I do two things differently. The one, I always ask for like three.”
“I say, "All right, how can they be a perfect 10? Like what's missing?" And then that's typically when they'll tell me like bad stuff.”
“One is you're just trying to figure out is this person one of the best people that they worked with at that company?”
From the episode
CEO Lessons: Bulls**t Mission Statements, Firing People & Saying ‘NO'
CEO Lessons