Short answer
#How to read an answer
Start here. The questions are worth less than the ability to hear what an answer is doing.
You cannot evaluate the technical content, and you do not need to. Four properties of an answer are legible to anybody:
- Specific beats general. "The payment service and the export job" is an answer. "The usual things" is a deflection, usually an unintentional one.
- Calibrated uncertainty beats confidence. "I am sure about A, less sure about B, and nobody has checked C" is the answer of somebody who has looked. "It is all fine" is the answer of somebody who has not been asked before.
- "I will find out" is a good answer — provided a date comes with it, and the finding-out actually happens.
- Demonstration beats description on exactly two questions: can we restore a backup, and can somebody else deploy. Both are answered by doing, not by saying.
The one answer to always follow up
"Nothing", in reply to "what do we not know about our own system?". Every real system has open questions. A team that cannot name any has either not looked or is protecting you from something, and both are worth a second question.
#Monthly — four
Short, and always the same four
These are a rhythm rather than an inquiry. Fifteen minutes.
- What changed structurally in the product this month? Not a list of tickets. A description: what is now different about the system.
- What broke, and did anything break twice? The second half is the important one. A recurring fault is a design problem wearing an incident costume.
- What is the thing you are most uncomfortable about right now?
- Is anything blocked on a decision from me?
#Quarterly — ten
Dependency and concentration
- Which parts of the system can only one person explain? Ask for names, not for reassurance. The answer is a risk register.
- Has anybody other than the usual person deployed this quarter?
- What outside services have we started depending on since we last talked?
- Is there anything running that is not in the repository?
Recovery
- When did we last restore a backup, and how long did it take?
- If our largest external dependency were down for a day, what would customers see?
Understanding
- What do we not know about our own system?
- Is the technical picture we have still accurate, and when was it last checked?
- What would a new engineer find confusing on their first week? A useful proxy for undocumented convention, and it is a question people enjoy answering.
- What have you been asking for that has not been approved? Ask it every quarter. The answer is often the cheapest problem on the list to fix.
#Annually — ten
Ownership
Accounts accumulate. Once a year is enough, and skipping it for three years is how the surprises happen.
- Whose name is the domain in, and where does its recovery email go?
- Whose account is production in, and who is billed?
- Are all third-party accounts in company names?
- Are the recovery contacts on every critical account company-controlled?
Continuity
- If you were unavailable for a month, what would stop?
- Could another team take this over, and how long would it take?
- What would we hand them?
Direction
- What are we carrying that we would not build the same way today?
- What is the single biggest technical risk to the business? To the business, not to the codebase. The distinction produces a different answer.
- What would you do with a month of nobody asking for features?
#How to ask without it becoming a test
Three things make the difference between a rhythm and an interrogation:
- Same questions, every time. Predictable questions get prepared answers, which is the point. A new question every month reads as fishing.
- Give the real reason. "I need to be able to answer this to an investor, an insurer or a large customer" is true, is not about them, and comes with a deadline that is not you.
- Ask what they need, too. The last quarterly question — what have you been asking for that has not been approved — turns the conversation into an exchange rather than a review.
A competent technical leader usually welcomes this. Most have wanted somebody to take the continuity questions seriously for years, and the ones who resist tend to be resisting the answer rather than the question.
#What goes wrong
Asking all twenty-four at once.
Instead Four monthly, ten quarterly, ten annually. A two-hour session once produces performance; a rhythm produces information.
Accepting reassurance as an answer.
Instead “It is fine” is not an answer to any question here. Ask for the specific thing: which parts, whose account, when did we last.
Not writing the answers down.
Instead Keep them. The value compounds — comparing this quarter with two quarters ago is where the pattern is.
Asking only when worried.
Instead Questions asked only in a crisis are read as accusations. Asked on a schedule, they are just the schedule.
Never asking what they need.
Instead It costs one question and it changes what the whole conversation is for.
#What this does not cover
What this does not do
- Evaluating technical decisions. These questions establish facts; whether a decision was right is not something a founder should be adjudicating from answers to a list.
- Performance management. This is not a review framework and using it as one will destroy its usefulness.
- Hiring a first technical leader, which is a different problem.
- Anything about product strategy or roadmap.