The short version
- People estimate their own tasks optimistically even when they know past tasks overran. This is the planning fallacy.
- Programming work has a long tail on the slow side: few outputs finish much faster than expected, some take two or three times longer.
- Estimate each piece as a range, add the ranges properly, and plan to a stated probability rather than to a single number.
The planning fallacy is not carelessness
In a well-known set of studies, students predicting when they would finish their honours theses were mostly late; most also missed the date they gave for the case where everything went as poorly as it possibly could, and recalling past overruns did little to correct their forecasts [1]. Kahneman and Tversky described the cause earlier: we build a forecast from the specific plan in front of us, the inside view, and neglect what usually happens to projects like it, the outside view [2]. A programmer estimating an efficacy table imagines the derivation going well. They do not imagine the unclear SAP footnote, the vendor data in a new format or the QC finding that sends it back.
Adding people late does not rescue it either. Brooks' observation that adding people to a late software project makes it later holds for statistical programming too: new people need onboarding, and communication paths grow with team size [3].
Why the sum of fair estimates is still too low
Suppose each output's effort is skewed: a typical case of 6 hours, a best case of 4 and a realistic worst case of 15. The typical case is the most likely single value, but the average is higher, because the slow tail is long and the fast tail is short. Add 40 such outputs using the typical figure and you get 240 hours. The expected total is noticeably higher, and the chance of finishing within 240 hours is small, because small overruns on many outputs accumulate while early finishes rarely give much back.
Going the other way, adding 40 worst cases gives 600 hours, which assumes everything goes wrong at once. That number protects nobody, because it will be cut in the first budget conversation.
The useful answer is a range for the total. A simple and transparent way is the three-point (PERT) approximation for each item, mean = (best + 4 × typical + worst) / 6, which here gives about 7.2 hours per output, or about 287 hours for 40. A spread for the total can then be built from the individual spreads, or by simulation, which is what the CDRN Capacity Estimator does with ranges you calibrate from your own completed work.
Table: Synthetic example, 40 outputs, hours
| Basis | Total | What it means |
|---|---|---|
| Sum of typical estimates | 240 | Usually missed |
| Three-point expected total | about 287 | Roughly a coin flip |
| Upper planning figure from simulation (P80 to P90) | well above 287, far below 600 | A date you can defend |
| Sum of worst cases | 600 | Never happens, never funded |
The figures are illustrative, not benchmarks. Your own ranges come from your own past work.
Turn the range into a conversation
A range is only useful if it changes a decision. State it like this: "With the current team, we have about an even chance of finishing by the 14th and a good chance, about eight in ten, by the 21st. If the population decisions arrive after the 7th, both dates move." That sentence gives the study lead a choice: accept the risk, add capacity early, reduce scope or move the date. A single number gives them only a surprise.
Do this today
- For your next deliverable, ask for a best, typical and realistic worst case for each output group instead of one figure.
- Compare last study's estimate with its actual hours, and use the ratio as your outside view.
- Restate your next delivery date as a range with a probability and the assumption that would move it.
Check yourself
One question
Show the answer
Answer 3. The sum of typical estimates ignores the long tail on the slow side, so it is usually beaten less than half the time. The sum of worst cases assumes every output goes wrong at once, which almost never happens. A range for the total, from the individual ranges, is both honest and usable.
Apply it
Turn it into a plan
The brief says what to check. The Navigator and the Handbook turn it into dated, owned actions for your milestone, in your browser, with no study data.
Sources
References
- Buehler R, Griffin D, Ross M. Exploring the "planning fallacy": why people underestimate their task completion times. Journal of Personality and Social Psychology. 1994;67(3):366-381.
- Kahneman D, Tversky A. Intuitive prediction: biases and corrective procedures. TIMS Studies in Management Science. 1979;12:313-327.
- Brooks FP. The Mythical Man-Month: Essays on Software Engineering. Anniversary ed. Addison-Wesley; 1995.
Standards and guidance change. Confirm the version in force for your study, sponsor and region before relying on any reference. Examples are synthetic and illustrative. This brief is generalized professional guidance, not regulatory advice.
Next brief · Friday 10:00 ET · Failure Friday