Thinking Out Loud

How to get it wrong less often

I don’t like to estimate on the sort of projects that I really like working on. But sometimes I have to quote, and I usually quote too low. I know I do, I try not to, but still I fall into the trap. Not any more.

There are very few people who can accurately estimate how much time or how much money it’s going to take to complete a software project. And there’s a well-known reason for that… the human brain is hard-wired to get it wrong, no matter the IQ.

Everyone has heard of, and maybe some have used, the quick-and-dirty method – Carefully cost everything out, then double your estimate. Multiply by a factor of 2.
And you’d be quite right to use the doubling method, but only if you are pushed into rough guessing without any thinking time – in the elevator, or on a phone call.

Going ahead with a project and finding that the costs blow out, that it limps on forever, then you run out of money, support, enthusiasm… that can kill your reputation as well as your confidence.

You want to be sure you get it right first time. Yes?
So you sit down to do a “realistic estimate”.

Now, there are only 2 reasons that you’d want to estimate the time or money that a project might consume – either you want it to go ahead, or you want to prove that it’s unwise to start at all.

If you successfully demonstrate that it’s a bad idea, or rather that the projected costs outweigh the projected benefits, then it ends quickly. Move on to the next one. End of story.

BUT – what if you actually want the project to go ahead?

The owner of the project wants the project to happen because the outcomes would be worthwhile. The contractor wants it to happen because it sounds interesting and worthwhile. They get their heads down and really look at the details of this worthwhile and fascinating project.

The more detailed your planning is, the worse your estimates are going to be.

Now we start encountering the human brain. It’s full of awesomely useful tools to deal with the world, predict outcomes, and get what it wants. And these mental tools have been finely honed over untold generations to work to our advantage. Along with their awesomeness come some sneaky little partners-in-crime called biases.

Optimism Bias causes us to focus only on the good things. (Bad things will not happen this time, because this is a great project. Besides, if we thought bad things were likely, we sure wouldn’t be doing this detailed estimate, now would we? Bad things messed us up a couple of times in the past, but now I am prepared)

Political Bias causes us to bend our perception because we desire the estimate to be optimistic. (If this estimate is accepted, then the project will go ahead. And if the project is successful, then I will personally benefit. Therefore it will be successful. I can taste the outcome and it is sweet.)

The owner of the project really wants the project to happen. She has visions of promotion and status for getting it done. She has a budget. She has heard stories of people doing much more than this, in a few short days. She is optimistic. If the estimate falls into her timeline and budget…

The person who is going to make this happen, also sees the personal benefit. It might be money, experience, accolades, or more projects and clients in future. If the estimate falls into the timeline and budget…

Focusing on the unique details of a project plan causes a person’s brain to decide that this plan is totally unique, nothing like the past failures, and in fact much better in so many respects. (I see no problems that are unforeseen, I see no unexpected costs. Therefor this is going to be a much easier project than we first thought… )

Easy fixed — we can adjust for our bias!

A bias cannot be undone by being aware of it. Biases are not random, and you’ll only be adjusting within your own (biased) frame of reference. Every expert is just as biased as everyone else. That’s a fact you cannot change. That’s a dead end.

So, how to get it wrong less often?

Step 1- Be Aware

If the project is going to take more than a day to complete, if you stand to gain by the estimate being acceptable, or if you have to draft a detailed plan in advance, then you’re in grave danger of underestimating.

Step 2 – Decide what “class” of project this is

By using a simple technique called Reference Class Forecasting, it’s possible to get a really useful estimate, even when there are lots of unknowns. And you’ll be more likely to bypass those pesky bias errors.

Your goal here is to gather a number of examples of similar projects from the past. Your “Reference Class” needs to be something like “iPhone apps with 10-15 interface screens”, or “training 10 new users”. It needs to be a tight fit – but it also needs to be broad enough to allow you to gather a solid 5-20 examples. They don’t need to be your personal projects, so you can Google away, or ask colleagues who make iPhone apps or train users.
Make sure to include examples that have failed, cost half as much as typical, or took twice as long as the others – in other words, be sure to include outliers, not just ones that fit your own hopes/expectations.

Step 3 – Find the median for your Reference Class

Arrange the results in time or money order, then find the middle ground.
Don’t calculate the average. We want the result that’s in the middle of the sorted list. That’s the one you want to use as your estimate.
If you absolutely need to avoid over-runs, then shift higher up your list.

The resulting answer will probably horrify you.

Maybe the estimate will be so high that the project is cancelled.
Good, you would have lost money/time on it.

Maybe your estimate will be higher than another contractor.
Good, let them lose money/time instead.

But it will be way more accurate than your carefully detailed estimate. Probably by a factor of 2.

Hofstadter’s Law:
“It always takes longer than you think, even when you take Hofstadter’s Law into account.”

Add comment