Sunday, 12 September 2010 00:00

Good Estimates Only Have a 50% Chance of Being Right

Rate this item
(0 votes)

Graphic Normal Estimation Distribution

Estimates are wrong if you cannot beat them half of the time; they are also wrong if you are not late half the time. Neither condition is one that should make management upset. In fact, matching that score is a great accomplishment. So how can people get so emotional about the statement? The answer is that people do not understand estimates and how they work. Through years of estimates being treated as quotes, we have been brainwashed into thinking our best effort is to meet the date, not better it. Heaven forbid if you are late. This lack of understanding is very evident by the number of blogs on the subject and some of their bewildering comments. The comments point out wildly different views. Some people think that Monte Carlo analysis gives you an estimate that has a 95% chance of being right and others believe that using Agile relieves people from having to make estimates entirely. Both of which are plainly not true.


Before the discussion can go any further, the obvious needs stating—to make an estimate we need to understanding what is being estimated. If we lack the experience to create a detailed work breakdown structure, our estimates will be useless. This is true whether you are planning to go grocery shopping or building a house on Mars. The former each of us has done hundreds of times. The tasks need to be broken down into their constituent subtasks, each subtask estimated, and those estimates added together to make a complete timeline and budget. Regardless of the level of planning, repeated execution of the same project will yield different performance results. Grocery lists change, stores run out of stock, traffic backs up, or checkout lines are too long; whatever the reason, the time and cost of any two trips are never the same. To give a good estimate there needs to be a complete understanding of what will influence the project.

If how or what is being built is unclear, do an experiment to see the type of problems that could be encountered. Build a prototype, try something similar, or build a subcomponent. This can help with honing estimates by providing experiential data. It does not obviate the need for estimates; it helps create them. This is the basis for NASA's Cone of Uncertainty principle and a reason that the Agile methodology works—explore so you can predict.

Odds of Completion

For people accustomed to critical chain, this section may be a little remedial. Good estimates will be beat 50% of the time, the rest of the time you will be late. If you are always beating your estimates, you are doing so at the cost of the project's timeline and budget. To understand this better, refer to image at teh top of the article. This is a plot of actual values accrued during workshops where attendees are given the assignment of flipping a quarter and record the results twenty times. By looking at this distribution, a good estimate for this task is about 107 seconds. With this estimate, half the time it will take you less time to do the task and the rest of the time you will be late. However, if you want to be right 80% of the time, you would use an estimate of 170 seconds. Although you will rarely get in trouble for being late on the task, a project made up of these tasks is going to be quite costly. Assume the units were hours, not seconds, and the project required this task reiterated fifteen times. You would have swollen a nominally yearlong project with seven months of pad. With nineteen months to do a twelve-month project, there are too many other factors to create problems. Student Syndrome (procrastinating until the last minute), Parkinson's Law (work swelling to fill available time), the Garage Syndrome (scope swelling to fill available technology), and a sixty percent longer project opens the project to a plethora special cause variations (business fluctuations, management turnover, economic changes, and so forth). In the end, you will most likely fail at the project because it has increased risk.

Tools: Three-Point to Monte Carlo

Graphic PERT Distribution
Figure 2. PERT Distribution

The method of doing estimates does not remove this problem. Instead, they have the effect of luring people into treating the estimate as a quote. Three-point estimates (for instance PERT) take a best, nominal, and worst case estimate and perform a weighted average to try to arrive at a value that simulates the midpoint of the estimation distribution. Figure 2 is an example of such a distribution. With a little imagination, you can see its similarity to the normal distribution in Figure 1. The problem is that it is generated from only three values provided by people trying to imagine all the possible problems, and their odds of occurrence. In a three-point estimation process, answers can be biased simply by asking the questions in a more positive or negative manner.

Monte Carlo, on the other hand, looks at schedules and the potential variations for each task and derives the most likely completion time. It generates a distribution much like that in Figure 1. However, instead of using empirical data from prior projects, it takes distributions for each task, the combinations of how those tasks fit into a schedule, the effects of the variation on the critical path, and determines the most likely time to the complete the project. If the project was run fifty or a hundred times, half would finish in less time, the rest would be late. If it were not for this fact, born in statistics, the estimate would be wrong.

Poor Estimates Result in a Lack of Urgency

In reality, humans make estimates and we rarely have sophisticated tools to help us see all the risk. Therefore, we need to choose a value that has slightly better chances than 50:50 of being met—maybe 60:40. This accounts for our natural optimism and the reality that some of the strangest events get in the way of completing assignments. Choosing an estimate the gives us an eighty or ninety percent chance of success creates a lackadaisical approach and we struggle, often failing, to complete the task on time.

Corporate culture has conditioned people to provide estimates that can be met ninety percent of the time. Managers and individuals use the pad to achieve other goals, none of which are focused on getting the project done faster or at less cost. The only solution is to educate people on providing real estimates.

Read 10497 times

Related items

  • Process Mapping

    Process is at the core of any business. It makes work predictable, repeatable, and transferable. Without it we cannot scale our businesses. However, process can be a bane to making progress. Processes that work for a $10 million company have difficulties supporting a $30 million company. Trying to scale them to a $300 million company will not only fail but not address the issues that larger companies have that were never dreamt of in a smaller organization. Processes need to be discarded, revamped, and built—all of that without creating an overburdening bureaucracy.

    Anytime you need to go someplace, you first have to know where you are. Processes are never static and your company's current state is probably far from where you think it is. Hence, the first step is mapping out you company's current state followed by defining the future state. This is more than a logical map of the process; it must also include physical maps. Whether your process is solely to provide a service (say, website development) or physical (say, manufacturing) there are logistical issues that complicate the process flow. Without fully understanding those nuances, future state processes will not reach the desired efficiencies.

    For more information about process mapping fill out the form to the left and we will get in touch with you.

  • Kill The White Knight

    There is a reason we do not teach classes on fixing failing projects. Many a cynic feels that we simply do not want to teach our trade, however, our reason is far nobler—we should be teaching prevention rather trying to create white knights to save the day. It is the same philosophy as building a fence at the cliff's edge rather than an emergency room at its base. Our language is replete with idioms telling us to look past the symptom and address problems at their root cause. 'An ounce of prevention versus a pound of cure' or 'a stitch in time saves nine.' Please, feel free to supply your own in the comments. Unfortunately, most of our businesses loathe this philosophy, waiting to address an issue until it is irrefutably broken.

  • Project Inception - Designing Organizations For Success

    Buy it now!

    A failing project’s fate is destined long before assigning a project manager. Its doom is sealed from the time the customer envisions the idea. Traditionally, project inception is defined as when the customer comes to a solution provider (internal or external to their organization) asking for a product or service. In actuality inception is much earlier. It starts when someone says, “Wouldn’t be neat if I could...” From that point forward the customer’s exceptions are set, changed, and reset as the process of discovery refines the concept. The customer’s ideas change from what they want to what they need, while continually constrained and formed by the realities of an ever-changing business environment. Although people cite unrealistic expectations as major problem during inception, the constant change in expectations causes the real issue—misalignment. For project managers to make a significant difference in a project’s success, they must use a new paradig.

  • Organization Change Management for Project Teams Workshop
    Want to buy it now?

    Ask for more info below, or if you are convinced, just add it to your cart.

    Projects are never a success when they are delivered—their product must be adopted to declare success. Whether you are delivering a process for HR, creating new model of cell phone for your customers, or implementing a new ERP system for your company, if they do not see value in the output of your project, it is a failure. Most project teams, however, are focused on maintaining scope, schedule, and budget, they are far removed from the end-user, and they have little concept on how to persuade someone to use what they are developing. The fact of the matter is, though, that if they are the first people involved in the making a tangible product that their customers can use, adapt, and enhance to create value.

    Organization Change Management for Project Teams helps your project manager, their teams, and their stakeholders:

  • Balanced Scorecard for Project Teams Workshop

    Too often, project managers and their stakeholders lack the visibility into how their project's fit into the business' grand vision. Think how wonderfully your business would run if everyone from the C-suite to the feet on the street understood how to maintain focus executing business strategies.

    Want to buy it now?

    Ask for more info below, or if you are convinced, just add it to your cart.

    Balanced Scorecard for Project Managers helps your project managers and their stakeholders:

    1. Understand what is valuable for your organization.
    2. Reduce miscommunication.
    3. Focus their energies and your resources.

    Organizations the world over use balanced scorecard to define their strategic goals. However, balanced scorecard only works if its information is disseminated throughout the organization. This workshop helps PMO managers, executive sponsors, project managers, and their project teams understand why and how a strategy is defined, the use of activity and strategy maps, and how they apply to the organization's projects.

Leave a comment

More Info on Project Recovery

Tell me More!

Please send me more information
on fixing a failing project.

Rescue The Problem Project

Internationally acclaimed

Image of RPP

For a signed and personalized copy in the US visit the our eCommerce website.

Amazon logo
Buy it in the United States Buy it in Canada Buy it in the United Kingdom
Buy it in Ireland Buy it in Germany Buy it in France
Buy it in Italy Buy it in the PRC
Buy it in Japan
Book sellers worldwide.

Upcoming Events

Other's References