A Weighting System

An Objective Project Scorecard May Need More Complexity (Part 2 of 2)

As you take more and more factors into account to determine project success, the last area you need to address is some type of weighting system. This will be necessary because not all of the performance factors are of equal importance. For instance, it may be vital that your project be completed by a certain date, and that may be more important than whether you hit your budget target. In fact, you may have to overspend your budget to make sure you hit the deadline. So, on that project, meeting the deadline date should be weighted more heavily than meeting the budget target.

The way you establish an overall weighting formula is to start off with a maximum point total and divide the achievement of that maximum total into the number of success criteria. Let’s look at an example of a project with five success criterion. The achievement of all the success criteria results in a 100-point maximum. In other words, if the project team did everything perfectly, their total score would be 100 points. Let's also say that the five criteria are deadline, budget, quality, client satisfaction and sponsor rating of success. You would develop a scorecard grid, as follows:

• 1. Deadline, 25 points, complete by December 31
• 2. Budget, 10 points, $250,00 plus or minus 10%
• 3. Quality, 20 points, five questions survey averages at least 3.5 out of 5
• 4. Client Satisfaction, 20 points, five question survey averages at least 3.5 out of 5
• 5. Sponsor rating of success, 25 points, three question survey averages at least 3.5 our of 5

Total possible points - 100

You can get very sophisticated with this scorecard system. However, you will find that each attempt to get more fact-based and less subjective requires you to get more and more precise in defining the overall scorecard rules. For instance, the following needs to be taken into account as you get more sophisticated.

You must define what the numerical total represents.

For instance, on the project above, you could say that any final rating over 80 represents success. In that case, the project above was successful. If there was a bonus to be paid for project success, you could say that a full bonus would be paid for an overall rating of 80 or above. No bonus (or a partial bonus) would be paid for a score under 80.

You would define whether partial credit is given to scores that do not meet the scorecard criteria

For instance, in the example above, the project went over budget by $20,000. This was within the acceptable range of plus of minus 10% ($225,000 to $275,000), so the team received full credit of 10 points.

On the other hand, the overall quality rating was 3.2 and the target was 3.5. The question is whether the project team receives zero points since they missed the criteria, or if they receive partial credit for getting close. In this example, the team received 10 points, or half the possible total. You could imagine that at some point; say under 3.0, the team would not have received any points. If the team had missed that criterion, their total points for the project would have fallen to 75. If the success target was 80, the project would have been viewed as a failure, even though the rest of the criteria were hit successfully, including an overall 4 out of 5 rating from the sponsor. That shows why it is important to have proper weighting for your project. If you are going to have this level of algorithmic objectivity, it is important that the team really focus on the important scorecard success factors. This is a powerful incentive.

You must decide whether the weightings represent the maximum allowed, or whether the team can score higher.

For example, on your Sponsor survey criteria, perhaps you could establish a rule that says that you receive an extra 10 points for every .5 over your target. In our example above, since the final score was 4.0, the team would have received 35 points for this criterion, which represents the target 25 points, plus an extra 10 points for exceeding the target. Again, if you allow extra credit, you must determine what that means. For instance, if you were paying a bonus on successful completion of the project, going over 100 may mean that the team receives a higher bonus, say 110% of the target bonus.

The Success Formula Must be Known Ahead of Time

The key to these rating systems is that they have to be defined ahead of time so that the entire team can focus on the areas that are important. If you don't finalize the criteria until the project is already started, you may find out that you are too far behind in certain categories to achieve them. If the team feels that they do not have a chance to achieve the targets, the scorecard will result in a lot of negative feeling and poor morale, which is just the opposite of what you are trying to achieve.

The other aspect of having the scorecard built early is that it gives the project team time to adjust if there are problems. For instance, in the project example above, let's assume that the project duration is around nine months. The project manager may send out the quality survey and the client satisfaction survey after three months and after six months. That way, if the survey results are not coming as expected, the project team can determine the cause, and the entire team has time to correct the problems before the project is completed. This is part of a process improvement process. If the scorecard is not completed early and the team does not have time for interim surveys, the first time they see survey results is when the project is completed, which is too late to take any corrective activity.

Summary

Is your project a success or not? Simple metrics are easy to capture, but they do not leave much room for telling the entire story on a project. In general, the simpler your metrics are, the harder it is to look at shades of gray. If your success is based strictly on your sponsor’s feedback, you can end up in an all-or-nothing situation. The more aspects of the project you take into account, the more sophisticated you need to be in terms of how the separate criteria get put together for the overall success level.

If you have multiple success criteria, you will find you also need to put together a weighting scale. When you were in high school, your final grade was determined by some sort of formula based on your test scores, quiz scores, homework, attendance, etc. For the most part, everything was numeric based, and could all be calculated based on the raw scores. Pretty simple, and no surprises.

On a project, however, it is not that simple, since you have to take into account client satisfaction and quality – both of which get into subjective data. Your project scorecard can try to compensate for the subjectivity, and attempt to make the process as objective and mathematically driven as possible – just like your high school grade.

However, the more objective and fact-driven you attempt to get, the more complex and intricate your scorecard data collection and interpretation needs to be. Your scorecard criteria should be balanced and comprehensive. You should have targets that are challenging but reasonable. You also need to weight the various aspects of the scorecard to make sure that the relative importance is established. If you do not think through these implications, you may have confusion and disappointment at the end of the project as your scorecard results are subject to multiple interpretations.