Measure Me!

by Carl Pritchard, Pritchard Management Associates

The scale is my enemy. Really. I hate it. As I step on and watch the dial whirl under the needle, I pray for a number I can live with. I never get that number. So, I'll take a break from it and come back in a few months. Maybe it will have softened by then. Or maybe I'll get my cholesterol number. Or my body fat percentage. Odds are I won't like any of the numbers, but who knows. There may just be one out there that makes me feel better about all of it.

I just finished a one-day workshop with a group of project professionals on project metrics. We skipped past the conventional metrics of earned value and milestone variance into the more esoteric territory of portfolio metrics and risk metrics. After it was over, one participant asked if there weren't more. Maybe he wanted his cholesterol number.

Metrics are compelling things. They have the allure of being a potential panacea for project ills. They will identify which projects to work on first, which projects deserve the most resources, which projects are the highest risks and which projects serve the organization best. But it depends on which metrics you apply. A portfolio model may indicate the project meets a host of strategic objectives, but may not identify the relative risk. A risk model may identify the relative risk, but not the resource consumption. A ratio of resource use to profit may show effective utilization in terms of current dollars, but may erode the organization's infrastructure.

In fact, that student's request for more metrics was probably well-grounded. But it's not just more metrics that we need. We need the right metrics. We need to find the metrics that accurately reflect the culture, the organization, its capabilities and how projects fit into that mix. Almost invariably, some customization seems to be required. But that creates its own set of headaches. If we customize metrics, we may lose the ability to evaluate our organizations in the context of other organizations that apply the same metrics. We may also delude ourselves into believing we're better than we are.

(As an aside, I had a scale once that consistently made everyone over 200 pounds fifteen pounds lighter. I dearly miss that scale.)

Which Metrics are Right?

The "right" metrics are those that accurately afford a consistent understanding of where projects and organizations stand at a given point in time, versus how they stood at another point in time. They reflect what matters most to the organization and how those within the organization are judged. The are both reflective and predictive´┐Żlooking to the past and the future.

The right metrics are also those that have some proven validity over time. Earned value is popular, in part, simply because it has been around for such a long time. There is a relative sense of how bad a .62 CPI might be and an understanding of the uselessness of SPI at the end of the project (where it eventually, always, hits 1.0).

If we're going to use new metrics or organizationally specific metrics, we will have to give them time to prove themselves. Without the time to evaluate whether or not the metrics actually serve our long-term interests, their information is as valuable as stepping on a neighbor's bathroom scale. Yes, some information is provided, but its real meaning may be obscured because we don't know or understand what it means.

The other issue that should be considered in determining if the "right" metric is being applied is whether or not it answers the question we're asking. Weight on a scale doesn't report on body fat. Earned value doesn't reflect team satisfaction. Customer satisfaction metrics don't reflect on corporate sustainability. Selecting the right metric means looking at project after project after project and asking "Which measurement really lets us know that x is happening or will happen?"

It may be organizationally dependent. In some organizations, customer satisfaction is clearly tied to customer retention. In others, earned value and CPI numbers may directly reflect the probability of technical success. But this won't be the case in all organization.

How Do We Know?

Perhaps the most effective way to establish the right metrics is to reverse engineer the entire process. By examining past projects and weighing them as they would be weighed under new metrics, we can get a relative sense of the potential for success with the metric(s). For example, if seven projects that were highly technically successful all had CPI of 93% or higher, we may have discovered a valid metric. If that same sample is tainted by four abysmal failures that had CPI of over 93%, it may indicate that the earned value metric is not appropriate in this particular instance.

In the rush to measure, some organizations are pushing hard to assign and establish meaning for numbers. In reality, those numbers may be meaningless. By working back from older projects and finding out what made them "tick," we may be able to discern which metrics are the true arbiters of success.

Do We Want to Know?

The other key question that really needs to be asked for any organization is whether or not we really want to know the numbers that the system or process may generate. When I step on the scale, I know then that I really didn't want to know, because I'm not prepared to act on the information. If your organization begins the slow crawl toward accurate metric measurement, it's completely fair to push back and ask how the numbers will be used, how they will affect behavior and performance, and whether or not they'll be around for the long-term. If we're simply marking time until a better metric is developed, we might be wiser to save organizational time and energy by waiting until we have a metric we can truly use.

And if we do want to know and have found meaningful metrics, success will largely be rooted, in the long term, by how long we use the same measure. Time is one of the great validators of metrics.

©Copyright 2000-2018 Emprend, Inc. All Rights Reserved.
About us   Site Map   View current sponsorship opportunities (PDF)
Contact us for more information or e-mail info@projectconnections.com
Terms of Service and Privacy Policy