Quality That is Worth the Cost

by Alan S. Koch, PMP

We never have enough money. We never have enough time. We are always under pressure to get more and more done with fewer and fewer resources. With so much pressure, we can't afford to waste time (or money) on Quality Assurance work that isn't necessary.

But how can I know which activities are worth the cost? The development work is obvious: we must build a product if we are to have something at the end of the project. But the QA work doesn't produce anything tangible. We can't sell defect reports!

How can I figure out if the QA work has real value to my project? Are we doing things that are valuable? Are we doing things that don't have value? Are we not doing things that would have value? How can I tell?

It is true that QA activities do not produce any tangible products. The Cost of Quality (CoQ) is an overhead cost, and like any overhead cost, it should be minimized. But without a clear understanding of all of the things that should be counted, many organizations do the wrong things and actually end up increasing their total CoQ.

What is Your Project's Total Cost of Quality?

The first step in deciding which quality-related activities are worthwhile is to count your project's total Cost of Quality (CoQ). What costs can you identify on your project that should be counted as CoQ?

(No, don't continue reading; stop and answer the question!)

Most of us will immediately focus on the preparation and running of tests. And if our team does reviews of requirements, code or designs, we will think of those as well. Can you think of anything else that you should count as CoQ on your project?

Things like testing and reviews fall in the category of Defect Detection Costs. But the Cost of Quality includes two other types of costs as well: Defect Prevention Costs and Failure Costs.

Defect Prevention Costs

Any activities that we engage in to prevent defects in the first place fall into this CoQ category. This would include things like adopting a formal design methodology, implementing a Computer-Aided Software Engineering (CASE) tool, defining a standard development process, and establishing coding standards. In addition, the activities of a team mentor, coach, or process auditor are also prevention costs.

Does your project spend anything on Defect Prevention? Stop and think about it. Which of you project activities fall into this category?

In some organizations, groups that are classified as organizational overhead handle these costs, so you don't need to allocate project time or costs to them. But those organizations are rare. More commonly, Prevention activities are not being done by anyone.

If no one in your organization is addressing defect prevention, then your project would be served well by allocating some project time and cost to these activities. Why? We will discuss this in a minute, but first we must lay the foundation for that discussion.

Failure Costs

Failure costs are those costs we incur every time there is a failure. This includes the cost of documenting the failure, reproducing and diagnosing it, devising a fix for it, reworking the product to achieve the fix, reworking documentation so it is consistent with the fixed product, testing the fix to assure it works and doesn't have undesirable side-effects, and getting the fix out to those who need it.

Does your project spend anything on failure? Of course you do. There are always failures that we must deal with. A better question is, "Do you know how much you are spending on failures?"

Most organizations don't count these things as costs of quality. If they count them at all, the customer-related parts are counted as "support" costs, the developers' costs are counted as "engineering" cost, and only the retesting is counted as "quality" costs.

Failing to count all of these failure costs as costs of quality is the root cause of most organizations' problems with achieving product quality and controlling CoQ. Of all of the costs of quality, the failure costs are almost always the largest ones.

"Almost always"? Let's put it this way: If you are not counting your failure costs as costs of quality, then they are certainly your largest quality costs. Only organizations that measure failure costs and take action on that data drive them down to the point that they are surpassed by detection and prevention costs.

Organizations that do not pay attention to their failure costs are allowing the largest component of their CoQ to become invisible. And when that happens, not only is their CoQ uncontrollable, but they tend to have poor quality performance as well. That is, in spite of what they do, they consistently deliver poorer quality than they intend to.

Leveraging Your Costs of Quality

Since Failure costs tend to be our biggest quality costs, the only effective way to manage CoQ is to invest in the Prevention and Detection activities that will minimize our Failure costs. This is the measure we use to determine which quality-related activities are "worth doing"! Those that save more in Failure costs than they cost are worth doing because they reduce our Total CoQ!

Let's look at an example. Since we talked about the value of Peer Reviews (Defect Detection) last time in this column, (see "Placing the Quality Bet"), let's look at an example of investing in prevention.

We start by looking at the most expensive defects we are finding. Let's say that we are finding really expensive interface defects between the systems we build and our existing systems; either in final testing, or when we are deploying new systems, we encounter defects that cost us hundreds of hours to diagnose, fix, re-test and deploy. Let's say that this has happened on 60% of our projects, and the average interface defect found in final testing and deployment costs us 250 engineer hours and 10 calendar days.

With defects like these that cost hundreds of hours a piece, investing in defect prevention will yield a significant reduction in our Total CoQ. The expected failure costs we are experiencing on each project are 60% of 250 engineer hours (150 eng hrs) and 60% of 10 calendar days (6 days). If we can prevent those defects by spending less that those amounts of time, we will come out ahead!

A prevention strategy for this sort of defect might be to fully document the system interfaces on each project (which would cost an additional 4 engineer hours and half a calendar day). Then we could subject them to review by the technical leads of the systems being interfaced to (costing 40 hours and another half a day). Finally, we would expect to have to rework some of those interface designs to fix problems (costing an additional 4 hours and half a day).

So, by investing a total of 48 engineer hours and about a day and a half in Prevention CoQ, we can save 150 engineer hours and 6 days in Failure CoQ. That is a net CoQ savings of 102 engineer hours and 4.5 calendar days on every project from now on! Obviously, you would base your decisions about investing in Prevention (or additional Detection) activities on the actual Failure costs you are experiencing and your best estimate of what the Prevention or Detection activities will cost.

The last step is to measure your actual results to find out if the investment really was worth it. Did you reduce your failure costs? (e.g. Did a certain type of failure stop happening, or happen much less frequently?) And if so, did you save more in avoided failure costs than you invested? If you did, then Great! Keep doing it! If not, then look for another way to avoid that problem.

Quality Investment

Quality activities that are worth doing are those that are investments in the project. Investments must produce a return that is bigger than the investment, or they are not worth doing! So knowing what we should be doing quality-wise is a matter of running the numbers, and seeing what they tell us. Just as with any other part of pour project, Quality should always be managed by the numbers to be sure we're doing the right things!

©Copyright 2000-2018 Emprend, Inc. All Rights Reserved.
About us   Site Map   View current sponsorship opportunities (PDF)
Contact us for more information or e-mail
Terms of Service and Privacy Policy