Measuring the "Hard Stuff"

Every organization has it...hard-to-measure work. It may be the intangible "employee morale", or an "elegant" engineering design, or an "innovative" marketing plan. But just because it's hard-to-measure doesn't mean it can't be measured.

Resistance to Measurement

One reason measurement is hard is because there can be resistance to measurement in the first place. In 20+ years of helping people figure out how to measure performance, the most common sources of resistance seem to include:

"My job is creative" (and therefore can't be measured)
"I don't make standard widgets" (so I can't be measured)
"I'm a professional" (so I can't be measured)
"I don't want to be measured" (because if you could measure my performance you'd realize I haven't been doing my job very well lately, and I may have to change, and I don't want to...)

Missing Starting Point

If resistance isn't a problem, measurement may be hard because you don't know where to start.

ZPG's Results Based Measurement process addresses that by guiding you to start with your organization's goals and your customers' needs, followed very quickly by a definition of the value-added results you need to product to support these goals and meet these needs.

If there are no clear organizational goals, then you stick with the customers of the position/department/function/division. If there aren't any customers either, than you'll have to think long and hard about why you're measuring in the first place.

Only Using Numbers to Measure

If you know where to start, the next sticking point tends to be trying to measure only using numbers. The underlying assumption seems to be that quantified measures are inherently more objective, and thus better, than any other kind of measure. This kind of thinking is limiting and can result in measuring what is easily quantifiable but not what is most important.

For example a report could be measured by it's size, weight, number of pages or even number of words. But in most cases, none of these easily measured aspects are as important as the "usefulness" or "clarity" of the report.

Creating Descriptive Measures

Where numbers don't make sense, ZPG recommends using what it calls "descriptive measures." Descriptive measures include two components, a judge and a list of factors.

In the case of a "useful report", the judge may be the person who requested the report or the person who needs to make a decision based on the report's contents. The list of factors important to these two judges might be things like:

Adequate information for making a decision
Justification supporting any conclusions
Completeness
Relevance to the original question the report was to address

Quantifiable vs Verifiable

A problem related to using only numbers is the assumption that quantification is the only objective way to measure, and objective is always better than subjective.

As mentioned above, the easy to quantify may not be the best measure. But how do you "measure" something like "adequate information" in an objective manner?

First, give up on trying to be objective. A more useful construct is "verifiability", for two reasons. First, as soon as you set a goal even the most objective measure becomes subjective. For example, a goal of 10% profit is a subjective judgement. Why not 20% or even 80% profit (especially if you're in the software business). Why not 2% or 3% (if you're in the grocery business?) The goal is subjective because it is set by a human being, even if it is based on relatively objective numbers.

Second, by shooting for "verifiability" you leave the door open to measurement techniques other than using numbers. It's easy to verify that you've met a goal or not if you're using numbers. By creating verifiable measures, can tell if a goal has been met or not without numbers.

For instance, adequate information for a decision may include

list of the options considered
list of pros and cons for each option
values important to making the decision
relative importance of each value
a recommended decision with reasons for the recommendation

You can tell if the report contains "adequate information" by answering yes or no to the presence of each of the above verifiable factors.

Verifiable measures give you a way to measure those aspects of performance for which numbers do not work well.

For more information on Measuring the Hard Stuff, see any of the following:

Article on Measuring the Hard Stuff
A CDROM or internet-viewable narrated PowerPoint Presentation on Measuring the Hard Stuff
How to Measure Employee Performance
How to Measure Team Performance
GoalTutor™