Megan McArdle shares some stories of metrics and their unintended consequences:
In December, doctors at a VA hospital in Oregon decided to admit an 81-year-old patient. He was dehydrated, malnourished, plagued by skin ulcers and broken ribs — in the medical professionals’ opinion, he was unable to care for himself at home. Administrators, however, overruled them.
Was there no bed for this poor man? No, the facility had plenty of beds; in fact, on an average day, more than half of the beds are empty, awaiting patients. Was there no money or medicine to care for him? No, and no. Reporting by the New York Times suggests that Walter Savage was, perversely, turned away because he was too sick. Very sick patients tend to worsen the performance measures by which VA hospitals are judged.
If this had happened in isolation, we could simply gape at the monstrosity that bureaucracies are occasionally capable of.
But such examples abound in health care. For example, in the 1990s, New York and Pennsylvania started publishing mortality data on hospitals and surgeons who did coronary bypasses. The idea was that more informed consumers would steer themselves toward the teams with the better statistics — theoretically good for patients, bad for slacking providers. The reality was less ideal: In those states, surgeons seem to have started doing more operations on healthier patients, while turning away the sickest ones who might otherwise have benefited.
From this we can take a few lessons. The first is one that has been well-known to other sorts of businesses: What you measure is what you get, not necessarily what you want. In fact, if your measurement is badly designed, you may get a great deal of something you don’t want.
To illustrate that, look at Wells Fargo, which recently paid a whopping fine because a badly designed compensation system encouraged low-level employees to muck around with customer bank accounts. These machinations generated effectively no revenue for the bank, and annoyed customers, but they did generate income for the employees — and eventually, a stinging, expensive rebuke from the Consumer Financial Protection Bureau.
[...]
I could reel off examples endlessly: purchasing managers who have cozy arrangements to buy a certain amount of product from their vendors in December, and ship it back in January, in order to help some sales director make quarterly targets … universities that compete to turn away as many students as possible, because doing so makes them rise in the U.S. News rankings … law schools that hired their own graduates for temporary make-work jobs in order to boost the schools’ employment statistics. All metrics will be gamed, and the games always have costs. And when the metrics involve our health, those costs can be very high indeed.
Health care and education are particularly ill-suited to management-by-measurement:
Most companies are dealing with reasonably standardized inputs, which can be turned into measurable outputs. But the less you deal with things, and the more you deal with human beings, the less useful productivity metrics are. Human bodies and human minds are both highly variable and immensely complicated. When you are working on them, it is hard to know how much of the final result is a result of your labor, and how much can be credited to the qualities of your initial starting material.
So when we measure outputs, we are getting at best a very distorted picture of the value of the services provided. Modern industrial management is simply not designed for this sort of situation. If you feed human inputs into a machine system, you are quite likely to grind up the humans in the process.