Starting Simple in Performance Management

By By Raman A. Shah, Ph.D. from ICMA Blog

“We should measure impact, not output.”

Across the public and nonprofit sectors, every time I’ve suggested implementing operational reporting as a management tool, I’ve heard this rebuttal. I doubt I’ve ever waited more than 15 seconds to hear it. In local government, this mostly comes up in relation to strategic planning—writing a new strategic plan or tracking progress toward an existing one.

Output looks inward, at operations. As an example of output measurement, a city may count the feet of bicycle lanes it builds. Impact looks outward, at society. As an example of impact measurement, the city may track the growth in bicycle ridership.

It’s common sense that impact is what ultimately matters; a badly routed bicycle lane might be worthless. Unfortunately, impact is slow to appear and hard to measure. It depends on the same messy internal data that powers output measurement. I’ve found that measuring output with precision and regularity is practically a prerequisite to trustworthy impact measurement.

Impact measurement is slow

Impact can be slow to appear because services can take a long time to soak in and change lives. A bicycle lane might take years to achieve full impact as residents’ habits and possessions evolve to use it. That’s in addition to the time needed for analysis, so the lead time for impact measurement is generally long regardless of the tools at your disposal.

Contrast this with counting the feet of bicycle lane. Perhaps two construction companies are working on bicycle lanes, and you find that one is moving three times faster than the other. Is the faster vendor cutting corners, or is it more efficient? (Both? Neither?) Following up could deliver vital value for residents, days or weeks after work commences—if good output measurement infrastructure is in place.

Impact measurement is hard

In the meetings where it’s suggested to measure impact and not output, the idea is to run a survey before and after service delivery. The change in the results is the impact. Easy, right?

This doesn’t work because the world changes and doesn’t look the same everywhere.

If the world were perfectly static, the simple survey idea could work. But sometimes a professional cyclist becomes a national hero, and everyone buys a bicycle; sometimes they fall into scandal, and the bicycles start collecting dust.