December 16, 2014 | Featured, Issues

by Besiki Kutateladze

In an article titled “Performance Indicators: How to Lie with Indices,” the Economist was right in certain respects: international indices are becoming increasingly popular and many of them are methodologically flawed. But the article neglected to acknowledge how one can build more reliable performance measures—and such measures do exist.

First, indicator developers should avoid producing indices in isolation, typically in well-equipped western offices. In fact, understanding local context, including political, legal, economic, and cultural climates, is essential for producing such indicator tools. Second, engaging local stakeholders, and particularly local community members, is vital for success. Third, given that no data source is perfect, combining multiple data sources can help overcome the unavailability and unreliability of data. Fourth, indicator tools need to be piloted and necessary revisions made before findings are reported.

All of this requires a lot of time and costs money. Funders must be convinced—and many are already becoming more aware of this—that sufficient resources need to be dedicated to identifying and overcoming the methodological challenges of indicator development. It may not be as sexy as Angelina Jolie’s endorsement of the indicators tool—in fact it’s probably not sexy at all—but without going through the proper steps for building indicators, such tools are likely to achieve the very thing they were intended to overcome: a distorted view of reality.

Besiki Kutateladze is ISLG’s Associate Director for Research. He is one of the architects of the UN Rule of Law Indicators Index, and he the author of the UNDP Guide on Performance Measurement.