On the Tyranny of Measures

Image Source: Photo by Austin Distel on Unsplash

**

One of the (admittedly dubious) benefits of having worked for the better part of twenty years in one industry is that my work has now taken in arguably the two largest companies in my industry. At the first one, I was a green-around-the-ears graduate engineer, fresh out of University and excited about the world. At the second, I was a hardy, more cynical “ex-pat”, with the benefit of having weathered work in different cultures in my back pocket. On the surface, both companies could not have been more different: both were in very different local environments steeped in centuries of traditions, one had a reputation for being a slow, lumbering supertanker only adopting cutting edge technology once it was proven elsewhere whilst the other was - and still is to some degree - held up as a paragon of world class performance. With the benefit of hindsight, that perceived dissimilarity was only skin deep; at their core both companies were beholden to an extreme focus on measures of performance, with everything from employee ratings to building reorgs being assiduously measured and monitored in the name of continuous improvement.

In thinking about how both companies could be so similar in spite of the powerful local influences, a number of possible causes come to mind. One consideration is the strong (some might say out-sized) concentration of American Ivy League-type MBAs in the upper and middle echelons of management. Often young relative to the Technical SMEs who delivered actual work, these necktie-wearing, business types - eager to perhaps prove their worth - were more than happy to trot out whatever would get them noticed. Another possible contribution - for different reasons in both contexts - is something Will Storr’s fascinating book calls the Status Game. At the first company, existing as it did within the context of a loose culture, the predominant games seemed to be ones of success, with perceived high-fliers rocketing up the corporate ladder and getting plum assignments every couple of years or so. The tried and tested - ideally by someone else - was as near a guarantee of success as there could be, hence the optimization towards safe, slow change. The wider societal context of a tight culture at the second lent itself more to prestige games. Here the optimization seemed to be towards new, shiny, exciting stuff that could be highlighted upwards as a means to gain the attention of those higher up the status hierarchy.

Of the host of metaphors which exist for organizations - as organisms, as systems with varying degrees of openness, and machines to name a few, I find that of a complex adaptive system the most useful for trying to understand how those two organizations have evolved. At the risk of oversimplification, one may theorize the prevailing culture of any organization as being a dynamic relationship between the wider societal norm in which it exists (tight vs loose), the stated culture of the organization and the behaviour of agents (employees and stakeholders) who relate with it both internally and externally. Marrying this with the idea that we all are involved in status games, one may further theorize that individual agents (of differing influence within the organization) optimize their strategy for different status games (prestige, dominance, virtue, success etc). Taking all of the above into account, it seems reasonable to proffer a third reason for why KPIs and measures are as ubiquitous as they are: organizations are intrinsically complex, and a good KPI is often seen as our best shot at introducing some objectivity into determining performance of both agents and the wider organization.

Circling back to the subject of the tyranny of measures, two things stand out as to why they can be especially dangerous. One is most succinctly captured by Goodhart’s Law, which is perhaps Heisenberg’s Uncertainty Principle for KPIs. Within this framework, the degree to which a target is attained and its usefulness are conjugate variables, a conclusion which is somewhat inevitable once the organization as a complex adaptive system framing is accepted. Of equal importance, at least, is the preponderance of power law type distributions for a number of the things KPIs are usually defined to mitigate such as preparedness for significant adverse effects. Here, the attainment of a target can provide a false sense of security, when in reality it is the relative rarity of the event ostensibly being mitigated that has prevented it from being realized. If there is a moral to this, it is perhaps that the most important time to be vigilant is when a KPI is “green”.