StratChat 20 August: KPIs

StratChat is a weekly virtual networking event for business strategists and anyone with an interest in developing and executing better business strategies. It is an informal community-led conversation hosted by StratNavApp.com. To sign up for future events click here.

StratChat is conducted under Chatham House Rules. Consequently, the summary below is presented without any attribution of who said what. As such, they are an agglomerate of views - not necessarily just those of the writer or indeed of any one individual.

This week's conversation focused on: Strategic KPIs why are they so important and why are they so hard?

Topic introduction

Strategic KPIs are a powerful tool for articulating strategy, for driving execution and for sensing the need for change. Get the right and the rewards are significant. But get them wrong and the consequences can be dire.

We've all heard mantras like:

• If you can't measure it you can't manage it.
  and
• You get what you measure.

If those are to be believed, and if we want to 'manage' our business strategies, and 'get' their outcomes, then measurement, in the form of KPIs must be a key part of our process.

And yet, not everyone is convinced. Measurement and KPIs remain problematic in many organisations.

Too much, yet not enough

We're generating data faster than ever before. Sometimes it feels like we're drowning in it.

But do we have the right data? And are we using it effectively?

Most of us are familiar with the concept of a car dashboard. It comes fairly naturally to us. Some car dashboards are relatively simple affairs. Others are more complex beasts. But for all dashboards, the objective is the same: to convey the necessary information without overwhelming the user. There is an elegant simplicity to a well-designed and effective dashboard. This should be our goal in setting strategic KPIs.

Car dashboards present a snapshot view of the relevant data at a point in time. Strategic KPI dashboards, however, should present the data over a period of time. (Typically a line chart with time on the x-axis, for example.) This better enables us to see the trends and spot any potential trend breaks.

Not all KPIs are created equal

There is a distinction between operational KPIs and Strategic KPIs. Operational KPIs tell you if the business is running smoothly. Strategic KPIs tell you if its strategy is working.

Many organisations are awash with financial KPIs. However, these tend to be backwards-looking. Coming up with forward-looking KPIs is inherently harder. But at least some of your strategic KPIs need to be forward-looking.

One way to do this is to surface the logic behind your strategy. This could be in the form of: by doing X we will achieve Y which in turn will achieve Z. Z will most often be a lagging indicator, and often a financial one. X will give you a clue as to what your leading indicators might be. Looking at KPIs in this way has the added advantage that it will help to confirm or disconfirm the beliefs underpinning your strategy.

The Balanced Scorecard is also a useful framework for ensuring a balance in your Strategic KPIs across a number of dimensions. Other frameworks, like OKR also provide structure. StratML is the ISO standard (ISO 17469-1) for strategy and performance plans (and which is baked into StratNavApp.com). It provides a clear hierarchy from vision/mission through goals, objectives, KPIs and to targets. So it incorporates much of the OKR approach.

The range of different standards used in organisations does make it harder to advise them. The goal should be to come up with something which is elegantly simple.

A maelstrom of anecdotes

The conversation was a maelstrom of anecdotes of what people had seen work well, and sadly more frequently what had not worked well.

• A city (supposedly in India) had a problem with snakes. Officials hit upon a plan of paying citizens to catch, kill and bring the snakes to the town hall. Entrepreneurs quickly worked out it was easier to breed snakes than to catch them. So the city was forced to cancel the programme. The entrepreneurs, no longer needing their snakes, released them. The problem was worse than ever.
• A web developer realised that removing the "cancel subscription" button from the website significantly increased its 'customer loyalty' measure.
• The new head of a central reporting function thought his team was producing many too many reports. So he simply stopped. End users only complained about not receiving a small percentage of the reports previously produced.
• A person producing a complex management report admitted they did not know on what basis the information they used as an input had been prepared. Nor did they know for what purpose the report they produced was being used and how it was being interpreted. As a result, they could not confirm that the analytical process they applied was still correct and relevant. But they continued to produce the report because that's what they'd always done.

(Recounting of specific anecdotes has been limited here under Chatham House Rules.)

Some typical problems

Typical problems encountered in organisations include:

1. Not having the right data: some organisations just don't collect the right data. Especially if an organisation has just changed strategic direction it is unlikely to have been collecting the data required to track the performance of its new strategy. Before that was its strategy, what need would it have had for that data?
2. Having the wrong data: organisations often simply focus on the data that is readily available or easily collected rather than what they really need. This often manifests itself in an over-reliance on financial data as that has to be collected to comply with accounting and reporting standards anyway.
3. Misinterpreting the data: using inappropriate analytical techniques or unsound statistical analysis.

The intelligence cycle

In military and intelligence communities have clearly define in an intelligence cycle: collection, fusion, analysis and action (which then usually generates more collection requirements). There are data collectors who gather intelligence from various corroborating and non-corroborating sources. This then has to be analysed before it can be turned into actionable intelligence.

There is a lot of raw data. You have to have smart people to be able to make sense of it.

Not all organisations are created equal

Your strategic KPIs need to be appropriate to the nature of your business. There is no one size fits all. For example:

• Some of the biggest fasted growing organisations in recent times, like Amazon, have been very successful in using operational data to drive phenomenal growth. The story is told that Amazon chose to start with books before expanding into almost everything else because books provided the easiest way to test their data-centric approach to business. They're also a culture of engineers - data is 'the water they swim in'. Many legacy organisations have been caught out by these new business models and are struggling to catch up. Part of the challenge is they don't have the same culture and mindset.
• Startups are typically more focused than more mature businesses. This makes it easier to discern appropriate KPIs. In fact, many startups are part of incubator, accelerator or funding programmes which more or less dictate the content and style of their reporting, including their KPIs. Some private equity companies also follow similar approaches. So their KPIs become fairly uniform. It takes a brave and committed entrepreneur to step outside of that prescription.
• More mature organisations have often lost their way and lack the level of focus found in startups. And the organisations are often larger and more complex. Defining a clear set of KPIs would be part of their process of regaining focus.
• More socially motivated organisations are often dealing with more complex, sometimes intractable issues. Outcomes can be very multi-dimensional and difficult to measure. Unintended consequences can be significant. But KPIs can help us to have more deep conversations as they stop us from talking about our goals and desired outcomes in woolly ill-defined terms.

The double-edged sword

The power of KPIs means they are a double-edged sword. The can be a very powerful tool for articulating and executing business strategy (amongst other things). But they can also be misused and abused.

Some misuse may be unintentional. But in other cases, KPIs could be abused and data manipulated to advance or buttress personal, political (including as in corporate politics) or simply unsavoury goals.

Where KPIs are attached to incentives, this may encourage people to misreport data. For example, health and safety-related data may go unreported or overtime may get recorded as something else. What started off as a well-intentioned KPI can turn into something else.

There is quite a lot research about the impact of linking KPIs to performance incentives. For example, some research (for example reported by Daniel Pink) suggests it works well for very repetitive mechanical tasks, but that it can actually reduce creativity in roles where that is required.

Artificial Intelligence

Will artificial intelligence (AI) enable more effective use of KPIs, particularly in complex situations?

One of the challenges in AI is with bias. So if there is inherent bias in the KPIs, that may simply be replicated in the AI.

AI will inevitably help us solve more fuzzy problems as it matures. But you will always need some human interaction, especially to deal with fuzzy human problems.

Quantitative versus qualitative analysis

KPI data will tell you exactly what is going on in terms of the metrics you've defined. It is all based on observable behaviours. Qualitative research can give you more insight into why it is happening. It can also help you to work out what metrics you've overlooked or misunderstood.

Conclusions

So where did we end up after an hour of lively discussion? Strategic KPIs are powerful and critical. But much too complex to completely address in only an hour. Definitely a subject we'll have to come back to for further discussion. Probably several times.

Next week

After a brief discussion, we resolved that next week we will look a the Business Model Canvas, and variants thereof, such as the Lean Canvas. So please join us to talk about what they are, and what they're good for.

Administrative issues

Apparently the Eventbrite process for actually joining the meeting is less than intuitive. Some attendees reported difficulties. If you are one of those who had signed up to the session but were unable to join for this reason, I do apologise. I suspect that other platforms will have similar issues, but if you know of a better approach, please let me know.

In the meantime, the Zoom login should be the same every week. So it may be easier to simply put a recurring appointment in your calendar with the link in it so that it is ready whenever you are able to join us. I will also try and email the link to everyone who has signed up about 30 minutes before we start.

Attendees (across 3 continents): Chris Fox (host and notes), Azfar Haider, Christopher Sable, Craig Rattray, Hugh Pierre, Jerald Welch and Philip Hodges.