KPIs and How Outcome-Oriented Teams Work

I had a conversation with a client today who correctly pointed out that KPIs can be leading or lagging indicators. He was disagreeing with some of our materials, where we treat KPIs as explicitly leading (and not lagging) indicators mapped to the success criteria of an epic.  Maybe in a little bit different of a writing tone than I normally use, here’s how I responded to our client – who I will call “Mark” (not his or her real name).

Hey Mark,

Thanks very much for initiating some great conversation. I also appreciate it as feedback to remind me that the diagram does need supporting context, to be able to express the key elements of the information architecture differently to support different clarifications and conversations.

I’d like to take your comment and break it down into two topics:

1. KPIs can be both leading and lagging indicators.
2. KPIs are non-financial measures of performance.

Here’s the short version of the long answer that follows. Completely agree with your two points. The diagram is intended to show this exactly, but clearly requires additional explanation of other elements of the design of the system to better reinforce this understanding instead of visually undermining it somehow.  

I’d like to touch on point 1 briefly, then dive into point 2, then come back to point 1 (because it resonates better for me when I have both topics in my hands at the same time).

KPIs can be both leading and lagging indicators.

On leading and lagging: all things can be either leading or lagging, depending on your perspective. A couple examples: 

• Does the code compile correctly?
  • This is a lagging indicator to the developer who typed the code, and is trying to avoid bugs. (“Lagging measure – bugs were created already”)
  • This is a leading indicator to the development team who is about to launch the code into production. (“Leading measure – if bugs had been created, problems are likely to arise”)

I like this example a lot because leading indicators do not describe certainly, they can only describe probably. A lack of compiler-bugs is not a guarantee of a problem-free deployment; a deployment without compiler-bugs is probably going to be better than one with compiler bugs.

Every leading indicator leads TO something, without assuring the inevitability of something. If it were inevitable, we would just measure the most convenient measurable thing. More on this after I dig into your second point.

KPIs are non-financial measures of performance.

Again, on this, we agree.

There is a key element to how the teams need to organize their work – and organize their thinking about their work, and that is the element of uncertainty.  In a project-centric, output-oriented world, teams make commitments and predictions about their outputs which are completely within their control.  For a whole host of reasons, their predictions are terrible – this system simply is not trustworthy. Situations change (and people’s understanding of the situation changes) after commitments were made. So they are always poorly made.

To escape this trap, we help teams shift to a mindset of causal relationships underlying the nature of all the work they do. We focus on two causal relationships specifically: 

First, anything the team builds is being built to solve a problem which results in a change in behavior, which results in an improvement in performance. Very specifically, there are (relatively) immediately observable changes as a result of the work the team did. When the team chooses to do some work over other work, their decisions are framed within this cause-and-effect relationship. Is this the best way to achieve the intended improvement in performance?

Second, every change in performance the team attempts to create is intentional – because the team believes this observable measurable (non-financial) performance change will result in a financial benefit to the organization. Very specifically, for any performance/behavior change we can create and observe right now, it is a leading indicator of probably realizing a lagging financial benefit. This benefit is also measurable. Unfortunately, we cannot measure it quickly enough for the teams to adapt. From the point of view of the portfolio team, this financial indicator is lagging. Because by the time we see the financial benefits or lack thereof,  it is too late for the portfolio team to coordinate a course-correction.

The KPI is initially the measurement of the change we are trying to create, and ultimately the thing we are measuring to tell us if we probably going to realize the intended benefit. This is both strategic alignment (before we “do”) and course-correction (while we are “doing”). The two are key elements of achieving business Agility.

KPIs can be both leading and lagging indicators.

So, yes, every KPI is from the perspective of the product team a lagging indicator. They do stuff to try and cause a change (which, by definition, lags).

From the perspective of the product line team, what is useful is to only select KPIs which are leading indicators relative to the desired outcomes they are trying to achieve. When we work with the teams, we help them choose KPIs which are useful. Which means they need to be “leading” relative to the outcomes the investments are intended to drive.

Within the context of each epic as a container for an investment decision in support of the business case, the KPIs – the success criteria – are only useful insomuch as they provide the teams with feedback that allows them to course-correct. If they pick a KPI that won’t resolve for a year, they won’t get the benefit of the feedback loop. And by definition, it implies an even longer time-frame for realizing the financial benefit we believe will result from the change in performance.