Why Statistical thinking is essential to choosing KPIs

Following on our recent theme of the importance of applying statistical methodologies to your Performance Measurement, here’s an article by PuMP founder Stacey Barr from back in 2012 that I think explains the concept very well:

“Statistical thinking is not so much about having knowledge of and being able to apply statistical techniques. It’s NOT about knowing how to perform a regression analysis or knowing the formula for putting a trend line through a time series.

You DON’T have to be a statistician or a numbers person to master statistical thinking. Statistical thinking is simply about understanding a few basic concepts:

Statistical Thinking Concept #1: All things vary.

Everything varies, or goes up and down seemingly erratically, due mostly to complexity and the interactions among many causal factors.

Sales vary because of the economy, marketing message design, amount of marketing activity, what’s happening in the lives of your target market, even the weather.

Workplace accidents, office supply expenses, satisfaction of customers, hours of rework people do each week, how late or early deliveries are, how much overtime is worked, your weight from day to day: everything goes up and down with random but very natural variation caused by all the complex interplay of factors that affect it.

Statistical Thinking Concept #2: Because things vary, there is uncertainty.

We can’t ever really know something with 100% complete knowledge. Statistics is not like mathematics, where you get exact answers when you combine numbers.

Statistics is the study of uncertainy, and its core purpose is to draw patterns out of data.

We don’t know exactly how many sales we’ll get next week because we don’t know how all the causal factors will play out.

Statistical Thinking Concept #3: The key to knowing something is to find measures of uncertainty, to make signals stand out.

We can look to the past to see how MUCH sales have varied from week to week so we can estimate next week’s sales within a likely RANGE.

This is why the concept of variation is fundamental to statistics. Variation measures the uncertainty. This routine variation is fundamental to how we can draw knowledge from data because it helps us gauge the amount of uncertainty inherent in whatever it is we want to measure and manage.

And what we’re managing is the pattern of variation, not the points of data!

The implication for us, and for performance measurement, is that we CANNOT find knowledge in individual points of data.

Knowledge can come only from patterns in data. And these patterns are patterns of variation. If the variation reduces or increases or moves, it generally is a signal that something happened to cause a change.

Sometimes when the pattern of variation DOESN’T change, it is also a signal that our efforts are having NO effect!

You cannot manage business performance without statistical thinking – when you ignore variation and uncertainty, you react to every fluctuation as though it means something significant happened.

But far more often than not, NOTHING significant happened! Those fluctuations are just a natural product of complex and interrelated causes.

So how do we know when something significant happened in a KPI?

How do we know when we need to take action to improve performance? We need to distinguish the routine and natural variation in our performance data from the abnormal or non-routine variation that signals a change.

And it’s easy to do this – even though most people don’t even know about it. It takes just your KPI data, a few easy statistical calculations, and one simple graph that I am now calling a Smart Chart. (They are technically known as XmR charts.)

0 Comment

Scroll to Top