Mark Twain wrote in his autobiography; “There are three kinds of lies: lies, damned lies, and statistics.” He gives credit to Benjamin Disraeli for the insight, so I’ll credit both as the inspiration for my article today. It’s something seen frequently in business planning and strategy sessions; stakeholders use the persuasive power of statistics – particularly percentages – to bolster a weak argument.
When delivered with authority, percentages can easily disguise the reality of a situation – and I know that 100% of the people reading this will agree or disagree with me on that point…
An article on the BBC News website caught my attention recently for this very reason. It was about how long you would have to wait to be seen by a specialist in different areas of the country. As part of the article, I was able to put in my post code and find out how my local health services were performing compared to others across the UK. This ‘NHS Tracker’ has now been reused by the BBC in several articles in the last month and the more the journalists query the data, the flimsier their conclusions are becoming. Why? Because there is a flaw in their KPIs, and it’s a common one I come across again and again. Percentages presented as answers, when they are actually masking the real results.
In these articles, the data they are quoting comes from a list, benchmarking every hospital in the UK against the same three targets, and ranking them on the percentage of each target reached. For example, one target is how many A&E patients were seen within 4 hours.
The BBC Health Correspondent Nick Triggle sings the praises of Luton and Dunstable Trust as being ‘head and shoulders above the rest’ having achieved all three key targets throughout the year. He then quietly mentions that this medium-sized hospital benefits from its size: “as it is not a large regional centre, which means it will avoid some of the most complex cases.” And there is the rub.
So, if we simplify this – one A&E could have 4 staff see 2 patients per day and hit 100% of the target, whereas another may have 200 staff and see 1,200 patients per day, and hit 92% of the target. The latter would be ranked as a much poorer hospital by these measures. In fact, if these were departments in a business, and a manager was making decisions on where to make cuts based on no information other than these measures, some very poor decisions could be made.
So many factors need to be taken into account in order to accurately benchmark these services: the number of staff; the type of staff; the frequency of patient visits; the overall number of patient visits; times of day when the targets are least likely to be reached; the severity of ailments on each visit…and that’s only a few off the top of my head.
As Stacey Barr, founder of PuMP, says “Percentages are great when you are interested in how much of a target population matches your performance result. But percentages assume your result is black or white. Either customers are satisfied, or they aren’t. Employees either had an accident at work or they didn’t. They don’t tell you the degree or extent, though, like how satisfied, or how injured.”
The problem with using percentages as markers of performance is that they often mask the real, or relevant information in exchange for a ‘palatable’ statistic. Just because it makes a great soundbite, doesn’t mean it’s a great KPI!