Facts Evasion

When I’m not tweeting or blogging, I sometimes have to do some real work. Last year, I was asked to put together a report to analyse the performance of a particular strategy my company was pursuing and then present it to my CEO and the board.

“That sounds well important,” I thought. So I did this:

• I clearly set out what I was going to measure and why
• I gathered as much information as possible and analysed it
• I accounted for any uncertainties in my measurements
• I accounted for any external factors which could have influenced the results
• I summarised how the strategy was performing based on these factors and tied it back to the things I said originally that I would measure
• I made recommendations as to how we should proceed based on the gathered evidence and uncertainties

While I would like to think I was the first to think of this nice structured way of doing things, in fact I wasn’t. This is just, at a high level, the way you approach changing anything important, even if you’re doing it subconsciously.

If for example you were buying a house, you would not do it this way:

• Buy a house
• Gather as much information about houses as possible
• Selectively discard any evidence which suggested you’d bought the wrong house
• Selectively include any evidence which suggested you’d bought the right house
• Set out what your house buying criteria were based on the bits of evidence you had not discarded

Well, you wouldn’t do it this way unless of course, you were a politician. Politicians, you see, work in the opposite way to everyone else. Let me explain with an example.

At the moment, the government is proposing a reform of the NHS. This is the evidence they are putting forward for why such radical changes are necessary:

Health Secretary, Andrew Lansley, speaking on PM:

You are twice as likely to die of a heart attack in the UK as you are in France.

Prime Minister, David Cameron, on the same day:

We’ve fallen behind the rest of Europe. We spend similar amounts of money but we’re more likely to die of cancer or heart disease. I don’t think we should put up with a second rate… errrr… with coming second best.

Note he actually almost called the NHS a second rate health service but thought better of it.

Well it all sounds extremely scary. But is it?

John Appleby, Chief Economist at the Kings Fund, London thinks things aren’t quite as bleak as the government is making out and writing in the British Medical Journal he explained why.

Although statistics from the Organisation for Economic Cooperation and Development (OECD) confirm that in 2006 the age standardised death rate for acute myocardial infarction was around 19/100 000 in France and 41/100 000 in the United Kingdom, comparing just one year—and with a country with the lowest death rate for myocardial infarction in Europe—reveals only part of the story. Not only has the UK had the largest fall in death rates from myocardial infarction between 1980 and 2006 of any European country, if trends over the past 30 years continue, it will have a lower death rate than France as soon as 2012.

You see, the government selectively chose one data point on the graph and discarded the rest of the data. This is what the data looks like if you don’t discard the rest of it:

This doesn’t look quite so scary, does it? You see, when you look at all the available data, you get a very different picture. The UK’s death rates from heart disease have been plummeting and if the trends shown in the graph continue, argues Appleby, then we will have a lower rate than France by 2012.

This looks to me, (and John Appleby) like things are going in the right direction with our current system. Also, as Appleby points out, Lansley and Cameron chose quite a tough comparison. France has the lowest death rate from heart disease in all of Europe. A lucky choice, or cherry-picking a number to support an argument?

Either way – good for France, right? Well, not yet. Something Lansley and Cameron didn’t take into account which is absolutely massively important is this:

Are France and the UK using the same criteria to determine their causes of death?

As it turns out, possibly not. Speaking on More or Less on Radio 4 on 21st January*, Dr. Pete Scarborough, a Senior Researcher at the British Heart Foundation Health Promotion Research Group at Oxford University (get a shorter job title) noted that in the UK, if a person with a record of high cholesterol or high blood pressure dies, the coroners are much more likely to record the cause of death as a heart attack in comparison to France where the cause of such a death unless a heart attack is clearly shown to have happened, is generally recorded as “unknown”.

An additional point which Lansley and Cameron failed to mention is death from heart disease has a lot to do with lifestyle. The best way of avoiding death from heart disease is to avoid getting heart disease in the first place. Diet, smoking rates and exercise were all omitted from their conclusion.

Appleby in his article stated:

The trajectory for many causes of death swoops up and down over decades—often linked to changes in lifestyle behaviours rather than spending on healthcare.

Which makes perfect sense. Perhaps due to lifestyle, a higher proportion of people in the UK turn up for treatment for heart disease than they do in France. If that’s the case we should hardly be blaming the NHS.

You’ll recall, David Cameron also brought cancer death rates into the equation. On that, Appleby noted this:

As Cancer Research UK has pointed out, although the Eurocare data often feed headlines that the UK is the “sick man of Europe” for many cancers, trends from Eurocare actually show improvements in survival rates for the UK. These are confirmed by the Office for National Statistics, which last year reported improvements in five year survival rates between 2001-6 and 2003-7 for nearly all cancers. But Eurocare is problematic; the latest study includes diagnoses only up to 2002, and coverage is patchy (French data cover around 10-15% of people with cancer, the UK, 100%). Furthermore, differences in survival rates may reflect variations in how early diagnoses are made, not the state of healthcare in different countries.

Overall, I think again, this doesn’t seem quite as clear cut as we were led to believe but there is yet another important statistic they failed to mention. If our substandard healthcare is really causing a significantly higher death rate then shouldn’t our life expectancy be significantly lower than France’s? I went to the World Health Organisation’s website to check this out and found this:

The most recent figures for average life expectancy they have are for 2008. In France it was 83 for women and 81 for men. In the UK it was 82 for women and 80 for men. Irrespective of what people are finally dying of, the average life expectancy between the two countries is extremely close.

The WHO also conveniently shows average life expectancy by global region and here we can see whether we are lagging behind the rest of Europe, as claimed by David Cameron:

Oh. As you can see, the average for Europe for 2008 was 79 for women and 71 for men.

So much for us lagging behind the rest of Europe but, anyway, if we truly are experiencing significantly higher death rates from heart disease and cancer than the rest of Europe it is a bit puzzling. If they aren’t dying of that but have lower life expectancy then what exactly is it that they are all dying of? Rabies?

My conclusion based on this data (which was by no means all of it but a lot more than Lansley or Cameron used) would be that we are doing well and heading in the right direction. So where is the big problem? Where is the big need for a radical NHS reform?

Let me move back to my initial example of the report I had to do at work. If I had made a conclusion in advance, backed it up with one or two pieces of cherry-picked data and gone in front of the board with it, my audience would have torn me to shreds.
My report was important to me but compared with the importance of the NHS to the UK it was really nothing at all. This is the NHS we’re talking about – people’s lives.

My audience wouldn’t have accepted such a poor analysis of the data for my crappy report – so don’t you dare do it either. You are the audience now and this is really important. Don’t accept these arguments as the complete story – understand them for what they are. A couple of cherry-picked, airbrushed, bullshit numbers selected to back up a plan that the government wanted to do anyway, regardless of the what the actual evidence suggested.

If I couldn’t get away with such behaviour in front of my board then why should the government get away with it in front of the British public?

Pulling basic information from the British Medical Journal and the World Health Organisation is not hard to do – I did it with no trouble at all. The government’s position is nothing more than pure deception and when we look at the real evidence – their position fucking stinks.

RedEaredRabbit

*More or Less’s excellent piece on this same subject helped a lot with putting this post together. It’s available as a free Podcast so have a look for it.