Facts Evasion

When I’m not tweeting or blogging, I sometimes have to do some real work. Last year, I was asked to put together a report to analyse the performance of a particular strategy my company was pursuing and then present it to my CEO and the board.

“That sounds well important,” I thought. So I did this:

  • I clearly set out what I was going to measure and why
  • I gathered as much information as possible and analysed it
  • I accounted for any uncertainties in my measurements
  • I accounted for any external factors which could have influenced the results
  • I summarised how the strategy was performing based on these factors and tied it back to the things I said originally that I would measure
  • I made recommendations as to how we should proceed based on the gathered evidence and uncertainties

While I would like to think I was the first to think of this nice structured way of doing things, in fact I wasn’t. This is just, at a high level, the way you approach changing anything important, even if you’re doing it subconsciously.

If for example you were buying a house, you would not do it this way:

  • Buy a house
  • Gather as much information about houses as possible
  • Selectively discard any evidence which suggested you’d bought the wrong house
  • Selectively include any evidence which suggested you’d bought the right house
  • Set out what your house buying criteria were based on the bits of evidence you had not discarded

Well, you wouldn’t do it this way unless of course, you were a politician. Politicians, you see, work in the opposite way to everyone else. Let me explain with an example.

At the moment, the government is proposing a reform of the NHS. This is the evidence they are putting forward for why such radical changes are necessary:

Health Secretary, Andrew Lansley, speaking on PM:

You are twice as likely to die of a heart attack in the UK as you are in France.

Prime Minister, David Cameron, on the same day:

We’ve fallen behind the rest of Europe. We spend similar amounts of money but we’re more likely to die of cancer or heart disease. I don’t think we should put up with a second rate… errrr… with coming second best.

Note he actually almost called the NHS a second rate health service but thought better of it.

Well it all sounds extremely scary. But is it?

John Appleby, Chief Economist at the Kings Fund, London thinks things aren’t quite as bleak as the government is making out and writing in the British Medical Journal he explained why.

Although statistics from the Organisation for Economic Cooperation and Development (OECD) confirm that in 2006 the age standardised death rate for acute myocardial infarction was around 19/100 000 in France and 41/100 000 in the United Kingdom, comparing just one year—and with a country with the lowest death rate for myocardial infarction in Europe—reveals only part of the story. Not only has the UK had the largest fall in death rates from myocardial infarction between 1980 and 2006 of any European country, if trends over the past 30 years continue, it will have a lower death rate than France as soon as 2012.

You see, the government selectively chose one data point on the graph and discarded the rest of the data. This is what the data looks like if you don’t discard the rest of it:

Heart Disease MortalityThis doesn’t look quite so scary, does it? You see, when you look at all the available data, you get a very different picture. The UK’s death rates from heart disease have been plummeting and if the trends shown in the graph continue, argues Appleby, then we will have a lower rate than France by 2012.

This looks to me, (and John Appleby) like things are going in the right direction with our current system. Also, as Appleby points out, Lansley and Cameron chose quite a tough comparison. France has the lowest death rate from heart disease in all of Europe. A lucky choice, or cherry-picking a number to support an argument?

Either way – good for France, right? Well, not yet. Something Lansley and Cameron didn’t take into account which is absolutely massively important is this:

Are France and the UK using the same criteria to determine their causes of death?

As it turns out, possibly not. Speaking on More or Less on Radio 4 on 21st January*, Dr. Pete Scarborough, a Senior Researcher at the British Heart Foundation Health Promotion Research Group at Oxford University (get a shorter job title) noted that in the UK, if a person with a record of high cholesterol or high blood pressure dies, the coroners are much more likely to record the cause of death as a heart attack in comparison to France where the cause of such a death unless a heart attack is clearly shown to have happened, is generally recorded as “unknown”.

An additional point which Lansley and Cameron failed to mention is death from heart disease has a lot to do with lifestyle. The best way of avoiding death from heart disease is to avoid getting heart disease in the first place. Diet, smoking rates and exercise were all omitted from their conclusion.

Appleby in his article stated:

The trajectory for many causes of death swoops up and down over decades—often linked to changes in lifestyle behaviours rather than spending on healthcare.

Which makes perfect sense. Perhaps due to lifestyle, a higher proportion of people in the UK turn up for treatment for heart disease than they do in France. If that’s the case we should hardly be blaming the NHS.

You’ll recall, David Cameron also brought cancer death rates into the equation. On that, Appleby noted this:

As Cancer Research UK has pointed out, although the Eurocare data often feed headlines that the UK is the “sick man of Europe” for many cancers, trends from Eurocare actually show improvements in survival rates for the UK. These are confirmed by the Office for National Statistics, which last year reported improvements in five year survival rates between 2001-6 and 2003-7 for nearly all cancers. But Eurocare is problematic; the latest study includes diagnoses only up to 2002, and coverage is patchy (French data cover around 10-15% of people with cancer, the UK, 100%). Furthermore, differences in survival rates may reflect variations in how early diagnoses are made, not the state of healthcare in different countries.

Overall, I think again, this doesn’t seem quite as clear cut as we were led to believe but there is yet another important statistic they failed to mention. If our substandard healthcare is really causing a significantly higher death rate then shouldn’t our life expectancy be significantly lower than France’s? I went to the World Health Organisation’s website to check this out and found this:

The most recent figures for average life expectancy they have are for 2008. In France it was 83 for women and 81 for men. In the UK it was 82 for women and 80 for men. Irrespective of what people are finally dying of, the average life expectancy between the two countries is extremely close.

The WHO also conveniently shows average life expectancy by global region and here we can see whether we are lagging behind the rest of Europe, as claimed by David Cameron:

Oh. As you can see, the average for Europe for 2008 was 79 for women and 71 for men.

So much for us lagging behind the rest of Europe but, anyway, if we truly are experiencing significantly higher death rates from heart disease and cancer than the rest of Europe it is a bit puzzling. If they aren’t dying of that but have lower life expectancy then what exactly is it that they are all dying of? Rabies?

My conclusion based on this data (which was by no means all of it but a lot more than Lansley or Cameron used) would be that we are doing well and heading in the right direction. So where is the big problem? Where is the big need for a radical NHS reform?

Let me move back to my initial example of the report I had to do at work. If I had made a conclusion in advance, backed it up with one or two pieces of cherry-picked data and gone in front of the board with it, my audience would have torn me to shreds.
My report was important to me but compared with the importance of the NHS to the UK it was really nothing at all. This is the NHS we’re talking about – people’s lives.

My audience wouldn’t have accepted such a poor analysis of the data for my crappy report – so don’t you dare do it either. You are the audience now and this is really important. Don’t accept these arguments as the complete story – understand them for what they are. A couple of cherry-picked, airbrushed, bullshit numbers selected to back up a plan that the government wanted to do anyway, regardless of the what the actual evidence suggested.

If I couldn’t get away with such behaviour in front of my board then why should the government get away with it in front of the British public?

Pulling basic information from the British Medical Journal and the World Health Organisation is not hard to do – I did it with no trouble at all. The government’s position is nothing more than pure deception and when we look at the real evidence – their position fucking stinks.

RedEaredRabbit

*More or Less’s excellent piece on this same subject helped a lot with putting this post together. It’s available as a free Podcast so have a look for it.

Advertisements

About RedEaredRabbit
My name is RedEaredRabbit, King of Kings. Look on my works ye Mighty and despair.

22 Responses to Facts Evasion

  1. ffolliet says:

    Lies, damned lies and statistics eh? Never let it be said that the Government would let the truth get in the way of political agendas. All very depressing. My sole comfort is that i didn’t vote for this lot.

  2. Pingback: Tweets that mention Fact Evasion « Musings of a Red Eared Rabbit -- Topsy.com

  3. Rachel Green says:

    Excellent: amusing and convincing but nobody on earth is daft enough to believe the Tories are reforming the NHS in order to improve health outcomes.

    They’re doing it to open up our health system to people who wish to make a big fat stinking profit out of it.

  4. Will says:

    Well I’m sure a lot of this makes sense, and is probably quite true, its not fair to dismiss all arguments for NHS reform.

    Still, my main point lies at your conclusions – you are extrapolating data based on past trends, for future ones. According to your big graph, the trend is for the UK to a have a 0 death rate for that health issue by around 2013. A practical impossibility.
    Past rates are good, but future ones unpredictable. To presume the rate will continue as it has done is as foolish as the government when they manipulate the data for their ends.

    • A clarification, the extrapolation was John Appleby’s not mine.

      I know what you are saying that future figures are unknown but in fairness, John Appleby didn’t say those were definitely what was going to happen.

      I don’t agree with you that using historical data to help forecast future figures is foolish though.

  5. Peter E says:

    Another superb post.

    There’s one more variable missing in all this too – cost. The UK spends around 7% of GDP on the NHS. France spends 11% (40% more). The US spends 16% (and misses out cover to a fifth of its population). Even if France’s system is qualitatively better (which you cast doubt upon) it is undoubtedly much more expensive.

    The simple fact is the NHS offers excellent service at relatively low cost – shame the Coalition are also not drawing attention to that fact. Messing with it for political ideological reasons is misconceived and will be hugely unpopular.

  6. Sidheag says:

    Not so lily-white, your own use of data here. E.g. when politicians compare us with “Europe” they generally mean the EU, or even the core EU countries. To WHO, Europe includes e.g. all the former USSR countries. (See http://www.who.int/about/regions/euro/en/index.html ) It’s hardly surprising or reassuring that we have greater life expectancy than Europe in that sense. Also your table shows *healthy* life expectancy at birth, which is systematically lower than the “average life expectancy” figures you use for the UK – a bogus comparison. The WHO site I just linked gives healthy life expectancy at birth figures from 2003 on a country by country basis. The first half-dozen European countries I clicked on all had better figures than the UK, although I didn’t check systematically.

    • Hi. In my defence, I did not know that when politicians mention Europe they actually mean the EU. If that’s the case they should say the EU instead because they are different.

      In any case, the reason you were able to check my numbers was because I showed where they came from. I was being completely open about the data I used.

      Regarding my table, the left hand side shows life expectancy and the right hand shows healthy life expectancy. I took figures from the left hand side so was comparing like with like.

  7. DaveA says:

    Excellent musing. Unless you are pre-disposed to believe what Tory politicians (or perhaps any politicians) say, then you will always find that they pick statements (not even always facts) to support their agenda. Many want to believe what they are told by ‘authority’ as it absolves them of responsibility. There are none so blind as those who will not see. Sadly most of us are so blinded my the misinformation that it’s hard to make really informed decisions. Thanks for shedding some light!

  8. ahteem says:

    Very interesting blog post. However I have to say that I’m not intrinsically opposed to reform of the NHS. Unless we want to claim that the NHS is a near-perfect organisation operating with the most efficient structure currently possible then it is right that we should be continuously debating how we can improve services – in fact, were the NHS not the subject of continuous debate I would argue this would show that it was no longer a political priority.

    I’m not sure what specific funding cuts having been mentioned by the government, and as far as I understand the NHS would remain free at the point of consumption for patients under the proposed reforms. I do not have a specific problem with increased use of the private sector, because I would see this would as a means of increasing access to health services for all. Currently you can choose to pay extra to purchase medical insurance in the UK as a means of gaining access to private health care in the event of your becoming ill. My personal inclination is that it would be better if the government were to pay private companies to provide some healthcare to increase access for all, rather than for the current divide between access to state-funded and private health care to increase.

    • Thanks. I’ll be honest and say I don’t know enough about healthcare to say what effects the proposed reform would have.

      I just wanted to make clear that what we’ve been fed so far is a long way from being good enough to convince me it is necessary.

      It very much concerns me that the government doesn’t seem to be able to deliver good reasons for a reform and instead revert to the tactics I describe above.

      If a reform is to the NHS’s benefit then the government should be able to come up with something better than what they have managed so far.

      • I’d like to comment from personal experience: the results of privatised ‘healthcare’ are already evident in the Nursing Home/Care Home sector which is vast. I work for the NHS and see the results of ‘competition’ first-hand. ‘Competition’ leads to ‘efficiency’ and ‘choice’ only in the sense that costs are cut in order to maximise profits for the individuals/companies who own the businesses (and don’t ever forget, that’s what they are). All too often they pursue this ‘efficiency’ through paying minimum wage to qualified carers, and through cutting the care they give to the core. It’s not unusual to enter a home and find it stinking of urine, and to see a circle of elderly people slumped in chairs ‘watching’ daytime tv on high volume. They’re locked in, and they have little in the way of proper social interaction or entertainment. There is ‘choice’ only for those who can afford to pay for it. Yes there are good homes out there, and there are lovely staff who work for a pittance and do their utmost to care properly for their ‘clients’. But they aren’t fairly remunerated for it.

        Remember too the recent scandal that unpicked the ‘not-for-profit’ scam of related companies owning and charging high rents for buildings. This is the future of the NHS, and it appals me.

  9. Good post. Actually there is a lot of arse-about-tit logic coming from Lansley. For example, does anyone know why he chose to cut administration by a third? The statement initially comes from their Draft Manifesto published a year ago:

    Our reforms will devolve decision-making closer to patients, removing the need for expensive layers of bureaucracy to oversee the NHS. As a result, we will be able to cut the cost of NHS administration by a third and transfer resources that Labour is currently wasting on bureaucracy to support doctors and nurses on the frontline.

    Evidence? There were plenty of health documents on the Tory web site a year ago, I downloaded them and nowhere could I find what would give a cut of a third.

    Of course, I am being logical. The statement actually says “we will spend 1/3 less on admin and we will hope that system continues to work”. Arse-about-tit.

    What about the cut of 45% in management? Well first, management in the NHS is about 4% which is low on international standards for healthcare and low when compared to industry. Again, the reason for this figure is Lansley starting with the figure he wants management to cost. Management in commissioning costs £1.85bn and Lansley’s fag packet calculation was to make it cost £1bn. Nice round one billion. Since it happens to be 55% of the current cost, Lansley then says that he will cut management by 45%. But where is the evidence that commissioning can be done for 45% less?

    Well, management has doubled since 1997 so you could argue that this cut just takes management numbers to 1997 figures. However, the NHS does 65% (hmmm, not 100% more) more work than in 1997 and it has 35% more employees (management makes up just 4% of that), so if you go back to 1997 management numbers the NHS will be under-managed compared to 1997 – you will need more managers to have the same manager-to-employee ratio, or manager-to-outputs ratio.

    Clearly Lansley hasn’t a clue about evidence. All he does is produces a figure and then makes the policy fit the figure. It is no wonder he’s the only person in the country who knows how his policy will work!

  10. Pingback: Cameron’s NHS Myths | Abetternhs's Blog

  11. abetternhs says:

    An excellent summary thank you. The Health White Paper was so poorly written and selective in its references that if one of my first year medical students had handed it in (I’m a GP and college tutor) I’d have insisted they wrote it again. The selective presentation of evidence by government deserves the kind of criticism you’ve given it. The sad thing is that most people believe whatever fits their ideology. Philosopher Steven Law has a book out soon called Believing Bullshit which follows on from Ben Goldacre’s Badscience and (to my mind the best of all) ‘The Demon-Haunted World, Science as a candle in the dark’ by Carl Sagan. Keep shining lights, Jonathon http://abetternhs.wordpress.com/

    • Thank you! The figures the government has been churning out to support proposed reform would probably not look out of place in Bad Science.

      All very depressing.

      Thanks for the book recommendation. I’ll keep an eye out for it.

  12. obligato says:

    Another reason why the comparison is invalid – Lansley’s reforms apply only to the NHS in England – so the appropriate comparator is to mortality rates in England NOT the UK. Given higher rates in the other nations of the UK should bridge that gap even quicker

  13. Pingback: How the Yes was Lost « Musings of a Red Eared Rabbit

  14. Pingback: Creationist Economics « Musings of a Red Eared Rabbit

  15. Pingback: Crappy New Year « Musings of a Red Eared Rabbit

  16. Pingback: Economic Bloodletting « Musings of a Red Eared Rabbit

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: