And Now A Small Confession…

30/08/2010 2 Comments

Last night, I published the results of my eagerly awaited jazz-sushi survey, where I attempted to find whether or not there was a correlation between liking jazz and liking sushi.

If you have not already done so you can read it here. (Feel free to skip the numbers bit if that stuff bores you.)

So I proved that a correlation existed and made it into a law and all is well. Well not quite. I should come clean about something. It didn’t really prove anything and I’ll explain why.

Firstly, (as many people pointed out), my questions asked for Yes/No answers to complex questions. There are lots of different types of jazz and usually someone doesn’t like or dislike all of them. My survey forced them to interpret the question as they saw fit. Worse, it caused people to give me long-winded answers which I had to interpret.

Why’s that worse? Well I knew what I wanted the outcome of my survey to be and while I didn’t consciously seek to influence the results in this manner, I am hardly in the best position to be a neutral judge.

Also as @mapsadaisical quickly pointed out, I had a self-selecting sample. This means people were free to choose whether to take part or not. Why is that bad? Well people knew that I was trying to find a correlation between people who liked jazz and people who liked sushi. When people know what is trying to be proven it influences whether they respond or not.

On Friday night I did my sums and found that there was a correlation but it was not significant enough to prove anything one way or the other. I explained this on Twitter and asked for some more responses. Of the next 12 responses 11 were either likes both or likes neither. This wasn’t coincidence, it was simply people wanting to help me show a correlation. Those who did like both or neither kindly though “I’ll help you out.”

Another example of this came when I was watching a morning day time TV show a few years ago. It was GMTV, or Anne & Nick or Richard & Judy or some bollocks, and they had a phone in poll. A phone in poll is even worse for this problem than Twitter because the effort of making the call is greater and they charge you money for doing so. You aren’t going to bother voting unless you have some compelling reason to do so.

The poll asked people to vote on whether or not they were currently in an abusive relationship. About 50% said yes. At no point did the programme mention that the surprisingly high result could be influenced by the fact that this poll was much more important to someone in an abusive relationship and they were therefore more compelled to vote than someone who wasn’t. In fairness to the programme they didn’t try to conclude that 50% of all relationships were abusive.

There is another problem with the way in which I gathered the stats. Even if everyone who saw the question had responded, I didn’t survey a proper cross-section of the public. Supposing I did a poll on Twitter to find out whether people thought Social Networking sites were a good thing. I would certainly get a higher proportion saying Yes than if I stopped people in the street and asked. Although there is no obvious reason for people who use Twitter to have different views on jazz/sushi to the public at large, the whole experiment was to find a correlation between two seemingly unrelated things so really I should have excluded any other similarities between the respondents.

A good example of this is in the polls which newspapers do on their online websites. If the Daily Mail asks a question about immigration on its website is the response going to reflect the views of the country at large? Probably not, because people who read the Daily Mail website are likely to have different views on immigration than the average person on the street.

You should treat with skepticism any survey that can’t show clearly how it gathered and interpreted its data to avoid external factors like this affecting the results. Companies like Ipsos MORI go to huge lengths to try to minimise these problems. I didn’t and as such you should just interpret my survey as a bit of fun.

RedEaredRabbit

Advertisement

Filed under Surveys, Twitter Tagged with , , , ,

The Jazz Sushi Survey

29/08/2010 6 Comments

When I was just a young rabbit, I was taken, as a treat, to see the National Youth Jazz Orchestra who happened to be playing in my village. It was an epiphany and I was transfixed. Never in my life had I imagined music could be made so utterly awful. Equally shocking was that a good many people around me seemed to be enjoying it, and not just a little bit either. A ginger man a couple of seats away with his eyes closed looked for all the world like he was having an orgasm for the entire concert and for all I know he was.

Years later, I was having a pint with a mate in our local pub, The King of Toss, near Marble Arch. Above the King of Toss was a restaurant to which neither of us had paid any attention in the years since we’d been drinking there. Seemingly no one else had been paying it any attention because on that night a member of the waiting staff came into the pub with a plate of sushi, offering free samples in a bid to entice some drinkers upstairs. So I tried some. This, ladies and gentlemen, was my second epiphany. Never in my life had I imagined food could be made so utterly awful. How could it possibly have been that bad? After all, I like rice, I like fish. In fact given the same ingredients I could probably have made something quite nice. This was anything but. The rice has a texture like it had been cooked the day before, left in the pan to dry then scraped off. It was topped with little red things which seemed to have been sprayed with essence of unwashed genitalia and it was wrapped in one of those unbreakable plastic ribbons that bind up telephone directories. Bizarrely my mate liked it.

At some point in the years since, it occurred to me that I thought about jazz and sushi in pretty much the same way. Not simply in my dislike for them but in the way that I just didn’t get them. I knew plenty of people who were enjoying these pleasures and I would never be able to understand why.
I don’t like Crufts but I can understand why people like it. They get to see the most classically beautiful dogs all standing in a row. I just prefer dogs when they’re fetching sticks and eating slippers but that’s just my preference and I understand theirs.

Jazz and sushi were incomprehensible to me though and the more I thought about it, the more I wondered if these two seemingly unconnected things were in fact connected through people’s preferences. i.e. was there a correlation between people who liked jazz and people who liked sushi? Were these two things completely unrelated or was there a disproportionately high proportion of us who liked both or disliked both compared with the proportion of people who liked one or the other?

This previously unidentified correlation has been an untested theory of mine in the years since but then came Twitter and suddenly I had the perfect opportunity to test it out.

Last week I asked people two Yes/No questions:

  • Do you like jazz?
  • Do you like sushi?

And thanks to those who responded and retweeted it I ended up with 112 responses.

And so to the numbers. Firstly, I worked out the proportion of people who like jazz and the proportion of people who like sushi. The results were:


Using these numbers, I worked out my ‘null hypothesis’. i.e. what the results would be if there is no correlation.
i.e. of the 112 respondents, if there is no correlation between liking jazz and liking sushi then the proportion of people who like sushi and like jazz is:

112 x (64.6% x 54.87%) = 40.41 people.

The full results of this are:

Then I compared this with what the 112 people actually said:

Interesting… there are more people in the like both and like neither than there should be if the null hypothesis is true. Sure enough when I calculated the correlation it came out at 0.17.

Correlation is expressed as a number between -1 and 1. A correlation of 1 means that the correlation is perfect i.e. for me to get a correlation of 1 everybody who liked sushi would have to like jazz and everybody who disliked sushi would have to dislike jazz. A correlation of -1 represents a perfect negative correlation. In my case this would have meant that everyone who liked jazz disliked sushi and everyone who disliked jazz liked sushi. A correlation of 0 would mean there was no correlation at all between the data. My correlation looked like this:

So I had a correlation and better still it was a positive one, but although my figures had a correlation could it just have been I got lucky?

To determine this I needed to work out what the probability of this happening by chance would be if the null hypothesis were true.
I decided to use a fairly standard way of testing significance – that the probability of such an outcome would have to be less than or equal to 1 in 20. i.e. if there is no correlation then results as convincing as mine could come up no more than 1 in every 20 repeats of such an experiment – a significance level of 0.05.

Therefore, if the probability of my set of results coming up is greater than 0.05 then the probability of it having happened by chance is too great, my correlation is not significant and my results are inconclusive. If the probability is less than 0.05 then the chances of this having happened by chance are negligible and my correlation is statistically significant.

Are you ready? Drum roll, please. The probability of a correlation as pronounced as mine having happened by chance is……..0.045!!

That’s right, I really did it. I really did find a correlation between liking jazz and liking sushi. The theory I have held for ages has at last been proven.

I am not going to call it RedEaredRabbit’s Law. After all it is too important to be just mine – it should belong to all of us. I am instead going to call it Cole’s Law. (Nothing to do with Cole Porter  – I’ve just always wanted to call a law that.)

At some point I’ll explain why my method of gathering the data wasn’t perfect but for now I’m just going to bask in my glory.

RedEaredRabbit

P.S. I didn’t mean to imply that jazz and sushi were awful in absolute terms. Just that I dislike them and am personally unable to appreciate them. Don’t lynch me, please.

P.P.S. Please also read the follow up post to this survey here.

Filed under Genius, Surveys, Twitter Tagged with , , ,