Why Poll Results Fail, By Yemi Kale

I was asked this recently. Why will the results of a survey/poll be wrong? Was it intentionally manipulated to mislead and if not, how can results say one thing and outcome another? Polls saying one thing and the outcome being different occurs everywhere in the world. But why does it happen?

This is an increasing trend with several polls/surveys in Nigeria. One major reason I have observed is the non-representation of the sample. Eg number of undecided/non-response/refused to say, is more than the total number of respondents which creates a bias and might be flawed or unreliable.

For example, you survey 10k people to say whether they will buy a new product and 7k refuse to respond or are undecided, etc. Of the remaining 3k, 2k say they will buy and 1k say no, so you conclude the product will be a hit as 66% say they will buy. It has a 66% approval rate is announced.

But upon enthusiastically releasing say 1 million units of the product and expecting about 660k in sales since the survey said 66% will buy, only 20% is sold. How can this be when you had a 66% survey result? Well, the key is in the 7k no response/undecided. Your representative sample was 10k.

Only 3k actually responded to the actual info you wanted. This is an important part of any survey/poll. It has to be representative. If it is not, then it’s results are not reliable and no useful insights can/should be made from it. But many polls/surveys in Nigeria make these errors.

So if you want to check the level of reliability of any poll/survey on whatever topic, go into the methodology and check the degree of representation and particularly the ratio of non-responses. If it’s too large then it might not be reliable and its results should be treated with caution.

*Dr. Kale, a former Statistician-General and CEO, the Nigerian Bureau of Statistics, shared his thoughts on Twitter via @sgyemikale