By Dr. Eduard Zaloshnja
Opinion polls in nations where democracy and elections are relatively new are often used by policymakers in the developed world to hone their policies toward such countries. However, one thing they should keep in mind is that voters in these countries may fear repercussions from non-democratic authorities that still hold power. An example illustrating voters’ fear in countries with a fragile democracy is the exit poll conducted recently by Zogby International in Albania .
Here, on Election Day (June 28), Zogby projected with certainty a huge advantage by 14 parliamentary seats for the ruling Democratic coalition in a parliament with only 140 seats. Meanwhile, 12 seats were considered by Zogby too close to call. But judging from the percentages predicted in each district, as reported live by Zogby’s representative on Top Channel (an Albanian TV network), it was clear that most of the undecided seats – 8 to be exact – were very easy to call. They should have been called for the ruling Democrats…
After almost all the votes were counted in Albania , the results showed that the district percentages predicted by Zogby on Election Day were statistical years far from reality. Why this spectacular failure?
In order to project the distribution of parliamentary seats among the competing coalitions, Zogby hired around 1,000 local interviewers, who interviewed voters in 500 polling stations across the country. Voters were selected randomly in each polling station and were asked to anonymously fill out questionnaires very similar to the real ballots, which were placed in a mock ballot box. A total of 1,000 questionnaires per district were completed.
Usually, not all the voters selected randomly for an exit poll take part in it. For example, in the U.S. , around 10% of selected voters refuse to take part. Whereas in Zogby’s Albanian exit poll, around 20% of selected voters, mainly in rural areas, refused to fill out questionnaires outside of polling stations. And the key question here is why so many Albanian refused to take part in Zogby’s exit poll.
In the U.S., the refusal to pollsters on Election Day happens mainly because some voters are too lazy or busy to spend a few minutes answering questions – not because opposition supporters are intimidated by the party in power. Not so in Albania .
The wide gap between Zogby’s projections and the real results in the Albanian elections was a strong indication that a wide majority of voters that refused to take part in the exit poll were opposition supporters. And indeed, a preliminary report by international observers released the day after elections stated that civil servants leaning toward the opposition were intimidated by the party in power.
Adding two and two together, it is easy to understand why samples in almost every electoral district in Albania were severely biased toward the incumbent Democrats. For example, in the district of Tirana (the biggest by far), Zogby’s sample suggested a 14% lead for the Democratic coalition, whereas the results showed that the real lead was a mere 1%. And such dramatic differences between Zogby’s samples and real results were registered in other big districts like Kor諬 Elbasan, etc.
If Zogby could find a way to estimate what percentage of the voters that refused to take part in its exit poll were opposition supporters, it could have reweighed its samples in each district so that they would represent more closely the reality. For example, if the refuters were distributed at a ratio of 3 to 1 in favor of the opposition, Zogby would have projected only a 3-seat advantage for the Democrats. The final results pegged them at a 4-seat lead…
And the question is whether there was a way to estimate how many voters that refused to take part in Zogby’s exit poll were opposition supporters? The answer, unfortunately, is no. But an alternative way for estimating the bias in Zogby’s samples existed. If a question on previous voting was included in the questionnaire, and a comparison between samples’ previous voting patterns and the real election results in the past was made, the bias could have become evident and, more importantly, measurable. And after measuring it, the bias could have been corrected.
But why Zogby did not include this magic question in its questionnaire? According to Zogby’s representative who presented the results live on Top Channel, they didn’t want to burden the interviewees with many questions. This, because they had assumed the Albanian voters to be like their American counterparts, who have a problem with polls only because of time pressures – not government pressures…
The take-home message for policymakers in the developed world from Zogby’s failure in Albania : Beware of polls in underdeveloped countries!
Dr. Zaloshnja is a Research Scientist at Pacific Institute for Research and Evaluation in Calverton. MD, USA . He examined polls in Albania on behalf of Top Channel, the Albanian TV network that commissioned Zogby International to conduct an exit poll on June 28, 2009.
