SURVEYS THAT CHANGED THE WORLD
If you wanted to compare something important across several countries – like, for example, a comparison of each country’s cholesterol levels – you would conduct a survey across those several countries, complete with proper survey samples and data analysis for all these countries. If the topic was something more sensitive and possibly controversial – like corruption, for example – you would think the survey would be even more rigorously conducted, and you would bet that anything less than that would be dismissed and forgotten rather quickly.
So how did a small non-governmental organization get away with a survey that was arguably flexible with those established research rules? And how in the world could such a not-so-well-done survey become an authoritative, internationally celebrated reference?
Well, it happened. In 1995. And that strange survey has since been repeated every year with much pomp and acclaim. That small NGO was Transparency International (TI), then just two years old. And that strange survey is the now highly respected and much quoted Corruption Perception Index.
So what happened?
The Idea That Turned TI Into an International Household Name
Sometime in 1993 a group of people met in Berlin, Germany, and formed an organization with the sort of high ambitions fitting for an international non-governmental organization. In their words, the idea was ‘to raise awareness of international corruption and create a coalition of interests from both the public and private sectors to combat it.’ They called their NGO Transparency International, a clever choice of name that would be abbreviated in conversation to simply TI.
For about two years, TI did its best to establish itself and make a name in the crowded field of international NGOs. It was growing in strength, but remained relatively small and largely unknown in most parts of the world. Then someone at TI hit on a brilliant idea: how about putting up a report ranking countries according to how corrupt they are?
It was the sort of idea that no one could object to. But there was a problem: how could anyone actually tell how corrupt one country is compared to another? The obvious option – conducting a survey – would clearly not work. Corruption is a crime with severe jail terms in all countries. It was therefore foolish to expect that the type of respondents that would be of real use – those who have either given or received bribes – would give truthful answers of their crimes in a survey.
There was also the problem of money. Even if enough corrupt people could, somehow, be persuaded to honestly respond to the survey, it would be an incredibly expensive project across hundreds of countries.
So, Dr Peter Eigen, Chairman of TI, talked to Dr. Johann Graf Lambsdorff, an economist from the University of Göttingen. Soon a plan was agreed on and the economist went to work.
How Transparency International Pulled Off a Survey Trick On the World
The plan was simple. Since Transparency International could not conduct a survey on corruption, it would ride on other surveys on the subject. It would look at surveys done by other credible organizations on corruption and rank countries accordingly. In other words, TI was going to do a ‘survey of surveys’. At least two surveys in each country would be the minimum.
As expected, there was not a single survey that had sampled primary data of the subject (that would be people who had either given or received bribes). But Dr. Lambsdorff found at least two surveys per country, for 41 countries, sampling the views of international business people and financial journalists. After crunching the numbers, he delivered a ranking of the 41 countries to Transparency International, which in July 1995 he released the results to the world.
The world was immediately intrigued. New Zealand, Denmark, Singapore, Finland, and Canada, were delighted to hear that they were the 5 least corrupt countries. Sweden, Australia, Switzerland, The Netherlands, and Norway ranked next – with their more famous counterparts, United Kingdom, Germany and USA ranking at the not so impressive numbers 12, 13 and 15, respectively.
||1995 TI Corruption Index
source: CPI 1995, Transparency International
On the other end of the list, Venezuela, Brazil, Philippines, India, and Thailand suddenly found themselves having to explain why they were the 5 most corrupt countries. And Italy, in particular, was embarrassed to find itself ranked as the 6th most corrupt country, the only first-world country among the top 10 worst performers.
Criticism of the TI Corruption Index
For a survey of such ambition, the TI survey of surveys was received well. International media and most governments either loved the Index or were muted in their criticism. Part of the reason was the sheer novelty of the survey and the subject matter – no one wants to be seen supporting corruption. It also helped that TI readily acknowledged its report’s limitations.
Among those points acknowledged by TI was that their survey carried whatever bias existed in the various surveys that it was based on, and that it was at best incomplete and at worst misleading because it was restricted to 41 countries where at least two surveys could be found.
But, naturally, TI was not going to get off the hook that easily. It was soon pointed out that while some countries were assessed on two surveys, others were ranked from as many as seven surveys. There was also the obvious fact that the Index was in fact not an indicator of corruption, but merely an indicator of how international business people and financial journalists thought or felt about a particular country. TI subsequently tried to clear that misconception by including ‘Perception’ in the official title of its Index, and pointed out that perception also matters, even if it may not necessarily reflect facts as they are.
The Index was also accused of being inherently biased against large countries, because perception of corruption is greatly influenced by the number of corruption incidents rather than percentage of the population affected by corruption. A large country would therefore almost automatically have more cases of corruption – and therefore rank poorly – compared to a smaller country with fewer cases affecting a larger segment of the country.
Another important argument was that people tend to notice corruption more in countries where it is not common than where it is prevalent – in effect punishing countries that were doing well. And then it was also argued that perceptions tend to be influenced greatly by a few reported cases of highly publicized corruption incidents.
The Impact of the 1996 Corruption Index
The Transparency International Corruption Index of 1995 was a watershed in the international fight against corruption. It marked the first time campaigners against corruption had a generally accepted reference point on which to engage governments without being accused of discrimination or ignorance. It also created a powerful tool of embarrassing governments that tolerate corruption as well as other governments and international agencies that support poorly ranked governments.
The index has since become an annual exercise, now better named as the Corruption Perception Index, and has expanded to cover 117 countries. Criticism still continues about the Index, not for the least because TI uses different methodologies and samples every year, making it impossible to properly compare data from one year to the next. The Index also continues to be accused of being insensitive to the dynamics of developing countries.
But there is little doubt that the Index, with all its imperfections, remains an important force for the good of humanity. It is an achievement few surveys can rightly claim. For that reason, the not-so-proper Transparency International survey of 1995 ranks as one of the greatest surveys ever conducted.
Do you know of an important survey that you would like us to cover? Feel free to send us your suggestions!
Photo credit: Royce Bair (thanks, Royce)
Transparency International (1995) CPI 1995, Available at: http://archive.transparency.org/policy_research/surveys_indices/cpi/previous_cpi, Last accessed 11th Feb 2014
You might also like...
The Kinsey Reports: Surveys That Questioned America's Sexuality
The Gallup Opinion Poll: Surveys That Sampled 50,000 Instead of 10 Million and Prevailed
The Lancet Surveys: Surveys That Increased Iraq's Death Rate By 150%
Feb 11, 2014