SURVEYS THAT CHANGED THE WORLD
With all these awesome survey sites around and survey and poll results being so well-represented in the news, it’s difficult to imagine today that there actually was a time when surveys of whatever kind were very rare.
Like just before the American Presidential election of 1936. Online surveys were of course unheard of because there was no Internet, but it was even worse than that. Quite simply, all governments, businesses, and the general public regarded surveys or opinion polls as some novelty only used by newspapers that needed something to write articles about. Only population censuses were taken seriously, and this was because they were not regarded as surveys.
That is until some guy called George Gallup changed everything.
This is what happened. It was in the 8th year of the Great Depression and the President, Franklin Roosevelt, was running for re-election. He was still trying to push through the provisions of his New Deal economic policy through Congress and the courts. Therefore, the election had the potential to change history one way or another.
His opponent was a republican called Alf Landon. That’s short for Alfred Landon.
Since there was not much TV at the time, newspapers and weekly magazines is where the presidential campaigns were mostly being held. So, one of the great magazines of that time, the Literary Digest, decided to do an opinion poll of the two candidates. This was big news, because this magazine was being read in at least a third of all households in the United States, and they had correctly predicted the previous five presidential elections.
A Contest of Two Surveys That Shook the World
Sure enough, the Literary Digest embarked on the most elaborate survey ever conducted. A mailing list of 10 million respondents was drawn from telephone directories, club memberships, and the magazine’s subscriber’s list. Each respondent was mailed a mock ballot with a stamped return envelop, and they were asked to mark it as they intended to do at the election, and then mail it back.
In the meantime, George Gallup, a man who had taught journalism at three universities and had worked at a major advertising agency, had recently started a polling company in New Jersey called America Institute of Public Opinion. He was also writing a weekly column in a national newspaper called America Speaks, in an increasingly unsuccessful attempt to persuade everybody that 1) there is such a thing as scientific surveys that do not require the participation of millions of people, and 2) that his upstart, poorly funded company could conduct scientific surveys that could correctly assess the country’s pulse. He wanted the whole world to disregard the Literary Digest’s way of conducting polls, because he considered it to be faulty.
Of course, everybody thought the guy was a fool with some serious envy issues against the great Literary Digest.
Then George Gallup got fed up with all this skepticism, so he decided to give them a dose of shock therapy. He declared that he would predict the outcome of the election more correctly than the Literary Digest. Moreover, he would do this by sampling only 50,000 people. To rub it in even deeper, he guaranteed all newspapers that were subscribed to his poll that he would refund them their money if the election results proved him wrong. It was a breathtaking gamble that was met with a general opinion that survey conductors even today know all too well – how in the world could a survey of relatively few people be more accurate than a survey of a much larger pool of respondents?
The Results of the Gallup and Literary Digest Surveys
So the United States of America was treated to two poll predictions.
The Literary Digest predicted that Alf Landon would win with 57% of the popular vote against Roosevelt’s 43%
Gallup predicted that Roosevelt would win by 55.7% of the popular vote against Landon’s 44.3%
Everybody waited for poor George to go bankrupt.
But on Election Day, the country was stumped: Roosevelt won with 60.8% of the popular vote. Neither of the two polls got the figures spot on, but Gallup’s candidate won handsomely. America was shocked. Perhaps if it was in a less stable democracy there would have been riots in America that day, because nobody in their right mind could have doubted the Literary Digest. Not since 1820, when President James Monroe had won against token opposition, had anyone won such a huge percentage of popular votes – and only Lyndon Johnson would later beat that record with his 61.1% win in 1964.
But, instead of rioting, everybody figured that maybe there actually was something to this crazy science of George Gallup. That maybe, just maybe, he was right all along.
That change of mind marked the beginning of surveys as we know today, as the media started embracing the scientific approach to survey taking.
How Survey Sampling Got Its Groove
So what happened? Survey sampling is what happened. The Literary Digest made the mistake of assuming that the more people interviewed, the higher the accuracy of the survey results. It’s a logical assumption, and they had the historical track record to back them. For all five previous elections they had done their surveys the same way, and each time they had come up with the correct prediction. So, why not this time too?
The explanation holds true today as it did back then. Simply put, sampling is about representing a diverse target population. If a population is very homogenous in terms of characteristics or thoughts, sampling has little effect – which ironically also means that a small sample of respondents would be as reliable as a much larger sample.
In the case of the unfortunate Literary Digest, the American population had changed dramatically in the years before that fateful election. An estimated 9 million people – nearly a third of the entire population – had lost employment, which meant their names were unlikely to be in the magazine’s subscription list or club membership. This is what is known today as selection bias, where there is an error in selecting the respondents. Secondly, the magazine ignored the fact that of the 10 million people they set out to reach, only 2,4 million actually replied. That was still a huge sample, but it made the actual survey sample about a quarter of the intended sample. That is called non-responsive bias, since a certain part of the intended sample did not respond.
The Impact of the Gallup Poll of 1936
The impact of Gallup’s poll of 1936 was momentous. Almost overnight, scientific survey techniques were accepted by governments, businesses, and the general public. From that point on, it was accepted that a survey doesn’t necessarily need to have a huge number of respondents for it to be credible. The media in particular took to using surveys almost routinely, and politicians began their enduring love affair with poll ratings.
Better still, the Gallup poll brought down the financial entry barrier into the industry, paving way for the growth of the industry that continues to date. Since it was no longer necessary to spend heavily on every survey, a lot more businesses suddenly could afford a survey. The effect was an explosive growth of the industry. Gallup’s firm, for example, rode the wave of his famous success to become one of the world’s largest and most respected polling firms.
But it was not to be smooth sailing all the way for Gallup and the business of surveys. Three elections later, in 1948, another shock would reverberate through the industry and change how surveys were conducted. For details of that, head on over to our third article in this series, called The Threesome Battle: Surveys That Elected the Wrong President.
Still though, if it hadn’t been for Gallup’s poll of 1936, who knows what would have become of the survey business. Today’s politics, business marketing, social sciences, and even personal behavior are, in one way or another, affected by surveys. And that survey of 1936 is where it all truly began.
Do you know of an important survey that you would like us to cover? Feel free to send us your suggestions!
Photo credit: Bart (thanks, Bart)
Jan 21, 2014