« Hunger Proxies | Main | Chapter 8 and 9 in It Ain't Necessarily So »
November 1, 2005
Are surveys and statistics really reliable?
Often times in news articles there are statistics derived from both experiments and surveys. The audience usually only sees the statistic such as 1 in 8 American children go hungry, but what they don't see is how that statistic was found. Chapter 6 and 7 in It Ain't Necessarily So discuss the problem with statistics and especially those statistics that arise from surveys.
School Choice Polls- The first survey discussed is one that was trying to determine whether people believed parents should have a choice in what school there child attends. More specifically, should federal money go to sending children to private schools? One survey reported in the Star Tribune reported that 60 percent were opposed to using public spending for sending children to public school, whereas another survey reported in the Christian Science Monitor reported that 61 percent felt money should go to helping public schools. On both of these surveys the questions asked could interfere with the answers. Some questions said the children would be sent at "public expense" and others said "public tax dollars." Someone reading the former might think it will be extra money out of their pocket, whereas reading the latter they might think it is already money they contribute so it does not seem as drastic. Also another survey began the questions with "how much do you support," which makes people think they have to support it. School choice Polls show how the questioning can interfere with receiving reliable statistics.
Gun Violence and Youths: In 1993 Louis Harris created a survey to determine how many teens have experienced gun related violence. The survey was done as part of a campaign to prevent firearm injuries. Therefore the person conducting the survey already believed teen gun violence was an issue, which would create a bias before the survey was even done. The results from the survey found that among the teens asked 13 percent had been threatened to be shot, 11 percent had actually been shot at, 4 percent claimed to be wounded by a gun shot, and 9 percent claimed to have shot a gun at someone else in their lifetime. By looking at these results it would appear that gun violence is a major threat to teenagers. Another official government survey done by NCVS found drastically less statistics: 1 percent were threatened, one-seventh of one percent claimed to be shot at, .02 percent claim to be injured by a gun, and only 1 percent claim to have shot a gun at someone else. Both surveys tried to ask the same questions, but the results were drastically different. One problem with this survey is who you choose to question. Violence prevalence is different from area to area, so the statistics for one group or one geographical area may be drastically different from a true representation of the country as a whole. Also in the one survey the principal was asked if students could be questioned, and like the book suggests if violence is prevalent in a certain school a principal might be more willing to let students take part, in hopes that it will decrease violence.
Holocaust- In 1993 a startling statistic was released by the American Jewish Committee which claimed that more than 1/3 of Americans doubted the reality of the Holocaust. This finding caused much concern throughout the country: “It was unclear whether a large portion of the American Population consisted of ignoramuses or anti-Semites-either way, the news was disheartening." Actually the American population consisted of people who answered incorrectly because of poorly worded questions. Many of the questions were worded as double negatives. When another survey was created, after the confusion of questioning was realized, only 1 percent doubted the occurrence of the Holocaust. This drastic change in stats shows the importance of how questions are worded. Johanna also addresses this topic on her blog.
Breast Cancer Debate: This book discusses how often times stats are published for shock value. The more startling a statistic, the more likely the audience is to be interested in it. A paper reported that 1 in 8 women suffer from Breast Cancer. As a woman, I find that to be very alarming and shocking. That would mean out of our news writing class several of my classmates will develop Breast Cancer. The National Cancer Institute clarified this statistic when they said lifetime risk is not the same as the amount of risk a woman faces at any one point in her life. In actuality 1.6 percent of women have the chance of getting Breast Cancer by age 50, 2.4 percent by age 60, and the closest statistic to the 1 in 8 was 1 in 24, which is much different than 1 in 8.
All of these articles, although limited, show that we can not always rely on the statistics we hear or read about. Before you become overly concerned by a statistic look into how that statistic was found. Was it found using a survey? Was it an experiment? Were the people questioned random?
Posted by JennaOBrocto at November 1, 2005 11:43 AM