November 02, 2005

Substance vs. procedures in peer review

In principle ... scientific peer review should judge the importance of a conclusion and the reliability of the procedures that led to it. The substance of the conclusion shouldn't really matter as long as it is significant, and as long as it follows logically from the procedures used to arrive at it. In practice, though, peer reviewers often have their own substantive biases, which lead them to praise research supporting substantive conclusions that they like and to condemn research that doesn't. (Murray, Schwartz, and Lichter 150)

Although I understood the basic process of peer reviewing before reading Chapter 9 in It Ain't Necessarily So, I never really considered that peer-reviewed academic articles could be considered unreliable sources.

It often seems like professors here at Seton Hill University (SHU) regard peer-reviewed academic articles as "Holy Writ," to use the words of the authors. When writing essays, professors say, it is always important to check the reliability and credibility of sources; but when it comes to peer-reviewed academic articles, how can one really know if it is reliable and credible or not?

As the authors of IANS point out, the researchers who write the articles could be biased, or the editors who choose which articles are published in their scholarly journals could publish unreliable articles even if they are reviewed by very few scholars, since they have control over their publications. The sources of researchers' funding can also come into play; for instance, an article about energy emissions and global warming will probably be biased if its research is mostly funded by energy companies who want to prove that energy emissions and global warming are not connected.

Although the authors reassure readers that most peer-reviewed academic articles are reliable scholarly sources, I will try to be more skeptical of such articles in the future, especially if I plan to use those sources for news articles.

Posted by ChrisU at November 2, 2005 08:25 AM | TrackBack
Post a comment

Remember personal info?