I’ve recently been thinking about the meaning of skepticism and what constitutes good and bad science, so I thought I would go back and look at my notes for my public health law course I took last semester. One of the lectures was devoted to junk science and our professor gave us some very good definitions of good, bad, and junk science. He also gave us a good list of factors of which we ought to be mindful when confronted with something somebody proclaims to be “science.” This post is largely derived from that lecture.
First, there is the cynical view of science. If you’re a scientist working for private industry, you earn your money by inventing something useful, or at least something that people will buy regardless of usefulness. On the other hand, if you’re a scientist dependent on government grants, you have to clamor about how bad things are in order to get your money. It’s more likely the government will give away money to someone who wants to solve a “problem” rather than to someone who wants to study how great things are. Furthermore, these horrible problems or disasters scientists predict often never never happen. There was no Y2K chaos and there was no disaster on 3 Mile Island - there weren’t even any deaths or radiation leaks.
So what is science? My professor defined science as the rigorous, systematic use of observations and logic in an attempt to support or falsify possible explanations of natural phenomena. Science, he said, is a way of doing things, a way of thinking. To illustrate what he meant, he compared science to religion. With religion, knowledge is acquired via revelation and the thought process is characterized by intuition and faith. With science, knowledge is acquired via the scientific method and the thought process is characterized by rational or logical thought.
Now that we have a definition of science, we can go into a little more detail about the factors that comprise “good” science. Good science is consistent meaning that it is built upon previous science. Good science is parsimonious - the more complicated an explanation the less likely it is to be true (think Occam’s razor). It is empirically testable - if it can’t be tested, its not science. Good science is also progressive, and useful.
Because science and scientists can command a great deal of respect and because there are many people out there who blindly accept and defer to whatever has been labeled as “science,” there have been unfortunate occurrences throughout history of pseudoscience or bogus science. Despite the often confusing nature surrounding science, the impressive credentials of scientists, and the astounding “discoveries” that have been made, there are some factors that anyone can use to help determine what might be good or bad science.
The first factor to consider in determining whether a recent discovery or study is real is whether there is an over reliance on surveys and statistical arguments. Another is whether there is a high degree of data filtering, i.e., is there a “grab bag” approach to the data. Also, is there a grab bag approach to gathering evidence? Does the scientist appeal to mysteries or myths. These two factors are fairly obvious but still worth mentioning: (1) is there a lack of controlled experiments and (2) has the scientist refused to revise his findings despite being proven wrong?
Continuing on, the following factors deal more with the actions of the scientist and how he portrays and presents his findings. Does the scientist make a pitch to the news media instead of bona fide scientific journals? Does the scientist make claims of suppression? Does the scientist propose an effect that is nearly impossible to detect? Is the supporting evidence mostly anecdotal? Does the scientist work in isolation? Does the scientist propose a new law of nature to explain the discovery?
These are all questions everyone should ask when confronted with science. Though science should certainly be treated with respect, bad or unproven science not be treated like religion. Wherever possible there must be independent confirmation of the facts. When the facts can’t be confirmed or if you sense the presence of one the previous factors, then it is most likely the facts or the scientist is bogus.
So now you would probably like some examples of bogus science? I won’t go into details, but here is a cursory list:
- Acupuncture
- Astrology
- Biorhythms
- Hollow Earth
- Hypnosis
Two examples that are a bit more interesting to me are the Russian polywater controversy of the 1960s and the cold fusion controversy of 1989 involving Fleischmann and Pons.
The last example, I promise, is the hockey stick graph controversy. If you watched Al Gore’s recent propaganda piece, the hockey stick graph played a rather dramatic part. Al Gore depicts the huge horizontal graph that spans the entire stage. However, with the last couple of centuries (the hockey stick part) Gore hops on to a mechanical platform that raises him up so he can show how much temperatures have risen.
