Every once in a while a science story hits the mainstream that sounds really cool. Then you get in to the details and notice some interesting commonalities:
- A condemnation of those who remain skeptical of the claims [Real scientists accept and encourage skepticism of their work because it provides them with ideas on how to test it so as to put it on a stronger footing.]
- A reliance on results which are very low but "statistically significant" [Statistical significance is a subjective choice. Many scientists use a standard but arbitrary cut off which states that, if the effect you are trying to measure is nonexistent and you were to do the experiment exactly the same way with the exact same number of trials 100 times you would expect to get the results you got only 5 times or 1 time depending on how significant you are trying to get. Statistics never prove an hypothesis, they only tell you whether, on the basis of your experiment, you can reject the hypothesis.
- The effect was discovered at Princeton University which has a thriving paranormal /parapsychology culture though not much to show for it.
- At some point meta-analysis is invoked. [Meta-analysis is a statistical trick used to turn many small scale experiments into one big one. Different statisticians can use perfectly acceptable standards for meta-analysis and get completely different results. And they are the ones who actually understand this crap. Many of the finest scientists have taken one stats course in their lives which is one reason why science journals need to be so choosy about what they print.]
- At some point quantum mechanics - or more precisely - poorly understood quantum principles of an unknown nature are invoked.
- The results will not appear for other people when they to reproduce the results.
This story by Wired News has all those elements which leads me to believe that these people who have worked for nearly 30 years to find evidence that the human mind can influence machines or random outcomes of physical processes should keep working and stop griping that people should be more accepting of their work. Princeton also has a project designed to show how major news events cause the collective consciousness of the planet to flip out and upset random number generators. Once again we are dealing with statistics and once again only people without a sound basis in statistics (meaning everyone) is supporting it.
On a similar line of thought, I have noticed that there is a world of difference between medical science (things your doctor tells you) and medical studies (things your nightly newscaster tells you); the latter being produced by drunken pre-med school frat boys or by companies or lobby groups with a major emotional or financial stake. Studies are typified by ridiculously small sample sizes, conclusions which completely do not follow and a high degree of popular press publicity. Even the popular science press can be sucked in by them at times, though they tend to be slightly more balanced in their reporting. (In the preceding New Scientist link if you were reading quickly with a pre-existing distrust of video games you would think the article was proof that they were harmful even though the exact opposite could also be true without the article being wrong. Most other media who reported on that story were not as balanced.
So let us review.
Words and terms which should activate your Bullshit detector:
- "Skeptics suck!" (or words to that effect.)
- Quantum entanglement. (unless the story is about particle physics.)
- Spooky action at a distance. (Any quote by Einstein that can be understood without a PhD or via paraphrasing is generally being taken out of context.)
- Statistically significant.
- A new medical study.