June 24, 2003
Can We Check Your Work?... Let's take five with Moira Gunn. This is "Five Minutes".
No one sits up at night, worrying about who's watching the scientists, making sure they don't just dream up new faux discoveries and try to pass them off as fact ... and for good reason.
Certainly, our experience has been that the science we see in the media has overwhelmingly been the truth, and the reason for this is not the fact-checking we expect from journalists. No, it's actually the "process checking" of the scientific community.
You see, scientists watch each other like hawks, and it can be a long and bumpy road between scientific conjecture and accepted fact.
Before a new scientific discovery gains full acceptability, other scientists must be able to whip up the same recipe and come up with arguably similar results, and it's not uncommon for independent groups - even in the same building - to have trouble duplicating an experiment.
--
This can be especially challenging when the breakthroughs depend on the leading - if not, bleeding - edge of technology. One lab's one-of-a-kind machine will often get a different reading from a similar one in the next lab over. So scientists have learned to check everything that can affect the outcome.
They've also learned to account for these differences from environment to environment, and basic material to basic material. And it makes for great science to keep asking the question: "Why can't we do it, too?" Especially since it's well known that problems can come from the most innocuous sources. For example, what appears to be the same paper filter provided by two different suppliers can yield entirely different results.
There is also a constant struggle between the physical experiment and the data which it generates, and being less-than-careful can create tremendous problems. In one case from the Lawrence Berkeley National Laboratory, it appeared that a staff scientist used computer numbers which did not match up with the original raw data collected. And you might well ask, how can that be?
--
Well, in science, the translation from raw to computerized data follows strict guidelines. For example, let's say a sensor failed so there is no read-out of a value. You might be tempted to write it down as a zero. No value. But a zero might also be a valid read-out, as in "The amount of oxygen in this sample is zero." So if the sensor is broken you must somehow code it as a "Missing Value," while in the second instance, it really is a zero. Then when you take the average or calculate the minimum and maximum values you aren't fooled by a zero which means the sensor is broken.
It may sound simple, but if you are a scientist and you don't pay attention, not only is your data bad, your conclusions are wrong and you can expect to be eaten alive - professionally, that is.
And once scientists from around the world start looking, they are relentless. In the Berkeley scientist's case, his earlier work from other research centers was also called into question, he was dismissed and the onus moved to him to prove that he didn't "fudge" his results all along. This can be difficult if you're out of a job and you don't even have a lab or computer system to revive your experiments.
As harsh as this all seems - and it is - these are some of the little known but extremely rigid rules which enable us to believe in science.
I'm Moira Gunn. This is Five Minutes.
