Saturday, November 28, 2009

Peer Review and the 'Climategate' scandal

There's been a lot of interesting background on the leaked e-mails and data from Britain's Climate Research Unit. Below, some important aspects of the scandal to keep in mind: The excessive reliance on compromised peer review, psychological pressure to conform with the "scientific consensus", and from Shannon Love, insightful comments about weaknesses in scientific software.

Mark Steyn points out the key methods for discrediting those who don't fall in line, and the chief reason for the focus among prominent scientists on supporting the "climate change catastrophe" theme. Read the whole thing:
. . . Look for the peer-reviewed label! And then just believe whatever it is they tell you!

The trouble with outsourcing your marbles to the peer-reviewed set is that, if you take away one single thing from the leaked documents, it's that the global warm-mongers have wholly corrupted the "peer-review" process. When it comes to promoting the impending ecopalypse, the Climate Research Unit is the nerve-center of the operation. The "science" of the CRU dominates the "science" behind the United Nations IPCC, which dominates the "science" behind the Congressional cap-and-trade boondoggle, the upcoming Copenhagen shakindownen of the developed world, and the now-routine phenomenon of leaders of advanced, prosperous societies talking like gibbering madmen escaped from the padded cell, whether it's President Barack Obama promising to end the rise of the oceans or the Prince of Wales saying we only have 96 months left to save the planet.

But don't worry, it's all "peer-reviewed". . . .

The e-mails of "Andy" (as his CRU chums fondly know him) are especially pitiful. Confronted by serious questions from Stephen McIntyre, the dogged Ontario retiree whose "Climate Audit" Web site exposed the fraud of Dr. Mann's global-warming "hockey stick" graph, "Andy" writes to Dr. Mann to say not to worry, he's going to "cover" the story from a more oblique angle:

"I'm going to blog on this as it relates to the value of the peer review process and not on the merits of the mcintyre et al attacks"
. . . .

Looking forward to Copenhagen, Herman Van Rumpoy, the new president of the European Union and an eager proponent of the ecopalypse, says 2009 is "the first year of global governance." Global government, huh? I wonder where you go to vote them out of office. Hey, but don't worry, it'll all be "peer-reviewed."
David Foster on the power of the in-group:
It now seems clear that many climate scientists have shown a most unscientific lack of interest in following the data wherever that data may lead, coupled with an unwholesome eagerness to disregard and to disrespect the opinions of anyone outside of a closed circle of “experts.”

In comments on a NYT blog (excerpted at Instapundit), someone comments:

“It is possible that some areas of climate science has become sclerotic. It is possible that climate science has become too partisan, too centralized. The tribalism that some of the leaked emails display is something more usually associated with social organization within primitive cultures; it is not attractive when we find it at work inside science.”

This kind of tribalism is by no means limited to “primitive cultures,” rather, it is dismayingly common in societies of all types. The phenomenon was astutely analyzed by C S Lewis in his writing on what he called the Inner Ring.

The desire to belong to an in-group, Lewis argues, “of all earthly powers is strongest to make men do very bad things before they are yet, individually, very bad men.”

To nine out of ten of you the choice which could lead to scoundrelism will come, when it does come, in no very dramatic colours. . . .

. . . And you will be drawn in, if you are drawn in, not by desire for gain or ease, but simply because at that moment, when the cup was so near your lips, you cannot bear to be thrust back again into the cold outer world. . . . It may end in a crash, a scandal, and penal servitude; it may end in millions, a peerage and giving the prizes at your old school. But you will be a scoundrel.
Shannon Love pointed out additional, excellent reasons not to trust peer review of climate studies (before we even knew that the raw data had been thrown away): 1. Scientists are not software engineers. Interesting reading. And 2. no one peer-reviews scientific software.

Maybe not in climate science. But when I was in the pharmaceutical industry, the software had to be fully validated before the statisticians started plugging in the data. The statisticians were also subject to grilling concerning their treatment of the data, similar to that which Love described at the link as part of standard software development - from their peers and from people knowledgeable in related fields. I've been in the room to see it happen a few times. Often tense. Apparently, climate science is different:
Most climatology papers submitted for peer review rely on large, complex and custom-written computer programs to produce their findings. The code for these programs is never provided to peer reviewers and even if it was, the peer climatologists doing the reviewing lack the time, resources and expertise to verify that the software works as its creators claim.

Even if the peer reviewers in climatology are as honest and objective as humanly possible, they cannot honestly say that they have actually performed a peer review to the standards of other fields like chemistry or physics which use well-understood scientific hardware. (Other fields that rely on heavily on custom-written software have the same problem.)

Too often these days when people want to use a scientific study to bolster a political position, they utter the phrase, “It was peer reviewed” like a magical spell to shut off any criticism of a paper’s findings.

Worse, the concept of “peer review” is increasingly being treated in the popular discourse as synonymous with “the findings were reproduced and proven beyond a shadow of a doubt.”

This is never what peer review was intended to accomplish. Peer review functions largely to catch trivial mistakes and to filter out the loons. It does not confirm or refute a paper’s findings. Indeed, many scientific frauds have passed easily through peer review because the scammers knew what information the reviewers needed to see.
Shannon Love again with a Thought for the day:
Those who can, do. Those who can’t, teach. Those who can’t, and can’t teach, create a fake ecological disaster so that they can get grant money.

No comments: