The Challenge with Peer Review

Svartbjørn

Senior Member.
Came across this interesting article in the NeuroLogica blog today. Goes into the ups and downs of peer review and asks if the current system is broken, or just needs to be updated. As much as we rely on peer review to vet our sources, understand the science and technology in the world, and basically use it as a way to make sure that we're not just fooling ourselves or falling victim to bad science/bad research, I figured this might be something that we, as a group of nerds and geeks, may want to discuss. Nothing really to debunk, this thread is specifically about opinions of and about the peer review process.

Faking Peer-Review
Published by Steven Novella under General Science
Comments: 3
I think we just have to face it – humans are cheaters. It’s in our nature.

We are also complex social creatures. The result, according to psychological research, is that most people will engage in small cheats if given the chance and they think they will get away with it. Various studies show, for example, that 75-98% of high-school and college students cheat at least once during their careers.

This makes sense given that we evolved in a resource-limited situation. There was an ever present real risk of starving to death, and so the willingness and ability to sneak a little extra food from the group would have had a distinct survival advantage.

At the same time there was a survival advantage to defending oneself against cheating, and living in groups meant that the group could defend against cheating using social pressure. Humans are therefore conflicted – we want to cheat but we feel bad about it because we also have a feeling of disgust toward cheating in order to pressure others into not cheating. We feel guilty and fear the shame of getting caught. When you balance all these things out, most people will cheat a little when they can get away with it. Those who cheat more are better at rationalizing their own cheating. Increased cheating may also result from greater pressure to perform, overcoming the social pressures against cheating.

...

Content from External Source
Fully article: http://theness.com/neurologicablog/index.php/faking-peer-review/#comments
 
Last edited by a moderator:
I served on our university promotion committee for a number of years. It was an interesting experience in that there was an array of best and worst behaviors with respect to academic credentials.

Some faculty published in journals where they actually served as editors or on the board of directors. Other faculty sought out journals with names that sounded academic, but were specifically designed to produce publications for promotion in specific disciplines. Some paid to have the manuscripts published in so-called "vanity" presses. I learned the contract term "subvention" that way.

On the good side, however, is the general accountability produced by a particular discipline or the general public domain. All thesis statements, research, and conclusions are subject to scrutiny, which becomes the ultimate arbiter of academic credibility. So in reality, there are multiple layers of peer review. Avoiding that process exposes a scholar to what happened to J. Marvin Herndon, something that any reasonable person, especially someone at a college or university wants to avoid at all costs.

You can only lose your credibility once.
 
Some faculty published in journals where they actually served as editors or on the board of directors.

This is kind of what the article points to.. this SHOULD be considered a conflict of interest and not allowed under peer review for any reason. I know scientists are known for objectivity and being honest, but as you pointed out Mike people get weird when their careers and paychecks are on the line. The article also points out how frustrating it is for follow ups to be done when the data is flawed or wrong to begin with.
 
This is kind of what the article points to.. this SHOULD be considered a conflict of interest and not allowed under peer review for any reason. I know scientists are known for objectivity and being honest, but as you pointed out Mike people get weird when their careers and paychecks are on the line. The article also points out how frustrating it is for follow ups to be done when the data is flawed or wrong to begin with.

One thing that I've learned over the course of an academic career is that professors are just like the rest of humanity, with all the inherent faults and talents.

It is one of the reasons why the peer review process for tenure and promotion has so many redundant (department, dean, university) layers. Due diligence and transparency are easy if it is accompanied by good faith on all sides, but that doesn't always happen.

Or, as you point out, sometimes the data is flawed. But again, it comes back to why peer review, with a little humility added, can be a very good thing for a career.

My two cents
 
yep, i agree.. and Humility is the key word there. I think peer review's gotten complacent somewhat.. dont misunderstand, I think its a great thing and Im glad we have it, I just happen to agree with the author that it might not be a bad time to revamp it a bit.
 
Back
Top