[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Subject Index][Author Index]

Re: Peering at review

> Date: Wed, 23 Oct 2002 10:56:27 -0400
> From: MKIRKALDY@aol.com
> One learns as much about one's colleagues as about science when
> editing, and the flaws in the system become obvious, but the flaws
> are all related to the primatological origins of the reviewers and
> editors.

Are you saying we should get ungulates to do it instead?  :-)

> Stick with peer review, a method that, agreed, sometimes fails to
> catch bad or dishonest science, and which sometimes prevents good
> papers from being published in certain journals, but which also
> ensures that a majority of published papers in scientific journals
> actually conform to the scientific method.

The trick seems to be getting the reviews to review the _method_
rather than the conclusions.  Seems to me that George, Tracey and Greg
are complaining about having had papers rejected because the reviewers
were opposed to their conclusions (or maybe even that they didn't
submit them for review for fear of rejection for that reason.)  That
can't be right, can it?

> 2.  Chuck review and replace it with.....?

In the long term?  Maybe the Internet will come to the rescue with
some kind of distributed reputation manager.  In this kind of scheme,
anyone who wants to be a write or reviewer can register, and anyone
can review anyone else's work.  The weight carried by any given review
is proportional to the reputation of the reviewer, and positive and
negative reviews force the authors reputation up and down
respectively.  This is a complex system from which emerges a consensus
of what "the community" think is good science.

(There are lots of possible refinements here, of course -- I have only
begun to sketch the way such a system might work.  An obvious tweak is
the wherewithal for reviews to be reviewed, so that the reputation of
bad reviewers suffers -- that would help to discourage people from
dissing the work of people they dislike, or whose conclusions they
disagree with _a priori_.)

 _/|_    _______________________________________________________________
/o ) \/  Mike Taylor   <mike@miketaylor.org.uk>   www.miketaylor.org.uk
)_v__/\  "You already have zero privacy.  Get over it" -- Sun CEO
         Scott McNeally.