[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Subject Index][Author Index]
speculation/testability Pt. II (peer review)
Mickey Rowe wrote:
"However, it would concern *me* if it didn't concern *you* while you
were reviewing papers....."
"However, if I personally were reviewing a paper, I'd be hesitant to
recommend acceptance if the author(s) couldn't even suggest a method for
testing the main premise..."
"In a current peer reviewed paper though..."
"As a reviewer, it may not be your job to suggest tests. As an author
it should be. And as a reviewer it should probably be your job to
recognize when an author hasn't done their job."
I would like to cite from "How to Write an Influential Review" by
Michael L. Rosenzweig, Jerrold I. Davis and James H. Brown. I stumbled
upon it on the Internet, but I don't remember where (sorry).
It is reproduced from Plant Science Bulletin 40(1):6-8 (1994), a reprint
of Plant Science Bulletin 34(2):5-7 (1988).
"Surely a review should seek to identify and encourage the most
promising and innovative research. Yet it is precisely the newest ideas
that are the least tested, the most controversial and easiest to
criticize. In fact, history reveals that new theories often are
incomplete and often do contain serious errors. But wouldn't it have
been a tragedy if Natural Selection had been rejected because Darwin
founded it upon incorrect mechanisms of inheritance.
"If we are to advance our science, it is necessary that we take
risks and actively encourage the development of new concepts, theories,
and methods. If we as reviewers are afraid of mistakes, and insist that
our peers write airtight proposals, then who will dare to tackle the
difficult questions? If we required that a proposal be so well
described that we can visualize every aspect of the research, is the
work really likely to produce any surprises or major new discoveries?
We have to be honest and point out potential problems, but, above all,
we must strive to identify and express our enthusiasm for new ideas and
"The more negative the tenor of a review, the more detail it has.
This is a fact. We know no reason why it has to be so, but it is.
Negative reviews are often full of well-reasoned objections. Positive
reviews are more often brief statements of approval. They may be full
of glowing adjectives, but they rarely contain the details and logical
arguments which would give them substance...."
"Many reviewers seems to think it is their primary responsibility to
discover and call attention to all the flaws in a proposal or
manuscript. Perhaps they have the attitude that it should be deemed
worthy until proven otherwise. They may sound fair, but it is, in
truth, pernicious. It makes us over into petty bookkeepers, subtracting
the value of each counterfeit penny without noticing that they are
coming from a solid gold box. If we adopt the attitude that a proposal
does not deserve funding unless the research is daring, novel or
interesting, then we should place more emphasis on the positive aspects
of a good proposal and write longer, more constructive positive reviews.
"We need to remember that it is much more damaging to our discipline
to suppress an important contribution than to fund or publish a
questionable piece of research. New ideas and conflicting data cannot
have any influence unless they are developed, whereas serious errors
will usually be detected and corrected, either by the investigator
before publication or by the scientific community soon afterward. This
is why it is essential to be broad-minded, and to consider the potential
importance of a piece of research as well as to search for flaws. "
"The above is not intended to suggest that we should endeavor to be
less critical. Serious criticisms and substantial concerns should
always be expressed, but this can be done by dispassionate language
without indulging in _ad hominem_ assaults.
"Critics of art and literature, whose criticisms are often published
and who earn their keep from them, feel they must entertain their
readers with a rapier wit, caustic comments, piercing put-downs, and
acid cuts. Many appear to have decided that criticism is a written
version of prize fighting, except that in boxing, low blows are against
the rules. Leon Wieseltier calls it "aggression as an intellectual
"Unfortunately, all too many negative scientific reviews seem to
have been written by put-down artists. This is not only cruel and
cowardly (at least the literary critic signs his piece), but it
minimizes their influence as well. The editor, panelist or program
director is driven to sympathize with the victim. This may mean that if
you and you alone noticed the flaws, but reported them intemperately,
your criticism will be ignored. Moreover, such aggressive attacks leave
a lasting impression of unprofessionalism on your part."
"...Dr. Janet V. Dorigan of the Department of Energy has observed
that when scientists are under attack, they circle 'round, wagon train
style. The physicists aim outward at their opponents. Biologists, on
the other hand, aim inward, at each other. Their weapons, of course,
are disparaging reviews and negative comments...."
"In order to convey a more accurate impression of the view of our
collective labors, we all need to make a conscious effort to tolerate
diverse ideas and unconventional approaches, and to promote independence
and originality. Robert Reich has written that "Technological
innovation is largely a process of imagining radical alternatives to
what is currently accepted." In our reviews, we must encourage that
dissent and emphasize the advances it will make possible..."