Critiquing scientific papers


You will be asked to review and critique scientific papers in this course.

If you have little experience in reading and writing scientific papers, then you may wish to consult Day's "How to write and publish a scientific paper."

If you have experience reading (and perhaps writing) scientific papers, then you might follow guidlines list below, which are the "Guidelines for Reviewers" of manuscripts submitted for publication in the journals of the Ecological Society of America.

COMMENTS FOR THE AUTHORS - What is the major contribution of the paper? What are its major strengths and weaknesses? Pleas include both general and specific comments bearing on these questions, and EMPHASIZE your most significant points.

GENERAL:

  1. importance and interest to Ecologists
  2. scientific soundness
  3. originality
  4. organization and clarity
  5. cohesiveness of argument
  6. degree to which conclusions are supported by the data
  7. length relative to the number of new ideas and information
  8. conciseness and writing style

SPECIFIC:

Support your general comments with specific evidence.

  1. Presentation - Does the paper tell a cohesive story? Is it a tightly reasoned argument evident throughout the paper? Where does the paper wander from this argument? Do the title, abstract, key words, introduction and conclusions accurately and consistently reflect the major point(s) of the paper? Is the writing concise, easy to follow, interesting?
  2. Length - What portions of the paper should be expanded? condensed? combined? deleted? (PLEASE don't advise an overall shortening by X %. Be specific!)
  3. Methods - Are they appropriate? current? described clearly enough so that the work could be repeated by someone else?
  4. Data presentation - When results are stated in the text of the paper, can you easily verify them by examining tables and figures? Are any of the results counter-intuitive? Are all tables and figure clearly labeled? well planned? too complex? necessary?
  5. Statistical design and analyses - Are they appropriate and correct? Can the reader readily discern which measurements or observations are independent of which other measurements or observations? Are replicates correctly identified? Are significance statements justified?
  6. Errors - Point out any errors in technique, fact, calculation, interpretation, or style. (For style, follow the CBE Style Manual, Fifth Edition, and the ASTM Standard for Metric Practice, E-380-84).
  7. Citations - Are all (and only ) pertinent references cited? Are they provide for all assertions of fact not supported by the article itself?
  8. Overlap - Does this paper report data or conclusions already published or in press? If so, please provide details.

 

FAIRNESS and OBJECTIVITY -- If the research reported in this paper is flawed, criticize the science, not the scientist. Harsh words in a review will cause the reader to doubt your objectivity; as a result, your criticism will be rejected, even if they are correct!

Comments directed to the author should convince the author that:

  1. you have read the entire paper carefully
  2. your criticisms are objective and correct, are not merely differences of opinion, and are intended to help the author improve his or her paper, and
  3. you are qualified to provide an expert opinion about the research reported in this paper

If you fail to win the author's respect and appreciation, your efforts will have been wasted.

 

ANONYMITY -- You may sign your review if you wish. If you choose to remain anonymous, avoid comments to the authors that might serve as clues to your identity, and do not use paper that bears the watermark of your institution.


Please submit your review in four parts:

  1. The rating form
  2. Comments for the author (DO NOT include your recommendation to accept or reject the paper)
  3. the manuscript (with or without comments written on it)
  4. A cover letter to the editor, providing a brief, candid summary of your opinion of the paper.
  5. How to write a review paper

A review paper is not an original publication, although it may contain new data. The purpose of a review paper is to review previously published literature and to put it into some kind of perspective.

Good reviews are much more than annotated bibliographies. The offer critical evaluation of the published literature and often provide important conclusions based on that literature.

A review paper follows a format different from that of a research paper. It might help you visualize the difference if you imagine a review paper is a research paper modified by expanding the introduction, deleting the Materials and Methods; deleting the Results; and greatly expanding the discussion.

Recall that a discussion should address the following points. What do these findings mean? Present the principles, relationships, and generalizations shown by the results. Point out exceptions and unsettled points. Show how the results agree or disagree with previously defined expectations. Discuss the implications and applications of the results. State your conclusions and the evidence that supports them. Discuss problems with the studies reviewed and suggest improvements.

More reviews prefer either a "state of the art" review or reviews that provide a new understanding of a rapidly moving field. Only the recent literature on the subject is catalogued or evaluated. If your are reviewing a subject that has not previously been reviewed or one in which misunderstandings or polemics have developed, a bit more coverage of the historical foundations would be appropriate. If the subject has been effectively reviewed before, the starting point for you review might well be the date of the pervious review (not publication date, but the actual date up to which the literature has been reviewed). And, of course, your review should start out by citing the previous review.

Readers of much influenced by the introduction of a review paper. They are likely to decide whether or not to read further on the basis of what they find in the first few paragraphs (if they haven't already been repelled by the title).

The conclusions section is essential and you should invest great care and consideration in writing it. Good summaries and simplifications are difficult to write, requiring compromises that satisfy both the professional and the amateur. Yet, good summaries will find their way into textbooks and mean a great deal to students yet to come and to the future directions for research.