Best Practices in Refereeing Journal Articles

Charles Pidgin weighed in with some thoughtful advice on best refereeing practices. Said he:

I’ve  had a lucky publishing career, most of my papers having been published in the first or second journal that I have sent them to, usually without major revisions. But I have from time to time encountered pain-in-the–ass referees, and I think the profession would benefit and the refereeing process would be speeded up if referees tried NOT to be pains-in-the-ass. So here are some hints on how do it.

1) Generally speaking, if you think a paper that you have been asked to referee is publishable but suffers from what you consider defects, don’t say ‘Revise and Resubmit’ or  ‘Publish subject to certain revisions’. Say ‘This paper is good to go as it stands, though there are some points the author might like to consider’. Reserve ‘Publish subject to certain revisions’ for papers with a DEFINITE MISTAKE that can easily be corrected. Reserve ‘Revise and resubmit’ for papers which really are not up to snuff but contain a good idea of which something might be made. Why do I think that this would save time? Because people with good papers would not be having to wend their way between the contradictory requirements of diametrically opposed referees. The comments will be there should the author wish to make use of them, but s/he won’t have to explain to the editor at tortuous length why s/he paid attention to this comment but not to that. The risk is of course that more than usually arrogant or tenure-pressed authors won’t pay attention to the comments, and some papers will be published which would have been a lot better if the authors had conscientiously tried to improve them. But I think that this is a cost that the profession can bear.

2)  Try NOT to write comments of the following form:
a) This paper is too long;
b) The author doesn’t say enough about X
At least you should only write comments of this form if you accompany them with helpful suggestions about which bits should be cut to accommodate the new material.

3) Don’t make dopey hat-doffing suggestions. Occasionally I used to get comments complaining that my paper did not doff its hat to some philosophical bigwig of whom I had a poor opinion. The reason I did not was usually because I was applying the modified Thumper principle ‘If you can’t say nuthin’ nice about Professor Big don’t say nuthin’ at all – unless the chief object of the paper is to expose Professor Big’s errors.’ The fact that Professor Big has written something vaguely relevant does not mean that Professor Big’s name has to be ostentatiously dropped. ONLY demand a reference to a famous  Name (or for that matter to a relatively obscure Name) under two conditions:

a) If you think that that Name has put forward a GOOD argument, which poses a direct challenge to one  of the author’s central claims.
b) If Professor Big has partially anticipated the author’s opinions.

4) My last suggestion (unlike the others) would slow things down but in this case, I think it would be worth it. Don’t tolerate obscure and pretentious writing. There are many famous philosophers who  are, if truth be told, wretchedly incompetent writers, obscure, turgid and long-winded, with long involved sentences with multiply nested parentheses. Often they are also affected and pretentious. Some of them are good philosophers despite their bad writing, others obscure their intellectual deficiencies behind the defects of their style. If referees had been a bit tougher on this sort of thing  of thing early on in their careers, we would all have been spared a lot of bad writing and a fair bit of bad philosophy. So if the paper is badly written say so and insist on revisions even if you think that the content is good. Here I think there are exceptions to principle 1) above.

I’d add:

  • Stop procrastinating.
  • Lead with the positive.
  • Don’t raise a substantive objection unless you’d be comfortable raising it non-anonymously. (Objections about style and clarity are a bit different, I think.)
  • Take your motivation to write a long and complicated objection to be an indication (defeasible, of course) that the paper is worth responding to, and so merits publication. (We might still be left with the question ‘does it merit publication in this journal?’.)

Gregory Wheeler also suggested in an earlier thread that referees take the time to summarize, in their own words, the gist of the paper under consideration.

Any other advice (or objections)?


Best Practices in Refereeing Journal Articles — 2 Comments

  1. I think Charles Pigdin’s first point is the most important. Too often referees seem to forget the nature of their task. The question they must answer is “is this paper publishable, as is, or in a state near its current form”? They should write reports with the answer to this question in mind. Doing this, and bearing in mind Professor Pigdin’s other points, would dispose them to spend less time on irrelevant objections, of the following kinds: paper says X, where X could have two interpretations, one of them false. Clearly the author means the other interpretation, but should say so at great length; or paper X makes a claim which entails something that has been debated elsewhere; author should make copious reference to that debate and say why they take the position they do on that debate….

    I think focusing on the argument of the actual paper and avoiding these kinds of issues will help avoid the kind of obscure and pretentious writing that Pidgin fears: it arises, in part, from the need to head off all the objections that a referee who forgets the real nature of their job will likely come up with.

  2. These days I’m increasingly hoping that reviewers and editors will do more along the lines of (4). Perhaps it is the institutional structure of journals, but I find that editors/reviewers more often take heat from authors than they do from readers. One reason is that scholarship that merely criticizes published work (without explicitly “contributing to the development of the field”) has virtually no publishing outlet. There is no “complaints department” or even “letters to the editor” section of journals. Editors are currently more worried about criticism from authors (about unfair or inefficient review processes) than from their readers. In fact, I think readers are reticent about critiquing already published work for fear of damaging the reputation of their field. There is a sort of tacit agreement to promote work in one’s own field as generally of high quality.

Leave a Reply

Your email address will not be published. Required fields are marked *