Sunday, August 26, 2012

Some memorable reviews of articles

Most review of manuscripts for journals are rather pedestrian. This part is fine, that part needs work, the text on the map can't be read at that scale, cite so-and-so, cite me, etc. Sometimes the reviews are more memorable, either in a positive or a negative fashion. Here are my recollections of three reviews of manuscripts of mine by reviewers for journals. They follow a progression from amusing to annoying.

1. Cite yourself.

Brian Tomasewski and I published a paper in the Journal of Historical Geography. This was an analysis of places in the Toluca Valley, based on the spatial depiction of the specific towns mentioned in individual native historical sources. We used those data to make come inferences about changing political dynamics. The journal sent the manuscript blind to reviewers--that is, the authors' names were omitted. One reviewer provided a helpful detailed review, but complained that the paper didn't cite Michael Smith's work sufficiently! Smith has worked in the Toluca Valley, noted the reviewer, and many of his papers are relevant to this manuscript. Then, the journal editor evidently didn't make the connection between the Smith mentioned in the review and the name of the second author, and asked us to cite this guy Smith. Now some authors cite themselves too much and others too little. I sometimes worry that I am closer to the former than the latter position, so I try not to go overboard. I later told the reviewer I was a co-author and we had a good laugh.

Tomaszewski, Brian M. and Michael E. Smith  2011    Politics, Territory, and Historical Change in Postclassic Matlatzinco (Toluca Valley, central Mexico). Journal of Historical Geography 37: 22-39.


2. You should have written a book, not this paper.

All too often, reviewers complain that an author has not written the paper that the reviewer would like them to write. In this case, I had not written the BOOK the author had wanted. One reviewer of this paper gave it a critical review--twice. First for a journal that rejected the paper, and then for the journal that eventually published it. As I recall, the criticisms were very general, more like objections to my overall approach than specific problems with the manuscript. After extensive revision (thanks to a number of very helpful and detailed critiques), the paper was published. This reviewer later told me that it would take a book to produce the kind of work they wanted me to write, not an article!

Smith, Michael E.2010    Sprawl, Squatters, and Sustainable Cities: Can Archaeological Data Shed Light on Modern Urban Issues? Cambridge Archaeological Journal 20: 229-253.

3. Yankee Imperialist Pig.

Shortly after the publication of an edited volume, a colleague in Mexico asked me to submit a paper (in Spanish) to a local journal on the general subject of one of my chapters in that volume. This person suggested that a Spanish translation of the chapter would be sufficient, but I thought it would be more appropriate to write a new paper based on the original chapter but with some new material and perspectives. I wrote the paper and submitted it to the journal. One of the reviewers was harshly critical, complaining about lazy U.S. scholarship. How dare I merely translate a book chapter and try to pass it off to a Mexican journal as a separate paper! And what's worse, the manuscript does not even say that it is a translation of a book chapter. This kind of arrogant scholarship, an example of academic imperialism, should not be tolerated!

I was really angry and ready to withdraw the paper. Luckily, my colleague smoothed things over and the paper was published. Authors often complain that a reviewer didn't really read the paper carefully. In my case, it is clear that the reviewers did not compare the paper to the original book chapter.

Academic imperialism is something archaeologists working in foreign countries need to watch out for. Academic imperialism refers to foreign scholars sweeping in to a foreign country, doing their research, and leaving, without much interaction with their local colleagues and without publishing in the journals of the country and region. I work hard to avoid academic imperialist practices, which is one reason I was so angered by the clueless review.


Publication Papers said...

"Academic imperialism is something archaeologists working in foreign countries need to watch out for. Academic imperialism refers to foreign scholars sweeping in to a foreign country, doing their research, and leaving, without much interaction with their local colleagues and without publishing in the journals of the country and region." I would agree.

Unknown said...

Thank you for your intermittent posts on your experiences with peer review. As a more junior archaeologist with fewer review battle scars, it's very encouraging to see that these things really do happen to everyone!

Michael E. Smith said...

@Unknown-- These are some of the interesting reviews, not the horror stories. I have been lucky to have received a relatively modest number of nasty, or completely clueless, or otherwise unprofessional and useless reviews of my papers and proposals. There have been a few, but I tend not to remember them.

But I have seen some reviews of manuscripts from (mostly junior) colleagues that really step over the bounds or propriety. To paraphrase some of these:

"I truly hate your dissertation chair, so anything you do must be worthless by definition." or

"If reviewers were as nit-picky with my own work as I am with your manuscript, I would never publish anything at all." or

"Your theoretical position is so contrary to mine that this must be a bad paper."

And I haven't said anything about bone-headed or clueless or biased decisions by journal editors .........

Anonymous said...

How come you knew how your reviewers were, in a couple of those cases? Is it just easy to figure out in your area of expertise (e.g. highly specialized), or do reviewers tend to reveal themselves in your discipline (whether during or after the review process)?

Michael E. Smith said...

Good question. My fields (Mesoamerican archaeology; ancient urbanism) are relatively small and many of us know one another. Of the two reviewers in this post that I identified, one emailed me with a copy of the review, and the other is a good friend with a distinctive style and set of knowledge that was easy to figure out.

It can sometimes be a good idea to send a copy of one's review directly to an author. A reviewer of a paper of mine for a well-respected but disorganized journal sent me her review once. Then the journal either lost her her review (likely), or perhaps never received it. When they complained to me that they couldn't find reviewers for my manuscript, I told the editor that I had a copy of one of the reviews. So I sent that review to the journal!!!! But most journals don't lose reviews (at least I hope not!).

Anonymous said...

Thanks for that. So reviewing is usually single-blind in archaeology, in the sense that the reviewers can see who the authors are? (Or maybe the reviewers are just able to figure out who the authors are?)

I thought single-blind reviewing was mostly a natural science thing.

Michael E. Smith said...

Most journals I am familiar with use single-blind reviewing, although some use double-blind.

Anonymous said...

Good post. What about conflict of interest. I have had a paper in a hellish review cycle for about 2 years now. The editor keeps holding me to the almost ad hom comments of one reviewer. This reviewer keeps citing herself so I know who it is, and she happens to be a prof at the same institution as 4 of the co-authors. How do I suggest to the editors that there might be a real conflict of interest at play here?


Michael E. Smith said...

@JG - I don't know enough of the context to really understand your situation. But in most cases the best recourse is a frank discussion with the journal editor. Their job is to negotiate through divergent opinions and potential conflicts of interest. If one does not trust the editor, then there is not much recourse. But you should not hesitate to tell the editor if you think there is a conflict of interest (or a potential one), or if you think that a particular reviewer is not being fair. Perhaps you are aware of a reason for a reviewer's problematic review that the editor doesn't know about.

Email is the best way to communicate with editors like this, since you will have a record of the exchange. But some editors may want to discuss delicate situations on the phone. But in any case, you need to communicate frankly with the editor.

This may not be relevant to your case here, but I have found that authors not infrequently assume or complain that editors are biased in one form or another. While I have seen cases of editorial bias having a negative effect on manuscript review, they seem pretty rare. Negotiating between divergent views is a difficult but important job of journal editors, and the more information they have about individual cases, the better. So contact your editor.