Thursday, March 11, 2010

What if you published an article, but nobody read it?

Here is some advice for grad students and beginning academics. When you publish an article in a major journal, don't assume people will start falling over themselves to congratulate you, or to comment on your paper, or even to acknowledge that they have seen it (much less read the paper). This is one of the big disappointments of starting out in academic publishing, and I don't think I am alone in feeling this way. When I email someone about one of their recent publications, I sometimes get a reply like "its nice to know that at least someone has read the paper." I recall my disappointment as a grad student when my first publications seemed to elicit no reaction. Then Robert Santley told me at an SAA meeting that his grad seminar had read my first major article. I felt great for about 5 seconds, until he told me that they critiqued the hell out of it. Well, at least it got some reads and reactions.

In 2003 and 2005 I published two critiques of research that purports to show that Classic Maya cities were built as cosmological models of the universe (these can be downloaded on my webpage).

2003 Can We Read Cosmology in Ancient Maya City Plans? Comment on Ashmore and Sabloff. Latin American Antiquity 14:221-228.

2005 Did the Maya Build Architectural Cosmograms? Latin American Antiquity 16:217-224.

I thought these were pretty strong critiques, and I anticipated a lively debate with Mayanists and others. These papers were deliberately polemic in tone, a device I used to try and elicit response and critique. I think it is likely that there may have been cosmological influence on Maya city planning, but the published arguments were based on poor methods and speculation. I had hoped that these papers might lead to improved methods and more rigorous work in this area.

But apart from one long email and a serious response from Anthony Aveni (see below), I got virtually no response to these papers. They are among my most infrequently-cited papers today. Why is that? I really don't know, but a few possibilities come to mind.
  1. My arguments were so overwhelming and convincing that no one would even think of taking a contrary perspective now. Yeah, right. As if.
  2. My arguments were so poor, based on faulty methods, assumptions, and data, that no serious scholar would take them seriously. They are beneath contempt, not worthy of reply by serious scholars. Well, I suppose this is possible, but I think I would have heard at least some indication of this at some point. A friend might have pointed this out to me, or perhaps someone who shares my views might have complained at what a lousy job I did in my critiques of the cosmology interpretations. Or the journal would not have accepted the papers.
  3. Well, I can't really think of a third alternative. I find this very puzzling.
Tony Aveni did take my arguments seriously. I heard him speak at some conferences, in which he said that I had raised some important points, and that Mayanists should use my critique to improve their methods and arguments. Hear, hear. That is just what I intended. Tony even went so far as to reprint these paper in a reader on archaeoastronomy and native astronomy:

Aveni, Anthony F. (editor)
2008 Foundations of New World Cultural Astronomy: A Reader with Commentary. University of Oklahoma Press, Norman.

So these papers are in the strange position of having been reprinted in a reader (usually a sign of good, influential papers), while remaining among my most infrequently cited papers.

I was reminded of this situation by two recent works. First, I was reviewing David Henige's work on pre-European New World demography. Henige mentions critiques that he and others made of the "high-count" school of historical demography, and notes that their various methodological criticisms have never been answered. The high counters keep making old arguments, while ignoring criticisms of their work. As someone who follows this research, I find this frustrating; Henige must find it VERY frustrating.

Henige, David P.
1998 Numbers From Nowhere: The American Indian Contact Population Debate. University of Oklahoma Press, Norman.

2008 Recent Work and Prospects in American Indian Contact Population. History Compass 6(1):183-206.

And today, I was reminded of these papers again in a book review by David Nicholas on Keith Lilley's new book on the role of cosmology in the design of medieval cities. Nicholas is very critical of Lilley's data and interpretations, and his arguments are very similar to mine. The empirical evidence for using a geometric/cosmological model of Jerusalem to lay out cities is extremely weak (a few possible cases out of hundreds of cities); and even if this argument were granted, such a feature would make almost no difference in the lives of the people who lived in the cities. Needless to say, I fired off an email to Nicholas telling him how brilliant his review is!

Lilley, Keith D.
2009 City and Cosmos: The Medieval World in Urban Form. Reaktion, London.

Nicholas, David
2010 Review of City and Cosmos by Lilley. H-Urban, H-Net Reviews (online). .

I apologize for rambling at such length about a minor peeve of mine. But the larger issue, which I still find puzzling after more than two decades in the trenches, is the lack of citation and acknowledgment of published papers in archaeology. And I don't think it is just me.


Jason Ur said...

If it makes you feel any better, we read both of the Latin American Antiquity articles in my landscape archaeology seminar, along with Ashmore, Aveni, Wheatley, and that Cambridge Archaeological Journal issue with the articles on planned cities. I bet if you could track your citation rate in graduate class papers, your numbers would go up.

Hans said...

I wonder if the lack of citation stems from the visibility of the Ashmore and Sabloff paper, at least in terms of citations; ISI Web of Science finds just over 1/3 as many citations for your 2003 paper as the Ashmore 2002 paper. A cursory examination of a few other papers with specific (non-letter) replies/comments found this fall off was about average (highest reply:original percentage was just over 50%, smallest was 3%).

That doesn't invalidate the point, of course. It is a curious pattern, however, that critiques are so rarely cited.

Keith Lilley said...

The review by Nicholas of 'City and Cosmos' requires some context - I would urge you to read the book first rather than the review, and you'll see there that the evidence I put forward is rather more compelling than Nicholas would give me credit for. Besides, how do you know that "such a feature would make almost no difference in the lives of the people who lived in the cities"? The evidence is there: please read the book!

Michael E. Smith said...

for Keith Lilley: Thank you for your comment. You are absolutely right. This is an example of the down side of blogging - I fire something off half-cocked, without pausing to reflect. I do, in fact, intend to look at the book; my quip about skipping the book was intended as a clever phrase, not an accurate statement of intention. I own your book, Urban Life in the Middle Ages, and think it is the best book by far on the topic, and I also have copies of 6 or 8 of your papers, which I like very much. My colleagues in anthropological archaeology tend to be very parochial about comparative urbanism and urban theory, and so I often recommend your work to them.

So, I do look forward to reading the book. I am a big fan of Amos Rapoport's work on high-level meaning, but I have been told by medievalists that his notions do not work well for many aspects of medieval architecture and urbanism. I look forward to learning more on this.

I sometimes wish there were venues where people could argue over intellectual issues, but more formal and professional than blogs, but more timely than journals. Anyway, I appreciate your comment and I apologize for my half-baked remarks.

Jerry Sabloff said...

Mike -- you might point out that Wendy Ashmore and I did reply in print to your article. Even if we did not agree, we certainly didn't ignore your criticisms.
I enjoy your blog. Thanks!

Michael E. Smith said...

Jerry- Sorry, I didn't mean to leave out your published reply to the first paper, which is interesting and useful:

Ashmore, Wendy, and Jeremy A. Sabloff (2003) Interpreting Ancient Maya City Plans: Reply to Smith. Latin American Antiquity 14:229-236.

What I was puzzling over was the lack of subsequent references to the papers or to that exchange. Why haven't we seen things like, "Smith's argument was effectively refuted by Ashmore and Sabloff", or, "Ashore and Sabloff did not succeed in countering Smith's critique." Perhaps people have just lost interest in the topic, I don't know.

Anonymous said...

As journals have gone electronic, perhaps they should offer an ego-stroking service to their contributors. Authors could receive an email everytime anyone accessed the full text of their article.

I do understand the need to know if people are actually reading and using the works we produce. Our work is pointless if nobody can use it.


Anonymous said...

I refer to one of your articles in my own critique of cosmological modelling in Mayanist studies:

Normark, Johan
2008.The triadic causeways of Ichmul: virtual highways becoming actual roads. Cambridge Archaeological Journal 18(2):215-237.