One benefit of the epistemology part of the class is that it has helped me organize my thoughts, and given me a better understanding of just what is wrong with much of the work published in archaeology today. In short, many archaeologists don't know how to make a solid argument. I've talked about this previously. They don't know how to put data together with theory to reach a rigorous conclusion about what likely happened in the past and why it happened. So, I plan to write an article about this problem. Right now I view the paper partly as instructions for students (here is how to make a good argument, and here are some pitfalls to avoid), and partly as a critique for the profession. One difficulty in writing this paper will be how to handle examples. Maybe I am getting soft in my old age, but I am not anxious to get a bunch of colleagues pissed off at me for featuring their work as negative examples. I seem to do a pretty good job of annoying people without deliberately poking a bunch of others with a stick. But would a journal editor accept an article about methodological and epistemological problems that did not include a lot of real examples?
Anyway, I think I'll use this blog to organize some of my thoughts for my paper on arguments. So let's start with a quote from economic historian Steve Haber:
- “the fundamental question of all serious fields of scholarly inquiry [is]: How would you know if you are wrong?”(Haber 1999:312)
Here is how Andrew Abbott (2004) discusses the problem of ensuring that you can be wrong:
- “it is surprising how many researchers—even graduate students in their dissertations—propose arguments that can’t be wrong. For example, research proposals of the form, ‘I am going to take a neo-institutionalist view of mental-hospital foundings’ or ‘This paper analyzes sexual assaults by combining a Goffmanian account of interaction and a semiotic approach to language’ are not interesting because they do not propose an idea that can be wrong. They boil down to classifying a phenomenon or, seen the other way around, simply illustrating a theory.” (p.216)
Abbott goes on to remark that,
- “Thinking without alternatives is a particular danger in ethnography and historical analysis, where the natural human desire to develop cohesive interpretations (and the need to present a cohesive interpretation at the end of the research) prompts us to notice only those aspects of reality that accord with our current ideas.” (p.216)
Does this sound familiar? As archaeologists we certainly want to write cohesive narratives, and this desire may lead us away from considering alternative ideas. One of the problems with using post-hoc interpretations, or what Lewis Binford called post-hoc accomodative arguments (see my prior discussion here), is that they can't be wrong. They are made up after the fact, after the research is done, and they are designed to fit the results you found. So almost by definition, such interpretations must be correct. They fit the data, and you haven't compared them to any other interpretation. Now there is nothing wrong with devising some interpretations after all the facts are in. But such interpretations should serve as input for further research that is designed to test them against new data. Otherwise they will remain a particularly weak argument.
So, how do you avoid doing research and constructing arguments that can't be wrong? Here are two suggestions:
(1) Devise multiple working hypotheses, and test them all. The one that best survives its battle with the empirical world, the one that is the last hypothesis still standing, is your best explanation. This concept of "multiple working hypotheses," usually with a mandatory citation of Chamberlin, comes up occasionally in archaeology. In fact, this method is part of the approach known as "strong inference." Want to hear more? Stay tuned. This will be one subsection of my paper, and I'll talk about it here before too long.
(2) Choose your theory wisely. If your theory is very abstract and philosophical, at a high epistemological level, then you can't test it against data, and you can't show that your interpretation is wrong. On the other hand, if your theory is of the middle range or low range (what I have called "empirical theory" - Smith 2011), then it CAN be tested against the empirical record, and it can be falsified.This is another well-established point in social science epistemology (see, for example, Ellen 2010, or see the discussion in my 2011 paper), although the post-processualists don't believe that there are different epistemological levels of theory (see footnote 4 to my 2011 paper on this).
For example, suppose I am working on regional ceramic exchange in Postclassic central Mexico. I could decide to test a middle-range theory, such as the proposition that "increasing involvement in the long-distance exchange networks of a commercialized economy will lead to higher levels of local and regional exchange at my site." I know from past work, and work in other areas, that the Late Postclassic period was a time of growing commercialization and increasing long-distance trade throughout Mesoamerica. I find more long-distance imports in my site through time. So I do some petrographic analysis and INAA of a sample of sherds from different time periods. I may find my hypothesis supported (that is, there are higher frequencies of regional imports through time), or I may find that I was wrong (frequences of local/regional imports declined when I expected them to increase). This kind of research, using middle-range theory, can lead to results that either confirm or deny my initial hypothesis. (or, they may lead to complex results that leave me scratching my head.....).
Now, consider an alternative use of theory. Suppose I choose to use a high-level abstract theory: post-structuralism. The growing influence of commercial exchange created social contradictions and tensions in local society. Political institutions at my site were instantiations of socially embedded practices. People negotiated their identity by using ceramics from different places of origin in arenas of display and tournaments of value. My analytical tests reveal the places of origin of regionally imported ceramic types, consistent with the notion of a fragmented and contested, yet socially embedded, economy. Now, can this interpretation be wrong? How could I convince a skeptic that it is correct and supported by data? High-level, abstract social theory simply cannot be tested, and it cannot be directly evaluated with data.
Anyway, sorry to go on at such lengths. I'm sure that clever social theory types will recognize my postcolonial scenario as a bunch of gobbledygook. But I think this issue of "how would you know if you are wrong?" lies at the heart of many problems of research and writing in archaeology today. Now, maybe some archaeologists don't see the need for rigorous testing of hypotheses or for designing arguments that can be proven wrong. Maybe a scientific epistemology is not valuable to these people. Maybe they don't want to be competitive for grants from the National Science Foundation. Fine. But for the rest of us, we should strive to make better arguments. If you haven't read Booth et al. (2008), that is probably the best place to start.
My paper about arguments will also cover good and bad uses of analogy, the argument template contained in Booth et al (2008), empty citations, natural experiments, strong inference, causal mechanisms, and perhaps even Monty Python's Argument Clinic (I've always wanted to cite a Monty Python sketch in a serious scholarly paper, and this may be my best chance yet).
2004 Methods of Discovery: Heuristics for the Social Sciences. Norton, New York.
Booth, Wayne C., Gergory G. Colomb, and Joseph M. Williams
2008 The Craft of Research. 3rd ed. University of Chicago Press, Chicago.
2010 Theories in Anthropology and "Anthropological Theory". Journal of the Royal Anthropological Institute 16: 387-404.
2007 Case Study Research: Principles and Practices. Cambridge University Press, New York.
1999 Anything Goes: Mexico's "New" Cultural History. Hispanic American Historical Review 79: 309-330.
2008 Salsa Dancing Into the Social Sciences: Research in an Age of Info-glut. Harvard University Press, Cambridge.
Popper, Karl R.
1934 The Logic of Scientific Discovery. Harper and Row, New York.
Ragin, Charles C. and Lisa M. Amoroso
2011 Constructing Social Research: The Unity and Diversity of Method. 2nd ed. Sage, Thousand Oaks, CA.
Smith, Michael E.2011 Empirical Urban Theory for Archaeologists. Journal of Archaeological Method and Theory 18: 167-192.