Sunday, December 2, 2012

How do archaeologists make arguments?

An unfortunate trend in archaeology today consists of making arguments using faulty logic and faulty methods of argumentation. This is one of my biggest beefs with the work of postmodern/postprocessual archaeologists. They don't test models or hypotheses, their theories are true by definition, and their arguments are based more on speculation than on empirical evaluation.

A colleague remarked recently I that don't seem to like archaeological works that deal with things like negotiation, identity, and agency. But these are important social science concepts and they all have their use. In archaeology today, however, they are buzz words. They are used more to signal adherence to a theoretical perspective than as analytical concepts that do intellectual work. Brubaker and Cooper (2000:19) talk about this with respect to identity; when scholars say that identity is "multiple, unstable, in flux, constructed, negotiated, and so on," these terms serve as "gestures signaling a stance rather than words conveying a meaning." For me, the concept of negotiation as a concept in collective action theory is great (Margater Levi; Blanton and Fargher; etc.). But when used as a postmodern buzz word, it really turns me off.

Anyway, back to arguments. Here is a form of argumentation that I have seen many times in recent years.

(1)  Specify a theme, framed in terms of a high-level philosophical social theory.
(2)  Discuss your archaeological data.
(3)  Claim that the theory provides an interpretation of the data. 


Now there is a problem here, one that some some writers acknowledge and others don't. High-level social theory is too far removed from data to actually provide any kind of empirical evaluation or test. That is, you can't come up with a way of establishing that the theory is correct or incorrect, or even applicable or not applicable. Why? Because high-level social theory is simply true. It floats high above the messy empirical world, impervious to attempts to test or evaluate it. And it is so ethereal that you can't directly evaluate it on the ground. If this isn't clear, check out Ellen (2010, Smith (2011), or Abend (2008). In the words of Kevin Fisher,

"While Giddens, Goffman and others [that is, high-level social theory] provide an overall theoretical orientation or examining the relationship between architecture, interaction and social transformation, their work does not offer the tools needed to analyze the material remains on the ground" (Fisher 2009: 440).

So only very naive scholars will stop with step #3. Many archaeologists recognize these limitations of high-level social theory, and they see the need for some kind of middle-level construct to justify their claim. Here are two faulty alternatives I've seen in recent publications:

(4A) Cite another study, perhaps from another region, where a similar argument was made. If someone else has already made the claim in another context (whether or not it is warranted, correct, or applicable), then I can make a similar claim in my context. The technical term for this kind of ploy is "empty citation" (Harzing 2002). This is a  citation to a work that does not contain any original data for the phenomenon underconsideration; the works are cited merely to lend an aura of support for anargument, when in fact they contain no empirical support. This is not a valid form of empirical argument.

(4B) Cite a study showing that one's particular interpretation fits with some comparative data, somewhere in the world. This is a definite epistemological improvement over step 4A, but it still does not constitute a valid argument. For example, suppose I claim that imported goods from distant lands were seen as imbued with power and magic at sites I have excavated. It will be hard or impossible to confirm this statement empirically with archaeological data, but I can cite Mary Helms to the effect that in some documented societies, this was indeed the case. But unless Helms shows that this is always the case, that the pattern exists in every known ethnographic and historical society, then this citation only provides plausibility for my argument. It certainly does not provide confirmation, or even empirical support short of confirmation. Now you can turn this into a reasonable analogical argument with a good comparative study. Pick a sample. Determine the contextual conditions under which exotic goods are seen to have power, and those under which that situation does not obtain. Compare your case to the comparative data, and make the appropriate interpretation. But no one seems to do this because it is much easier to make a quick speculative interpretation than to go out and do a big comparative study.

Well, I guess we simply cannot test our high-level social interpretation of archaeological data. Let me quote the clear-thinking sociologist Andrew Abbott (2004)  here:


  • “in order to be tested, all of these ideas and implications must be framed in such a way that they can be wrong.” (p.215)  [See my prior post, where I reference Steve Haber's discussion of this issue --MES]
  • “it is surprising how many researchers—even graduate students in their dissertations—propose arguments that can’t be wrong. For example, research proposals of the form, ‘I am going to take a neo-institutionalist view of mental-hospital foundings’ or ‘This paper analyzes sexual assaults by combining a Goffmanian account of interaction and a semniotic approach to language’ are not interesting because they do not propose an idea that can be wrong. They boil down to classifying a phenomenon or, seen the other way around, simply illustrating a theory.” (p.216)
  • “Thinking without alternatives is a particular danger in ethnography and historical [and archaeological --MES] analysis, where the natural human desire to develop cohesive interpretations (and the need to present a cohesive interpretation at the end of the research) prompts us to notice only those aspects of reality that accord with our current ideas.” (p.216)
So if this common form of archaeological argumentation (high-level theory + data = speculative interpretation) is faulty, then how SHOULD archaeologists construct arguments? If you haven't read The Craft of Research  by Booth et al (2008), go buy the book and read it right now. Although designed for undergraduates, it contains much of value to graduate students and professionals. Here is how they suggest making arguments (chapter 8, "Making Good Arguments"):


  1. What is the claim?
  2. What reasons support the claim?
  3. What evidence supports the reasons?
  4. Acknowledge alternatives / complications / objections
  5. What warrant or principle justifies connecting the reasons to the claim?


I won't elaborate on this scheme; I don't want to give you an excuse to avoid looking at Booth et al. And one more suggestion, if you have ANY interest at all in the topic of identity, and you have not read Brubaker and Cooper (2000), you are doing yourself a big disservice.

References:

Abbott, Andrew
2004    Methods of Discovery: Heuristics for the Social Sciences. Norton, New York.

Abend, Gabriel
2008    The Meaning of "Theory". Sociological Theory 26: 173-199.

Booth, Wayne C., Gergory G. Colomb, and Joseph M. Williams
2008    The Craft of Research. 3rd ed. University of Chicago Press, Chicago.

Brubaker, Rogers and Frederick Cooper
2000    Beyond "Identity". Theory and Society 29: 1-47.

Ellen, Roy
2010    Theories in Anthropology and "Anthropological Theory". Journal of the Royal Anthropological Institute 16: 387-404.

Fisher, Kevin D.
2009    Placing Social Interaction: An Integrative Approach to Analyzing Past Built Environments. Journal of Anthropological Archaeology 28: 439-457.

Harzing, Anne-Wil
2002    Are Our Referencing Errors Undermining our Scholarship and Credibility? The Case of Expatriate Failure Rates. Journal of Organizational Behavior 23: 127-148.

Smith, Michael E.2011    Empirical Urban Theory for Archaeologists. Journal of Archaeological Method and Theory 18: 167-192.



7 comments:

Al West said...

"For me, the concept of negotiation as a concept in collective action theory is great (Margater Levi; Blanton and Fargher; etc.). But when used as a postmodern buzz word, it really turns me off."

Definitely one of the worst things about the whole postmodern phenomenon. It's hard to find ways to discuss these things that don't employ this over-used jargon.

Robert said...

This is a great blog post. As I have been developing as a grad student, I have been stunned at how anthropologically trained colleagues do not seem to have a clear idea as to what they are doing when they construct arguments. Arguments are superficial and confused. It is very disappointing because I do not think that this was always the cast in the discipline.

- said...

I think you are committing a mistake by evaluating the use of post-modern theory with the criteria of positivist science. Saying that one uses concepts from Goffman and Semiotics is a statement about what vocabulary one is casting the description, not a claim that one is testing whether Goffman or C.S. Peirce were right or wrong.

It is possible to learn something new by applying a theory without using the theory to formulate a falsifiable hypothesis.

The reason it makes sense to use these kinds of social theory is not because we want to see if the theory is right, but because we can use the theory to look at data from a new perspective.

High level social theory is good for formulating new questions, because each theory is a way of looking at and interpreting the world as a system of meaning. Yes, archeologists have to find ways of connecting these questions through the data by using midlevel theory to formulate working hypotheses. Using post-modern concepts is only useful if they generate interesting new questions. I think that mostly they do, although perhaps some archeologists do seem to use them simply as a signifyier while they actually keep working in the positivist-processual mindset.

Michael E. Smith said...

My response to this takes the form of two food metaphors. First, the proof is in the pudding; and second, Where's the beef? I would not want to claim that high-level social theory is completely worthless. But if one's focus is on empirical analysis of society, (including a search for causes and explanations; descriptions of how different social institutions relate to one another; etc.) then I have yet to be convinced that high-level social theory is of much use in that endeavor. If, on the other hand, one is interested in a philosophical understanding of society and culture, social theory is probably important.

More than ten years ago, a new graduate student told me confidently that he/she was going to show how practice theory could be applied in a useful empirical sense to archaeological data. That student did not fulfill the promise, and I am still waiting.

- said...

I think it is reasonable that some archeologists should study "causes and explanations; descriptions of how different social institutions relate to one another; etc." But I also think that if that is all archaeology can do then its usefulness is rather limited. The accounts it can produce is limited to series of facts - someone threw a pot here, someone was buried there. I believe that another objective of archaeology should be to find ways of approaching how people in the past made sense of the worlds in which they lived, and for that "a philosophical understanding of culture" is indispensible in my opinion. I think many of the archeologists whose use of theory you dismiss as mere parading of buzz-words (perhaps also the work of your practice theorizing graduate student) falls into this second type of archeology.

Michael E. Smith said...

If the choice for archaeology were between where people threw their pots and the philosophical thought processes of ancient people ("how people in the past made sense of the worlds in which they lived"), as you suggest, then I would be give up today. Archaeology wouldn't be worth the effort. Yeah, I want to know where they threw their pots, but I have little interest in how they made sense of their world. Not that the latter is not interesting for some reasons, but largely because I think it is impossible to do for the distant past without written evidence.

These issues are hard to debate within archaeology because the discipline has had such an idiosyncratic and perhaps bizarre recent intellectual history, and a resultant impoverished vocabulary. The scientific crowd got off on the limb of logical positivism, not realizing that the limb had collapsed before they even climbed on. Meanwhile the intprepretivist humanities crowd gave up on the scientific tree altogether. Within archaeology, we don't even have the terminology, much less the theoretical tools, to debate these things.

Take a look at empirical social science research: Robert Sampson in sociology, John Gerring in political science, Charles Tilly in historical social science, Daniel Little in the philosophy of social science. This is what I want to do, but in the deep past and with archaeological data. These guys don't get hung up on philosophical social theory and worrying about how people make sense of their world. They employ causal models for behavior, social structure, and institutions. Their goal is to describe and explain the social world, using what Merton called middle-range theory.

Middle-range theory sits far above the level of where people threw their pots, and far below the level of practice theory, actor-network theory, Foucault, and other kinds of philosophical social theory. It is called "middle-range" theory because it is in the middle with respect to types and scales of theory.

I talked to a social theorist in December (Stephen Read, from Delft), and I asked him why postprocessual archaeologists (Ian Hodder and Matthew Johnson in particular) are hostile to the notion that theory exists on multiple levels (see my urban theory paper, where I point this out but in a puzzled way). This seems to be part of their hostility to Mertonian middle-range theory. Your choice of options as being formation processes or philosphical theory also leaves out middle range theory. Read was amused at theorists denying the existence of multiple levels of theory, and suggested that it came from adhering too closely to Cartesian duality. Now I can't evaluate that notion (too abstract for me), but at least it might make sense of published statements by Hodder and Johnson, who clearly are hostile to middle-range theory (of both the Binfordian and Mertonian versions).

I am less interested in making grand pronouncements about archaeology (although I seem to do a bunch of that in this blog. But see my publications for more rigorous treatments of issues relating to theory and research). My major interest is to promote and carry out an archaeology that is a comparative, historical, causal, and explanatory social science. I want to be able to talk to other social scientists (of the ilk I mentioned above) who may have some interest in my data and findings. In my experience, those folks want to know about the social science of the past -- not about ancient thought processes nor about where they tossed the trash.

If others want to spend their time with high level social theory, that is just fine. They probably want to interact more with literary theorists or postmodern scholars in other disciplines. But I have yet to see any reason to bother with philosophical social theory, since it does little to promote the kind of archaeology I want to do.

- said...

Yes, it boils down to taste and interest. The kind of archeology you want to do is fine and respectable (I read your blog because I know and thoroughly appreciate your work, and I have taught your book on the Aztecs and several of your articles) - but I also think it is reasonable that other archeologist might have different interests and that it should be possible for archeologists with different interests to have meaningful conversations without belittling eachother's choice of perspective. I do agree that mid range theory is useful for connecting high level social theory to the pots and skeletons in the ground. I for one am interested in how people in the past made sense of their world, and less interested in making ceramic chronologies, but I am glad someone else does the chronologies so that others can focus on producing the kind of knowledge that interests me.