A good day in Mexico: Carbon, thin sections, an index, and chicharron

I received a number of good things by email this morning, including radiocarbon dates, ceramic petrography results, and a completed index for a book! When it rains it pours. Dealing with this stuff left little time for the sherd drawing I am supposed to be doing in the lab. This is the first time I have used a professional indexer for a book. Normally I index my own books, and I like the process. An index is an important tool, and constructing a good index is an intellectual exercise as well as an organizational task. Don't you hate it when a book has a lame, four-page index and you are trying to find some specific information? Don't you REALLY hate it when a book lacks an index entirely? But indexing takes time, and with three co-editors for this book we decided to hire a professional indexer.

This is a good book, buy it, you'll like it. You see, after going on and on in this blog about how most edited volumes in archaeology are worthless, I can't afford to edit a bad book. (Here is my original post on worthless edited volumes, and a later related post). So any volume I edit now must be good, almost by definition (please suspend your critical thinking skills here temporarily).

Random ceramic thin section (internet)

I can't say much about the petrographic results. This is our first batch of ceramic petrographic samples, submitted by my student Julie Novic and done by Jenny Meanwell of MIT. Julie hasn't had time to see how they look. Do our macroscopic paste types match petrographic reality? What about our ceramic types? Do the petrographic data support our hypotheses on ceramic production, exchange and consumption? Julie is working on neighborhoods and urban spatial organization at Calixtlahuaca, using our surface data (and she is my co-author in chapter 1 of the above book). We have another sampling scheme for petrography for the excavated ceramics, and we should get the results before too long. I'm glad customs or airport security didn't get weirded out by the saw blade I carried to Mexico in my luggage!

The most exciting news today was a new batch of radiocarbon results from the University of Arizona AMS lab. This is the first bunch from our second batch of dates. We are waiting for the entire suite to run them through Ox-Cal, but we are also working with the uncalibrated dates, not to assign ages, but to estimate phase lengths. It turns out that my colleagues George Cowgill and Keith Kintigh wrote a handy-dandy program a while ago using monte carlo simulation to estimate likely phase lengths from a suite of radiocarbon dates. We ran our initial batch of 20 dates, and we will run the entire group when they are all done. The simulation results in conjunction with the calibrated dates will give us estimates for the calendar dates of our ceramic phases. I am in the process of bugging George and Keith to actually publish their nice study and their algorithm, which illustrates some features of radiocarbon results that seem counterintuitive to many archaeologists.

I must admit to my occasional surprise when some archaeological analysis or another comes out with excellent results. With so many potential confounding factors, it sometimes seems amazing when we get solid, rigorous, and clean results about things that happened centuries or millennia ago. It is still soon to get too excited, but the dates look great. Angela and I did the ceramic seriation and defined three phases based solely on ceramic type similarities. Then we looked at the seriated deposits stratigraphically and they were almost always in the right order. So two independent types of evidence agree. Now we look at the radiocarbon ages, and lo and behold the ceramic phases plot out in  nice chronological sequence, with only a very small amount of overlap (yes, I know, once we calibrate the dates it will be much messier with lots of overlap. I have the bad fortune of working in a period when the calibration curve goes back on itself and ALL relevant dates have multiple age ranges. This is where the Cowgill/Kintigh procedure will help).

So what could be better than all this stuff arriving first thing in the morning? Well, our lunch at the lab turned out to be fresh chicharron, avocadoes, and double cream cheese (queso de doble crema). It didn't look exactly like this photo from the internet, but we put the stuff into hot tortillas and it doesn't get much better than that! And I did get a few sherds drawn as well.

"Rigorous evaluation of human behavior"

I've been reading the June 15 issue of Science, and I am struck by the irony of two articles. The first is a news item titled "Social scientists hope for reprieve from the senate." The U.S. House of Representatives recently voted to prohibit the NSF from funding political science research, and to reduce the scale of the American Community Survey (a census-based social survey). The bill was co-sponsored by an  unenlightened congressman from my own (unenlightened) state of Arizona, Jeff Flake (jokes about "what's in a name" come to mind here). "Flake says political science isn't sufficiently rigorous to warrant federal support." Does Flake base his policies and views on rigorous research?  I doubt it. Conservative politicians periodically go after the social sciences in Washington, and we should all hope that the current attack is as unsuccessful as previous ones have been. Is archaeology more or less rigorous than political science? If we use science definition 1, I think we are in bad shape. But we can always take refuge in science definition 2 ("see, we use complicated scientific technology!") to assert our rigor.

The second article in the June 15 Science was a short essay in a section called "Science for Sustainable Development." This essay, titled "Rigorous Evaluation of Human Behavior" is written by Esther Duflo, an economist at MIT. She makes the valid and important point that the role of science in promoting sustainable development and alleviating poverty should include social scientific studies of behavior. I wonder what Jeff Flake would think about this. When I saw the title I was encouraged, but then I got to the heart of Duflo's essay: the way to conduct "rigorous" studies of human behavior is to use randomized controlled trials. "This makes for good science: these experiments make it possible to test scientific hypotheses with a degree of rigor that was not available before."

In some fields of social science and public health, the randomized controlled trial (RTC) has become the supposed "gold standard" of research methods, proclaimed to be far superior to other approaches. Apart from the fact that we simply cannot do RCT's in archaeology (except perhaps in a few very limited situations that I can't think of offhand), I must admit that I am more supportive of the growing critique and contextualization of RCTs in social science. RCT is a narrow approach that achieves internal rigor at the expense of external relevance and validity. Philosopher of science Nancy Cartwright puts it this way, using economics to illustrate the trade-off of internal rigor and external validity:

“Economists make a huge investment to achieve rigor inside their models, that is to achieve internal validity. But how do they decide what lessons to draw about target situations outside from conclusions rigorously derived inside the model? That is, how do they establish external validity? We find: thought, discussion, debate; relatively secure knowledge; past practice; good bets. But not rules, check lists, detailed practicable procedures; nothing with the rigor demanded inside the models.” (Cartwright 2007:18).


Or consider a recent paper by sociologist Robert J. Sampson (2010), who promotes the value of observational research in criminology and sociology. He deflates three myths of RCTs in criminology:


Myth 1: Randomization solves the causal inference problem.
Myth 2: Experiments are assumption (theory) free.
Myth 3: Experiments are more policy relevant than observational studies.


If you want more context on internal vs. external validity, or how various social science methods relate to an experimental ideal, see Gerring (2007). He is one of those political scientists who, according to Congressman Flake, must be non-rigorous. But the message here is analogous to my views on science types 1 and 2 in archaeology. Just as archaeologists can do scientifically rigorous and valid research without involving technological methods from the hard sciences, so too can other social scientists do scientifically rigorous and valid research without the aid of formal experiments (RCTs).

Cartwright, Nancy
2007    Are RCT's the Gold Standard? BioSocieties 2(1):11-20.

Gerring, John
2007    Case Study Research: Principles and Practices. Cambridge University Press, New York.

Sampson, Robert J.
2010    Gold Standard Myths: Observations on the Experimental Turn in Quantitative Criminology. Journal of Quantitative Criminology 26(4):489-500.

Postscript--No, I don't read or keep up with the Journal of Quantitative Criminology. Robert Sampson is one of my social science heroes--someone whose research I tremendously admire, and whose  methods and approaches give me inspiration (John Gerring is another). I remembered reading a passage criticizing the RCT craze in Sampson's (outstanding) 2012 book, Great American City (which is where I got the Cartwright citation). But I am in Toluca, Mexico, right now without access to my books, so I searched for "randomized controlled trials" AND "Robert J. Sampson" on Google-Scholar, and came up with his 2010 paper. My name is Mike Smith and I am a Google-Scholar addict.

Science type 1 vs. Science type 2

In a previous post, Rejected by Science!, I identified two different concepts of science in archaeology. Archaeological science type 1 is the pursuit of knowledge in a way that conforms to a scientific epistemology. In the words of John Gerring:

·         “Inquiry of a scientific mature, I stipulate, aims to be cumulative, evidence-based (empirical), falsifiable, generalizing, nonsubjective, replicable, rigorous, skeptical, systematic, transparent, and grounded in rational argument. There are differences of opinion over whether, or to what extent, science lives up to these high ideals. Even so, these are the ideals to which natural and social scientists generally aspire, and they help to define the enterprise in a general way and to demarcate it from other realms.” (Gerring 2012:11).

Archaeological science type 2, on the other hand, is the use of non-archaeological scientific techniques by archaeologists, for whatever purpose. Ideally, science type 2 is done in pursuit of the goals of science type 1, but such is not always the case. In my previous post, I identified two situations when archaeological science type 2 is done in ways that do not conform to type 1:

• Relativist, post-modern archaeologists who criticize a scientific epistemology for archaeology often use archaeometric methods (science type 2), in pursuit of goals that are not scientific.
• Methodologically sloppy archaeologists sometimes aim to use science type 2 methods to further science type 1 ends, but their sloppiness prevents progress.

There is a third condition where archaeological science type 2 can be done at odds  with type 1 science that I did not discuss:

• Non-archaeological scientific techniques are often used to make exaggerated, sensationalist claims that go beyond the "replicable, rigorous, skeptical" nature of scientific research.

For a very nice example of this phenomenon, see Rosemary Joyce's blog posting, "Good science, big hype, bad archaeology." NSF-supported LIDAR mapping in Honduras (a science type 2 method, for sure) has been interpreted, without ground-truthing, as possibly identifying a legendary "lost city" of Ciudad Blanca (see the Univ. of Houston press release here).  Never mind that scholars have identified this "city" as a figment of legend, not fact. Now in these circumstances it is often difficult to know just who thinks that Ciudad Blanca may have been found. Did the LIDAR scientists tell their press agent this story? Or did they merely mention Ciudad Blanca as an aside, but it was trumpeted by the press release (a frequent practice)? Government officials in Honduras are evidently jumping on the bandwagon too.

To my mind, this episode illustrates the problems that can occur when the two types of archaeological science are in conflict with one another. But right now it is merely a controversy in the realm of press releases and blogs and the internet. The rubber will hit the road when the research is submitted to a scholarly journal. And at that point one can only hope, as I suggested in my earlier post, that the editors will not be fooled into thinking that archaeological science type 2 that is done is opposition to science type 1 is really a scientific endeavor epistemologically.

Gerring, John
2012    Social Science Methodology: A Unified Framework. 2nd ed. Cambridge University Press, New York.

By the way, that earlier post "Rejected by Science!" is BY FAR the most popular post in the history of this blog, with perhaps more hits than all of the other posts combined. I am puzzled by this, not sure why it is so popular. I am not complaining, just curious. If you have any ideas, let me know.