Should archaeologists worry about attacks from politicians?

Last May, the U.S. House of Representative passed an amendment that would allow Congress to interfere with the peer review process at the National Science Foundation. Specifically, the bill would prohibit the NSF from funding research in political science. Does this have any implications for archaeology?

The amendment was sponsored by Rep. Jeff Flake (R) from my state of Arizona. Flake wants to politicize funding at NSF. He objects to wasteful spending on merit-less research by political scientists, such as $700,000 to develop a new model for international climate change analysis and $600,000 to try to figure out if policymakers actually do what citizens want them to do.

There is quite a bit online about this, including Ezra Klein's blog at the Washington Post and an excellent editorial in Nature, which includes the statement, " To conclude that hard problems are better solved by not studying them is ludicrous. Should we slash the physics budget if the problems of dark-matter and dark-energy are not solved?" I was just reminded of this episode by an eloquent letter to the editor in the September 21 issue of Science by Arthur Lupia. Fortunately, Flake's amendment has little chance of passing the Senate.This is just the latest is a long line of conservative attacks on science, and on the social sciences in particular.

So, what about archaeology? If political science is not seen as a rigorous science, where does that leave us? Can we expect the likes of Jeff Flake (who is running for Senate from Arizona right now, an outcome scarier than Halloween) to come after archaeology? Some archaeologists would like to see our field as providing data and findings relevant to contemporary concerns. Should we tone down any such implications to keep the politicians at bay? If political science research on climate change is ideologically suspect, what about archaeological research on past climates, or on sustainability?

Maybe we should all pursue postmodern themes. After all, Frederic Jameson tells us that postmodernism is "the cultural logic of late capitalism." (Verso, 1991)
Conservative politicians ought to like that! If we write in postmodern prose, people like Jeff Flake won't be able to understand what we are saying, and it will be hard for them to find it objectionable. No, on second thought, if we go postmodern I won't be able to understand it either. (By the way, I see that the Postmodern Text Generator is still going strong. Just click the refresh button a few times for new essays).

My suggestion is that we should do our work as rigorously and as scientifically as we can. I am more worried about politicians claiming that archaeology is not really a science (and thus not worthy of NSF support) than about claims that our work is too political, as in Flake's case. Of course by this I mean Science-1 (scientific epistemology), not Science-2 (the occasional use of scientific techniques, sometimes in service of an anti-science agenda like postmodern archaeology). See my prior posts on this: Rejected by Science!, or Science type 1 vs. Science type 2, or John Gerring: Methodological Unity or Diversity?. 

"Rigorous evaluation of human behavior"

I've been reading the June 15 issue of Science, and I am struck by the irony of two articles. The first is a news item titled "Social scientists hope for reprieve from the senate." The U.S. House of Representatives recently voted to prohibit the NSF from funding political science research, and to reduce the scale of the American Community Survey (a census-based social survey). The bill was co-sponsored by an  unenlightened congressman from my own (unenlightened) state of Arizona, Jeff Flake (jokes about "what's in a name" come to mind here). "Flake says political science isn't sufficiently rigorous to warrant federal support." Does Flake base his policies and views on rigorous research?  I doubt it. Conservative politicians periodically go after the social sciences in Washington, and we should all hope that the current attack is as unsuccessful as previous ones have been. Is archaeology more or less rigorous than political science? If we use science definition 1, I think we are in bad shape. But we can always take refuge in science definition 2 ("see, we use complicated scientific technology!") to assert our rigor.

The second article in the June 15 Science was a short essay in a section called "Science for Sustainable Development." This essay, titled "Rigorous Evaluation of Human Behavior" is written by Esther Duflo, an economist at MIT. She makes the valid and important point that the role of science in promoting sustainable development and alleviating poverty should include social scientific studies of behavior. I wonder what Jeff Flake would think about this. When I saw the title I was encouraged, but then I got to the heart of Duflo's essay: the way to conduct "rigorous" studies of human behavior is to use randomized controlled trials. "This makes for good science: these experiments make it possible to test scientific hypotheses with a degree of rigor that was not available before."

In some fields of social science and public health, the randomized controlled trial (RTC) has become the supposed "gold standard" of research methods, proclaimed to be far superior to other approaches. Apart from the fact that we simply cannot do RCT's in archaeology (except perhaps in a few very limited situations that I can't think of offhand), I must admit that I am more supportive of the growing critique and contextualization of RCTs in social science. RCT is a narrow approach that achieves internal rigor at the expense of external relevance and validity. Philosopher of science Nancy Cartwright puts it this way, using economics to illustrate the trade-off of internal rigor and external validity:

“Economists make a huge investment to achieve rigor inside their models, that is to achieve internal validity. But how do they decide what lessons to draw about target situations outside from conclusions rigorously derived inside the model? That is, how do they establish external validity? We find: thought, discussion, debate; relatively secure knowledge; past practice; good bets. But not rules, check lists, detailed practicable procedures; nothing with the rigor demanded inside the models.” (Cartwright 2007:18).


Or consider a recent paper by sociologist Robert J. Sampson (2010), who promotes the value of observational research in criminology and sociology. He deflates three myths of RCTs in criminology:


Myth 1: Randomization solves the causal inference problem.
Myth 2: Experiments are assumption (theory) free.
Myth 3: Experiments are more policy relevant than observational studies.


If you want more context on internal vs. external validity, or how various social science methods relate to an experimental ideal, see Gerring (2007). He is one of those political scientists who, according to Congressman Flake, must be non-rigorous. But the message here is analogous to my views on science types 1 and 2 in archaeology. Just as archaeologists can do scientifically rigorous and valid research without involving technological methods from the hard sciences, so too can other social scientists do scientifically rigorous and valid research without the aid of formal experiments (RCTs).

Cartwright, Nancy
2007    Are RCT's the Gold Standard? BioSocieties 2(1):11-20.

Gerring, John
2007    Case Study Research: Principles and Practices. Cambridge University Press, New York.

Sampson, Robert J.
2010    Gold Standard Myths: Observations on the Experimental Turn in Quantitative Criminology. Journal of Quantitative Criminology 26(4):489-500.

Postscript--No, I don't read or keep up with the Journal of Quantitative Criminology. Robert Sampson is one of my social science heroes--someone whose research I tremendously admire, and whose  methods and approaches give me inspiration (John Gerring is another). I remembered reading a passage criticizing the RCT craze in Sampson's (outstanding) 2012 book, Great American City (which is where I got the Cartwright citation). But I am in Toluca, Mexico, right now without access to my books, so I searched for "randomized controlled trials" AND "Robert J. Sampson" on Google-Scholar, and came up with his 2010 paper. My name is Mike Smith and I am a Google-Scholar addict.