Wednesday, April 25, 2012

MTurk (non-)advocacy

I'm apparently becoming (at least labeled as) one of those people who is obsessed with the possibilities of some new specific technology. In particular, MTurk, and using MTurk for social science research.

I'm uncomfortable with that label, and I think I figured out why: By choosing to do experiments on MTurk, I chose a niche to develop a comparative advantage in. That's different from choosing a cause to advocate for. I don't have a problem with part of my identity being my set of crusades, but MTurk just isn't one of them.

I don't think my niche is an absolute best niche. It has certain relative advantages and disadvantages and certainly within some contexts and compared to specific other things it has more absolute advantages and disadvantages, but it's not the end-all be-all of experimental methodology. Lab experiments are great too; field experiments are wonderful. I don't want to force every experiment to fit with the MTurk platform. MTurk is often my first thought when thinking about how to test things, but that's because I'm more and more familiar with it and have more and more of a comparative advantage in using it, so of course I explore that possibility first.

Even beyond that, I don't particularly want to convince the rest of the profession to follow me to MTurk. First of all, obviously, then my advantage will be smaller. More importantly, then I won't have all those other wonderful and diversely robust results from other domains to refer to and compare to. You can't build a house with only a hammer and the fact that I've specialized in hammers makes me hope everyone else has everything else under control.

This is very different from picking a cause to crusade for. But it's understandable that I'd be seen as doing so. I get the impression that John List and Stefano DellaVigna are on more of a field experiment crusade than niche-selection. A little of both, for sure; I don't know the balance. There are certain MTurk researchers who are definitely crusaders; yesterday I heard someone claim that if you were going to get anything at all out of a series of seminars on behavior change research, it's that MTurk is the best thing in the world. I appreciate enthusiasm, and I appreciate MTurk, but that's a bit too much of an overstated non sequitur to be taken seriously :)

With that all serving as a caveat, check out this hilariously awesome real-technology application of MTurk.

No comments: