Tuesday, September 29, 2015

research advice

This advice by David Weil on doing research, directed at economic PhDs, is very good. I ignored/ignore number 7 and 8 too much because I'm too introverted to make appointments and too stubborn to take advice, but I admit I should follow them.

But number 4... I'm somewhere in between disagreeing with it and wanting to augment/rephrase it substantially. I definitely understand where he's coming from. The research question necessarily evolves with the project, and some of it doesn't get developed at all until the very end when you figure out how to "frame" your paper (i.e. how to sell it, to what journal/audience/subfield, etc.) Sometimes your data doesn't provide a clean answer to the question you thought it would, but you can reformulate the question. Sometimes your experiment completely fails to demonstrate what you expected, but something entirely unexpected happens that you can report on. Sometimes you start building a theory with the intention of understanding one scenario, but you prove something you didn't anticipate at all, or you're forced to change your assumptions to make things tractable and you end up understanding something else. (In fact, this should happen with some regularity, because your model isn't adding much value if you can foresee all of the consequences of your assumptions from the get-go!)

So yes, I agree that formulating a question and then setting about answering it isn't an approach you can count on. And since you can't count on it, you shouldn't spend an enormous amount of time an effort formulating your question before getting started. But, it's still valuable to think of research as starting with a question and striving for that ideal to whatever extent is practical. For a few reasons:
  1. Most importantly, for students especially, an easy route to take in research is to tweak existing research or to "try something and see what happens." That's great for learning, but see number 3: Learn as you go, don't worry about mastering techniques and knowledge ahead of time. Try things that you have a reason to think are valuable from a scientific perspective, and learn from that. And a reason to think something is scientifically valuable is to have a question in mind and design your project to answer it. It'll probably change as you go, and it's certainly helpful to think about those contingency plans ahead of time, but that's going even a step further than starting with a question, not a step backwards.
  2. Same principle as in number 1, but from a perspective later in time. The most important question you ultimately have to answer, to audiences or editors, is "why should I care about your results?". "It answers this question" is a good response. This sounds really trivial but it's not: the question shouldn't be something borderline tautological like "the data analysis answers the question of what the data says."
  3. Starting with a question ensures that your approach is appropriate for the question. There are lots of ways to answer questions and some are clearly better than others. If you take a suboptimal path, and then discover that you're answering a question that should have been answered in a better way, now you have to go back and do it right.
  4. Having a question in mind is very motivating on a big picture level. I tell people I'm interested in how social norms form and change, although the actual research I do is so remote from answering that question that it's comical, and that's such a huge question I can't even think of a single paper-sized project that can be claimed to primarily address it. But having it in the back of my mind is highly motivating and lends order to the mess of topics I actually spend time thinking about.
[Link stolen from Chris Blattman, iirc.]

No comments: