What Makes a Paper Amazing? Good? Bad?

I was at the NBER SI this past week and saw a lot of papers! I can't say that I saw any bad papers (this is the NBER after all), but I did find myself thinking about what makes a paper amazing vs. just really good. I guess the answer is that amazing papers answer important questions in clever and credible ways. Most papers don't do all three but scoring high on even one of these can get you pretty far. I would also add that presenting well makes a big difference.

Anyway, I thought this would be a good time for me to show you this blog post on why research fails. I agree with all three of her points. Pay particular attention to the part at the end explaining when research does not fail (even though it might feel that way).

Addendum: Someone isn't so happy with the state of research in the sociology of education field. I'm not so sure how this compares to the state of research within economics, but I can say that I have had similar thoughts when refereeing papers. It breaks my heart to have to recommend rejecting papers that have potentially good ideas hidden deep within them. That said, there is also the danger of spending months and months to write one footnote that nobody really cares about. Always tricky to figure out when to let go of a paper....I struggle with this. 

Applied Micro People: Stop Everything and Read this Now

This new Athey-Imbens paper has it all! Intuitive descriptions of the most fashionable identification strategies, plus examples, plus practical suggestions for best practices. The paper does excellent job of describing the strategies we know well (RD, difs-in-difs), but then also describes newer variations of these strategies (RKD, synthetic controls, nonlinear difs-in-difs) that we might not yet have used. My favorite section is the one on supplementary analyses! I have used many of these techniques in my papers, but I have never thought about doing them in any systematic way. There is also a section on machine learning.  I'm not at all familiar with these techniques, but if Athey and Imbens put this section in here, I should probably learn it!

This paper is a must-read for all applied micro students as well as anyone teaching any applied micro class at the graduate level. Actually, all applied micro people should have a look every now and then. Even if you don't use these techniques in your own research, knowing the "state of applied microeconomics" will make you better referees, better seminar/conference participants, better human beings? Ok, maybe not necessarily better human beings...well, maybe.  :)

Adorable Goats Show Us What It's Like to Do Research

This is not what I thought research would be like. I thought the process went something like this: Come up with brilliant idea. Test it with data. Write it up. Send to journal. Move on to next brilliant idea. Actually, even now, I find myself thinking that this is exactly how it works for "other" researchers. But maybe it isn't. Maybe all (most?) researchers do research exactly like these goats accomplish their goals. Enjoy!..both the goat video and the process of doing research!

Stata Tip: Multiway Fixed Effects

I just discovered a funky little ado file that may be useful when running models with multiple sets of fixed effects on large data sets: rehgdfe.ado. I haven't tried it yet, but here is a description from the help file:

reghdfe is a generalization of areg (and xtreg,fe, xtivreg,fe) for multiple levels of fixed effects (including heterogeneous slopes), alternative estimators (2sls, gmm2s, liml), and additional robust standard errors (multi-way clustering, hac standard errors, etc).

Additional features include:

A novel and robust algorithm to efficiently absorb the fixed effects (extending the work of Guimaraes and Portugal, 2010).
Coded in Mata, which in most scenarios makes it even faster than areg and xtreg for a single fixed effect (see benchmarks on the Github page).
Can save the point estimates of the fixed effects (caveat emptor: the fixed effects may not be identified, see the references).
Calculates the degrees-of-freedom lost due to the fixed effects (note: beyond two levels of fixed effects, this is still an open problem, but we provide a conservative approximation).
Iteratively removes singleton groups by default, to avoid biasing the standard errors (see ancillary document).