Are We Any Better Than the Social Psychologists?

The NYTimes ran an article last week explaining the ordeal with power poses and Amy Cuddy. The article describes the big issues with all empirical work: the scarcity of replications studies and p-hacking. I read the article just before bed one evening after spending the afternoon tinkering with data. I wouldn't be lying if I told you that it kept me up that night. 

I really do believe that we are learning about the world by doing research (however imperfect the process is), but can we as researchers do better? Would we all be better off if the incentives were changed to reward more replication (even if it comes at the expense of new studies)? And what about papers with non-results? Would the world be better off if these papers were ever published in good journals? 

I suppose it's difficult for me to single-handedly change the reward system within economics, but I do have control over the types of projects I start (and continue). I also have a tiny bit of control over what gets published were via refereeing. I have blogged in the past about how the referee process has changed over the years (papers have gotten longer, so many appendix tables, etc). On that sleepless night last week, I decided that maybe all the robustness checks aren't such a bad idea after all. In a way, they are mini-replication studies. They also make p-hacking a lot more difficult! I think we have come a long way in the field, and we are getting even better. I am so happy that more and more journals are requiring data and code to be available to readers. But I still think we can do better. 

Anyway, see Chris Blattman's thoughts on this. 

And Dan Hammermesh's recent paper on replication studies within economics.