Friday, September 27, 2019

Another PSA: Always Check Your Code

..better yet, have someone else check your code? Better yet, build checks into your code to help you catch mistakes. 

So someone made a mistake. Instead of controlling for country of origin fixed effects, the person "controlled for" country of origin by including the country code as a continuous variable. Ouch! I cringed when read this because I have actually seen this mistake being made. It's an easy mistake to make: instead of typing in "i.country" into Stata, you just type "country". I feel for the paper's authors. 

But one thing to catch a typo in the early stages of research (especially for graduate students who are just learning to code), but quite another for this to be caught after a study has been given significant media attention. The paper was about whether religiosity promotes generosity. See the description here

And now my plea to journals: Please require code to be made available for all published papers. This is not only a way for mistakes to be caught quickly, but it provides stronger incentives for paper authors to write better, nicely organized code. 

But now a question: What about working papers? Papers often get significant media attention even before they're published in a journal. Requiring code for publication doesn't help if all of the coverage happens before publication. You may that journalists shouldn't write about working papers, but I'm not sure journalists should necessarily wait until publication given how long it takes for a paper to get through the referee process. 

So maybe another plea to the journals: speed up the referee process. I'd be happy to be given less time to write my referee reports in exchange for prompter reports on my own submissions.

1 comment:

  1. Another case: https://jamanetwork.com/journals/jama/fullarticle/2752474

    ReplyDelete