What Not to do in Graduate School

See this article in Nature for advice on what not to do in graduate school. It actually kind of surprises me how appropriate advice meant for hard scientists (the ones that do their work in a lab) is for social scientists doing data work with Stata!

Some of my favorite big lessons (in no particular order): 

1. More work is NOT always better

2. Don't always trust your data----if it's saying something that seems too good to be true, it may very well be too good to be true.

This Happens.

You have come up with an important question, you have the perfect data to answer the question, you have come up with a solid identification strategy, you have no mistakes in your code, but then...

What to do when this happens? See David Evans old blog entry here. Good journals will publish this type of work, but it needs to be done well. Also, they should really change people's priors. If no one believes that X will affect Y and find that indeed X has no impact on Y, well, that's probably not going to a top journal. But if everyone takes for granted that X affects Y and you find no evidence of this, then that's something. My favorite recent example of this is this paper showing that nudging college students to study, doesn't really help. Ok, but what if you find yourself in the first category. No one really thought X would affect Y and "surprise", you find no evidence of it. Well, there's a journal for those results, too! The Journal of Unsurprising Results in Economics (SURE). Read more about it here. I'm really excited about this journal, not only because it may be a home for future papers, but because it's good for science! Those insignificant, unsurprising results also help us learn about the world. They shouldn't all be hidden in people's hard drives and filing cabinets. 

P.S.

But before concluding that there is no effect, be sure to check your code for coding errors. And now check again. 

Beamer Tips

Job market candidates! I know it's mid-July and you're perfecting your job market papers. But if you can't stand to look at Stata for another second and writer's block has made it impossible to revise that introduction (again), maybe you can start working on your slides. 

Have a look at Paul Goldsmith-Pinkham's super useful tips for making excellent Beamer presentations! He even includes the code. I really like his idea of using TikZ to make figures for showing difference in differences timing (see page 45) of the slides. 

Calvin and Hobbes and Referee 2

There are certain things we only do if required to by about an editor or referee. Sure, sometimes referee 2 recommends things (without much thought) that are just too hard and not worthwhile for the paper. But sometimes, those recommendations make not only for a better paper but they make you a better researcher. Always good to have more tricks. But be careful about investing too much in tricks that are not worthwhile (unless a referee at a good journal insists)...

Here's the link to the comic.

Cool Data Alert: NCHS Data Linked to HUD Housing Assistance Program Files

Yeah, yeah...that Scandinavian register data is amazing, but these days I'm pretty excited about new links being made between large U.S. surveys and administrative data. For those of you interested in the relationship between housing (or housing-related policies) and health, I have great news! Copied from an email I just received

NCHS has linked 1999-2016 National Health Interview Survey (NHIS) and 1999-2016 National Health and Nutrition Examination Survey (NHANES) to administrative data through 2016 for the Department of Housing and Urban Development’s (HUD) largest housing assistance programs: the Housing Choice Voucher program, public housing, and privately owned, subsidized multifamily housing. Linkage of NCHS survey participants with HUD administrative records provides the opportunity to examine relationships between housing and health.

For more details, visit this website. 

I have used the NHIS before, and I can say that sample sizes are large (large enough even to study immigrants!), and there are many, many interesting variables. Although the survey focuses on health, there are questions related to income, education, country of birth, etc. 

The downside: To use many of the most interesting variables, you have to go to a restricted data center. 

The upside: You can very quickly and easily get a sense of the NHIS data from the IPUMS page before traveling to such a center. 

And Even More Advice on Writing Referee Reports

Summer is great for getting research done because we have uninterrupted time to work on projects. I feel like a good chunk of my time during the school year is spent figuring out where I left off on a project before being distracted by teaching, meetings, etc. But I just got back from a lovely trip to Belgium, don't remember where I left off on any of my projects, don't know which to tackle first...and so decided to get some referee reports written.  :) 

I happened to come across this advice column on writing good referee reports (thank you, Academic Sequitur). I really like the general advice that my main job as a referee is to decide whether the paper is publishable in the journal or not and then to make the case for or against to the editor. Although it is certainly nice to make suggestions for improving the paper, that is not the main responsibility of the referee. But then why do I spend so much time making such suggestions even when I know the paper will be rejected? Because I know it is so hard to get feedback on papers outside of the refereeing process. Yes, some people present at various prestigious conferences/seminars and get excellent feedback that way, but my sense is that many papers are written and published without ever being carefully read or thought about by anyone but the authors and referees. This is a shame. Especially for early-career researchers.  

Then again, it is important to keep in mind when writing these reports that what is most useful for the authors (and the editors) is not my pointing out the typo on page 21, but my evaluation of the paper in general and my thoughts on the "must do's" vs. "nice to do's". 

Ok, and now, back to referee reports....