Read This if You're Using a DiD Design (and Who Isn't?): "Difference-in-Differences Designs: A Practitioner’s Guide"

My favorite thing: Practitioner's guides to state-of-the art econometrics! 

Here is one fresh of the presses for difference in differences. 

Pedro Sant’Anna’s Difference-in-Differences Checklist

 

Pedro provided the checklist. Scott Cunningham shows us how to go through the checklist, step by step, with Stata code! These guys sure do make it easier for us to do the right thing. Here Scott shows us Step 1. And here is Step 2. Oh, and there's Step 3, too! 

And just this morning, Step 4. 

Triple Differences Models

We all remember the first time we were introduced to differences in differences models. Was it the Card and Krueger minimum wage paper? Was it Card's Mariel boatlift paper?  You surely saw the nice tables with before and after in the treatment and control groups. Pretty intuitive, right? A bit more complicated to link the tables to regression estimates but doable. Since then, you've added to your difs in difs repertoire and maybe you've even done a few triple differences analyses. Everything is OK until...you go to write up your results and realize that the estimate on that triple interaction is not so easy to explain! Don't worry, we've all been there. 

My suggestion: Have another look at my favorite explanation of differences in differences (in differences) models. This video starring my colleague, Nishith Prakash, and his coauthor, Karthik Muralidharan. Then make similar diagrams for your own paper. Tell your story in the same way that they tell their story about bicycles. Maybe make a video? I think this will help you to tell your story in your paper. 

I also recommend reading the section on triple differences in Scott Cunningham's online book, Causal Inference: The Mixtape. I really like the entire chapter on differences in differences (starts on page 263), but I especially like the discussion of differences in differences in differences (starting on page 273). He provides lots of examples of papers that use triple differences techniques. You can refer to them to help you write up your results. He also provides sample Stata code for a DDD model! 

Thank you, Shiyi, for inspiring this post! I hope it's helpful.

What Does a Difs in Difs Estimate Actually Tell Us?

Most differences in differences papers these days have a treatment that turns on and/or off at different times. A specific group can be in the treated category or the control category depending on the year. What does this imply for our interpretation of a differences in differences estimate? Does this matter? Maybe not if a particular policy has one causal impact all the time, no matter what, but if the impact of a policy changes over time (and most things in life do change), then this matters. For a detailed and formal discussion of the issues, see Andrew Goodman-Bacon's NBER working paper, "Difference-in-Differences with Variation in Treatment Timing." For an easy (and fun!) to read explanation, see his twitter thread. 

For all authors of econometrics/econometrics-y papers, I'd love to read more twitter threads like Andrew's describing the main ideas of your work. And if you can insert a gif in between, that just makes everything more enjoyable. 

Are You Writing a Difs in Difs Paper Exploiting Policy Variation Across U.S. States?

It doesn't matter which policy you're evaluating or what outcome you're considering, referees and seminar participants will always be concerned that states adopting a new policy will by coincidence (or not so much by coincidence) adopt other policies at the same time that may be driving your results. Another possibility is that states adopt new polices in response to changing characteristics of the population---and these changes are driving the variation of your outcomes. It is impossible for researchers to control for all changes in policies and all demographic characteristics, but we can certainly assuage concerns by adding to our models controls for changes in other policies or demographic characteristics. And good news: thanks to the folks at IPPSR, you can find data on many of these things in one easy to use data set! Below is the suggested citation and you can click on the link for more information: 

Suggested Citation:

Jordan, Marty P. and Matt Grossmann. 2017. The Correlates of State Policy Project v.2.0. East Lansing, MI: Institute for Public Policy and Social Research (IPPSR). 

PS

Advanced undergrads and beginner graduate students looking for data and a paper topic for a term paper, I bet that perusing through the data may inspire many great ideas!