It's funny how it happens. Last week I was talking with a coauthor about heterogeneous impacts and how this makes interpretation of results difficult even in perfectly identified models. And then this week, there's a brand new NBER paper discussing this very issue! Thank you, NBER (and Douglas L. Miller, Na’ama Shenhav, Michel Z. Grosz--the paper's authors) for delivering this careful analysis exactly when I was thinking about this stuff.
Na'ama explains everything so nicely in this twitter thread. The basic idea: Imagine you have a family fixed effects model (same family, but some children exposed to a policy and others not exposed because of when they happened to be born). It's a nice natural experiment and would give you the right average treatment effect if everyone was affected by the policy in the same way and/or every family had some children exposed and others not exposed to the policy. The problem: the policy may not affect all families the same way, and we're only identifying the effect on the families who are treated. In the example in the paper, treated families are more likely to be big families and big families tend to be more strongly impacted by the policy (Head Start, in the example).
So if this were just a paper describing some problem with the way things are typically done, it would be..well, depressing. But the great news: the authors suggest a reweighting technique to get us the average treatment effects! Hoooray!
Thank you, Na'ama, for the twitter thread. So helpful.
No comments:
Post a Comment