Careful with IVs...

You know when you just have a feeling about something. Maybe it's a gut instinct that comes from nowhere, but more often, the feeling comes from your experiences. Anyway, for years I have had that type of feeling about instrumental variables (IV) analyses. Yes, I understand the math behind how IV techniques can work wonders in terms of correcting bias. But after playing with IVs in various contexts, using various data sources, for various papers, etc., I have gotten a sense that IV results are just not as robust as I would like. 

Well, it turns out that my feeling is probably right, and Alwyn Young, an economist at LSE, has shown this formally in a recent paper about inference in IV models. Yes, you should read the paper, but since you're probably busy with writing or grading final exams these days, have a quick look at Marc Bellemare's post about the article. These are some of the most important points, as described in Marc's post: 

1. Conventional tests tend to overreject the null hypothesis that the 2SLS coefficient is equal to zero.

2. 2SLS estimates are falsely declared significant one third to one half of the time, depending on the method used for bootstrapping.

3. The 99-percent confidence intervals (CIs) of those 2SLS estimates include the OLS point estimate over 90 of the time. They include the full OLS 99-percent CI over 75 percent of the time.

4. 2SLS estimates are extremely sensitive to outliers.

So what to do? Does this mean we should abandon IV analyses and just report OLS results? Absolutely not. A good exogenous IV can correct some pretty major problems in the OLS. I guess what I would say is just to be careful. And to always report both the IV and OLS results!

In the meantime, a fun activity: come up with new slogans for OLS! Check out some recent candidates from the world of twitter!