Regression Discontinuity is an econometric method that has become popular in recent years. Let me give you an example where regression discontinuity would be valid.
Let us say that all students who score 1000 or more on their SATs matriculate at Ivy U and all students who score below 1000 attend college at State U. The research question is what impact going to Ivy U has on wages.
If we simply compare the average salaries of those at Ivy U and those at State U, this will likely not reveal the true effect that Ivy U had on its graduates. Those at Ivy U were likely smarter and more motivated than those at State U. Thus, the impact of Ivy U’s education is confounded with the individual’s own talent and motivation.
Regression discontinuity, however, can solve this problem. If we compare individuals who scored just above and just below 1000, these individuals are likely very similar in terms of intelligence and motivation. The only difference would the impact of Ivy U’s education and networking possibilities against State U’s. We could compare average scores just above and below the 1000 mark. However, we could also fit a polynomial function of test scores on wages with a discrete jump term at 1000. Mathematically, this means the following:
- Effect = limx↓c E[Yi|Xi=x] – limx↑c E[Yi|Xi=x]
- In this example, Yi is the wages, Xi is the test scores, and the the cutoff value, c, is 1000.
Can we use Regression Discontinuity to estimate the impact of school districts on schooling? We could compare houses on each side of the school district boundaries and then see if these similar houses have different test scores. However, this will likely not produce reliable results if parents choose their house based on the school district. Thus, even if two identical houses are right next to each other, if high achieving parents always choose the better school district, then there will be perfect sorting between school districts.
David S. Lee and Thomas Lemieux (2009) have a great “user guide” about how to use Regression Discontinuity in practice. Some of their top tips are the following:
RD designs can be invalid if individuals can precisely manipulate the “forcing variable”.
- In the school district choose example, where parents can precisely choose their school district RD may be invalid.
If individuals – even while having some influence – are unable to precisely manipulate the forcing variable, a consequence of this is that the variation in treatment near the threshold is randomized as though from a randomized experiment.
- Intuitively, when individuals have imprecise control over the forcing variable, even if some are especially likely to have values of X near the cutoff, every individual will have approximately the same probability of having an X that is just above (receiving the treatment) or just below (being denied the treatment) the cutoff – similar to a coin-flip experiment. This is the case of people who score around 1000 on the SAT and thus have an approximately equal probability of getting into Ivy U or State U.
RD designs can be analyzed – and tested – like randomized experiments.
- If variation in the treatment near the threshold is approximately randomized, then it follows that all “baseline characteristics” – all those variables determined prior to the realization of the forcing variable – should have the same distribution just above and just below the cutoff.
Non-parametric estimation does not represent a “solution” to functional form issues raised by RD designs. It is therefore helpful to view it as a complement to – rather than a substitute for – parametric estimation.
- Parametric functions are what are traditionally used. These are generally polynomial that regress the dependent variable of interest onto the X variable. In my example, this would be a polynomial regression with future wages as the dependent variable and test scores as the independent X variable. Non-parametric estimation techniques include local linear regression.
Goodness-of-fit and other statistical tests can help rule out overly restrictive specifications.
- Although there is no simple formula that works in all situations and contexts for weeding out inappropriate specifications, it seems reasonable, at a minimum, not to rely on an estimate resulting from a specification that can be rejected by the data when tested against a strictly more flexible specification. For example, it seems wise to place less confidence in results from a low-order polynomial model, when it is rejected in favor of a less restrictive model (e.g., separate means for each discrete value of X). Similarly, there seems little reason to prefer a specification that uses all the data, if using the same specification but restricting to observations closer to the threshold gives a substantially (and statistically) different answer.
Citation
- David S. Lee and Thomas Lemieux (2009) “Regression Discontinuity Designs in Economics” NBER Working Paper 14723.