Should instrumental variables (IV) be used for real-world evaluation of the comparative effectiveness of different studies? It depends on who you ask.Garabedian et al. (2014) state
Although no observational method can completely eliminate confounding, we recommend against treating instrumental variable analysis as a solution to the inherent biases in observational CER studies.
On the other hand, Glymour, Tchetgen, and Robins (2012) state:
Given that it will often benearly free to conduct IVanalyses with secondary data, they may prove extremely valuable in many research areas . . . [however if IV] is uncritically adopted into the epidemiologic toolbox, without aggressive evaluations of the validity of the design in each case, it may generate a host of false or misleading findings.”
IV approaches are problematic if you have a weak instrument (i.e., it is weakly correlated with sorting into treatment) or if the exclusion restriction (i.e., that the IV is uncorrelated with outcomes except through the probability of receiving treatment) does not hold. One can readily test the strength of the relationship between the instrument and the probability of treatment assignment (see Stock and Yogo). Testing the exclusion restriction, however, is harder to test.
A paper by Pizer et al. (2015) recommends using falsification test to see whether it is likely that the exclusion restriction holds. He writes:
Although falsification tests in general can take many forms, there are two particularly useful strategies for testing the exclusion restriction in IV CER studies: (1) investigating an alternative outcome that ought not to be affected by the treatment under study but would be affected by potential confounders that might be correlated with the proposed IV; and (2) investigating an alternative population that again ought not to be affected by the treatment but would be affected by potential confounders.
One example he gives is the US of IV to measure stroke outcomes among patients using atrial fibrillation.
Garabedian et al. (2014) argued that practice pattern IV studies are often vulnerable to bias because they fail to control for one or more of the following patient characteristics: race, education, income, age, insurance status, health status, and health behaviors. For example, if health behaviors are correlated with anticoagulant prescribing patterns and the outcomes under study, this could indeed be a problem. However, patients without atrial fibrillation but who have carotid artery disease are also at elevated risk for stroke and should not be treated with anticoagulants. If anticoagulant prescribing patterns are unrelated to stroke outcomes for carotid disease patients, then it is less likely that confounding health behaviors are correlated with anticoagulant prescribing patterns (panel B of Figure 2). Instead of using an alternative population (those with carotid disease), another option would be to choose an alternative outcome that should not be affected by the treatment but would be affected by health behaviors (e.g., incident lung cancer).
Not that although it is never possible to prove for certain that there is no confounding, falsification tests provide evidence that the IV exclusion restriction is valid.
Source:
- Pizer, S. D. (2015), Falsification Testing of Instrumental Variables Methods for Comparative Effectiveness Research. Health Services Research. doi: 10.1111/1475-6773.12355