From Davern (2013):
…the three surveys studied showed that for the same total survey budget approximately twice the effective sample size (e.g., from 10,000 to 20,000 effective sample size completed responses) could be obtained if a less aggressive protocol were followed and the research team were willing to accept an approximately 40 percent lower response rate (e.g., from 45 percent to 26 percent). The paper also showed that the approximately 40 percent reduction in response rate would not have produced one statistically different result across the 33 estimates examined (Davern et al. 2010).
The author makes three suggestions that the survey industry could do to better address non-response bias.
- Surveys should produce and make available high quality paradata (data collected about the survey operations) to accompany the data collected on the survey questions themselves, so researchers can access and call histories, contact histories, know what mode the survey was completed in and whether an incentive was provided, and know whether a proxy interview was accepted as they conduct their analyses.
- Organizations conducting surveys should also be judged by their commitment to transparency (such as participation in AAPOR’s transparency initiative)
- Every survey sponsored by the federal statistical system, or if the results are being published in a scientific journal, should reference a publicly available quality profile that includes a nonresponse bias analysis.
Source:
- Davern, M. (2013), Nonresponse Rates are a Problematic Indicator of Nonresponse Bias in Survey Research. Health Services Research, 48: 905–912. doi: 10.1111/1475-6773.12070