Comparing Multiple Imputation and Propensity-Score Weighting in Unit-Nonresponse Adjustments: A Simulation Study

Abstract

The usual approach to unit-nonresponse bias detection and adjustment in social surveys has been post-stratification weights, or more recently, propensity-score weighting (PSW) based on auxiliary information. There exists a third approach, which is far less popular: using multiple imputed values for each missing unit of the survey outcome(s). We suggest multiple imputation (MI) as an alternative to PSW since the latter is known to increase variance substantially without reducing bias when auxiliary variables are not associated with the survey outcome of interest. Given that most social surveys have multiple target variables, creating imputed data sets may address bias in survey outcomes with less variance inflation. We examine the performance of PSW and MI on mean estimates under various conditions using fully simulated data. To evaluate the performance of the methods, we report average bias, root mean squared error, and percent coverage of 95 percent confidence intervals. MI performs better under some of our scenarios, but PSW performs better under others. Even within certain scenarios, PSW performs better on coverage or root mean squared error while MI performs better on the other criteria. Therefore, robust methods that simultaneously model both the outcomes and the (non)response may be a promising alternative in the future.