Compensating for low topic interest and long surveys: a field experiment on nonresponse in web surveys

Abstract

Certain survey characteristics proven to affect response rates, such as a survey’s length and topic, are often under limited control of the researcher. Therefore, survey researchers sometimes seek to compensate for such undesired effects on response rates by employing countermeasures such as material or nonmaterial incentives. The scarce evidence on those factors’ effects in web survey contexts is far from being conclusive. This study is aimed at filling this gap by examining the effects of four factors along with selected interactions presumed to affect response rates in web surveys. Requests to complete a web-based, selfadministered survey were sent to 2,152 owners of personal websites. The 2 X 2 X 2 X 2 fully crossed factorial design encompassed the experimental conditions of (a) high versus low topic salience, (b) short versus long survey, (c) lottery incentive versus no incentive, and (d) no feedback and general feedback (study results) versus personal feedback (individual profile of results). As expected, highly salient and shorter surveys yielded considerably higher unit-response rates. Moreover, partial support was found for interaction hypotheses derived from the leverage-salience theory of survey participation. Offering personalized feedback compensated for the negative effects of low topic salience on response rates. Also, the lottery incentive tended to evoke more responses only if the survey was short (versus long), but this interaction effect was only marginally significant. The results stress the usefulness of a multifactorial approach encompassing interaction effects to understand participation differences in web surveys. (auhtor'S abstract)