Bookmark and Share

Mode system effects in an online panel study: comparing a probability-based online panel with two face-to-face reference surveys

[journal article]

Struminskaya, Bella; Leeuw, Edith de; Kaczmirek, Lars

fulltextDownloadDownload full text

(external source)

Citation Suggestion

Please use the following Persistent Identifier (PID) to cite this document:http://dx.doi.org/10.12758/mda.2015.001

Further Details
Abstract "One of the methods for evaluating online panels in terms of data quality is comparing the estimates that the panels provide with benchmark sources. For probability-based online panels, high-quality surveys or government statistics can be used as references. If differences among the benchmark and the online panel estimates are found, these can have several causes. First, the question wordings can differ between the sources, which can lead to differences in measurement. Second, the reference and the online panel may not be comparable in terms of sample composition. Finally, since the reference estimates are usually collected face-to-face or by telephone, mode effects might be expected. In this article, we investigate mode system effects, an alternative to mode effects that does not focus solely on measurement differences between the modes, but also incorporates survey design features into the comparison. The data from a probability-based offline-recruited online panel is compared to the data from two face-to-face surveys with almost identical recruitment protocols. In the analysis, the distinction is made between factual and attitudinal questions. We report both effect sizes of the differences and significances. The results show that the online panel differs from face-to-face surveys in both attitudinal and factual measures. However, the reference surveys only differ in attitudinal measures and show no significant differences for factual questions. We attribute this to the instability of attitudes and thus show the importance of triangulation and using two surveys of the same mode for comparison." (author's abstract)
Keywords panel; online survey; survey research; response behavior; data capture; data quality
Classification Methods and Techniques of Data Collection and Data Analysis, Statistical Methods, Computer Methods
Method basic research; development of methods
Free Keywords mode system effect; benchmark survey comparison
Document language English
Publication Year 2015
Page/Pages p. 3-56
Journal Methods, data, analyses : a journal for quantitative methods and survey methodology (mda), 9 (2015) 1
ISSN 1864-6956
Status Published Version; peer reviewed
Licence Creative Commons - Attribution