Within-household selection of target-respondents impairs demographic representativeness of probabilistic samples: evidence from seven rounds of the European Social Survey

Abstract

This paper examines the demographic representativeness of different types of probabilistic samples based on the results of seven rounds of the European Social Survey. Focusing on the distinction between personal-register and non-personal-register samples, it demonstrates that the latter exhibit systematically larger gender- and age-biases. Expanding upon a 'gold standard' evaluation based on external criteria derived from Eurostat population statistics, an internal criteria analysis leads to the conclusion that the inferior quality of surveys involving interviewer-driven within-household selection of target respondents results from interviewer discretion. Such interference results in the selection of individuals with higher levels of readiness and availability, which superficially improves survey outcome rates while yielding samples of actually inferior quality. The internal-criteria approach provides a straightforward and undemanding way of monitoring representativeness of samples, and proves especially handy when it comes to large cross-country projects, as it requires no data external to the survey results, and allows for comparing surveys regardless of possible differences in sampling frames, sampling design and fieldwork execution procedures.