Validity in Survey Research - From Research Design to Measurement Instruments

Abstract

The ability to draw valid conclusions from data is crucial for any empirical research. Thus, validity is one of the leading quality criteria in the social and behavioral sciences. However, the term validity is used very differently across disciplines and time, creating terminological confusion that can render the concept elusive. This survey guideline provides a compact overview of different meanings associated with the term validity in the social and behavioral sciences. To acknowledge the term's full breadth, we first distinguish between (a) validity pertaining to the research design and (b) validity pertaining to measurement instruments. We show that validity is fundamentally about whether the research design and measurement instruments used for a study are true to what they are theoretically supposed to represent or capture. Subsequently, we focus on providing practical guidance on assessing measurement validity, that is, a measurement instrument's ability to measure what it purports to measure. In particular, we discuss the types of evidence supporting measurement validity and the methods researchers can use to provide such evidence for survey research. Our aim is to equip researchers with a conceptual understanding of measurement validity and a toolkit for assessing the validity of measurement instruments. We emphasize that validity is not a fixed property of a measurement instrument. Instead, researchers should view validity as a dynamic process of validation. This ongoing practice involves supporting and justifying conclusions drawn from survey data through a combination of theoretical reasoning and empirical evidence.