Question Order Effects in Cross-Cultural Web Probing: Pretesting Behavior and Attitude Questions

Abstract

Cognitive pretesting is an essential method of piloting questionnaires and ensuring quality of survey data. Web probing has emerged as an innovative method of cognitive pretesting, especially for cross-cultural and web surveys. The order of presenting questions in cognitive pretesting can differ from the order of presentation in the later survey. Yet empirical evidence is missing whether the order of presenting survey questions influences the answers to open-ended probing questions. The present study examines the effect of question order on web probing in the United States and Germany. Results indicate that probe responses are not strongly impacted by question order. However, both content and consistency of probe responses may differ cross-culturally. Implications for cognitive pretesting are discussed.