A Closer Look at Attitude Scales with Positive and Negative Items: Response Latency Perspectives on Measurement Quality

Abstract

Measurement quality in standardized surveys has been a core issue for decades in survey research. For questionnaire designers, it is common to use a mix of positive and negative worded items to measure multi-item-constructs in order to control for response effects like acquiescence bias. The paper shows that paradata such as response latency measurement can be used to identify specific subgroups of respondents with specific types of cognitive response modes. These response modes moderate the occurrence of response effects, systematic and random measurement errors, and thus the reliability and validity of attitudinal measurement models. Therefore, adapting paradata to detect low measurement quality can be used as a tool leading to a better understanding of respondents' cognitive processes. Data of a German CATI-survey with experimental design and measurement of response latencies are used to analyze data quality of a measurement model of attitudes towards health nutrition with mixed items. Response effects are analyzed through the experimental variation of question order of negative and positive worded items. Structural equation models are estimated in a multiple-group moderator design to test validity and reliability of the latent attitude construct. As a result, the attitude scale shows acceptable values of validity and reliability only under the condition of spontaneous answers where question order effects appear.