Improving Anchoring Vignette Methodology in Health Surveys with Image Vignettes

Abstract

The anchoring vignette method is designed to improve comparisons across population groups and adjust for differential item functioning (DIF). Vignette questions are brief de­scriptions of hypothetical persons for respondents to rate. Although this method has been adopted widely in health surveys, there remain challenges. In particular, vignettes are com­plex, increasing survey time and respondent burden. Further, the assumptions underlying this method are often violated. To overcome such challenges, this paper introduces an inno­vative technique, namely image anchoring vignettes, conveying vignette information with varying health levels in images. We conducted a cross-cultural experimental study to ex­amine the performance of image and standard text vignettes in terms of response time, how well they satisfy the assumptions, and their DIF-adjusting quality using a confirmatory factor analysis. The study revealed that respondents can better differentiate the intensity levels of the three vignettes in the image vignette condition, compared to text vignettes. Response consistency assumption appears to be better satisfied for image vignettes than text vignettes. Using well-designed image vignettes greatly reduces survey time without losing the DIF-adjustment quality, indicating the potential of image vignettes to improve overall efficiencies of the anchoring vignette method. Improving vignette equivalence (i.e., minimizing different interpretations of vignettes by different groups), remains a challenge for both text and image vignettes. This study generates new insights into the design and use of image anchoring vignettes.