The Place of Expert Review in Translation and Questionnaire Evaluation for Hard-to-Count Populations in National Surveys

Abstract

Many researchers consider it a best practice to include respondent pretesting as part of the survey translation process (Survey Research Center, 2016). The U.S. Census Bureau has a Pretesting Standard that delineates requirements to ensure that any data collection instrument "works," by verifying that it can be administered as intended by interviewers and understood and responded to appropriately by respondents (U.S. Census Bureau, 2015). Expert review is included in the standard but it is described as a sort of evaluation method of last resort due to its lack of inclusion of respondent pretesting in the process. In large survey organizations, there can be a variety of types of materials in need of translation ranging from actual survey questionnaires to other support materials. Due to resource limitations, expert review is a method that often comes into play in the absence of resources for respondent testing. Recent discussions have involved defining and modernizing the Census Bureau’s approach to translation methodological expert reviews and looking at how they can best fit into the overall translation process. This paper provides a review of the literature on expert review, a description of how this type of review is currently done at the U.S. Census Bureau, along with limitations, challenges, and a plan for future research in order to further develop the method.