Volltext herunterladen
(393.0 KB)
Zitationshinweis
Bitte beziehen Sie sich beim Zitieren dieses Dokumentes immer auf folgenden Persistent Identifier (PID):
https://nbn-resolving.org/urn:nbn:de:0168-ssoar-71619-9
Export für Ihre Literaturverwaltung
Coding Text Answers to Open-ended Questions: Human Coders and Statistical Learning Algorithms Make Similar Mistakes
[Zeitschriftenartikel]
Abstract Text answers to open-ended questions are often manually coded into one of several predefined categories or classes. More recently, researchers have begun to employ statistical models to automatically classify such text responses. It is unclear whether such automated coders and human coders find the ... mehr
Text answers to open-ended questions are often manually coded into one of several predefined categories or classes. More recently, researchers have begun to employ statistical models to automatically classify such text responses. It is unclear whether such automated coders and human coders find the same type of observations difficult to code or whether humans and models might be able to compensate for each other’s weaknesses. We analyze correlations between estimated error probabilities of human and automated coders and find: 1) Statistical models have higher error rates than human coders 2) Automated coders (models) and human coders tend to make similar coding mistakes. Specifically, the correlation between the estimated coding error of a statistical model and that of a human is comparable to that of two humans. 3) Two very different statistical models give highly correlated estimated coding errors. Therefore, a) the choice of statistical model does not matter, and b) having a second automated coder would be redundant.... weniger
Thesaurusschlagwörter
Codierung; statistische Methode; Fehler; Analyse; Umfrageforschung; Datenaufbereitung
Klassifikation
Erhebungstechniken und Analysetechniken der Sozialwissenschaften
Freie Schlagwörter
open-ended question; manual coding; automatic coding; text classification; text answer
Sprache Dokument
Englisch
Publikationsjahr
2021
Seitenangabe
S. 103-120
Zeitschriftentitel
Methods, data, analyses : a journal for quantitative methods and survey methodology (mda), 15 (2021) 1
Heftthema
The Use of Open-ended Questions in Surveys
DOI
https://doi.org/10.12758/mda.2020.10
ISSN
2190-4936
Status
Veröffentlichungsversion; begutachtet (peer reviewed)