Volltext herunterladen
(1.276 MB)
Zitationshinweis
Bitte beziehen Sie sich beim Zitieren dieses Dokumentes immer auf folgenden Persistent Identifier (PID):
https://nbn-resolving.org/urn:nbn:de:0168-ssoar-81272-3
Export für Ihre Literaturverwaltung
Scaling up search engine audits: Practical insights for algorithm auditing
[Zeitschriftenartikel]
Abstract Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, vir... mehr
Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this article focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with diverse experimental designs, and point to different changes and strategies that improve the quality of the method. We conclude that virtual agents are a promising venue for monitoring the performance of algorithms across long periods of time, and we hope that this article can serve as a basis for further research in this area.... weniger
Thesaurusschlagwörter
Nachrichten; Datengewinnung; Algorithmus; Bild; Text; Video; Online-Dienst; Monitoring; Suchmaschine
Klassifikation
Erhebungstechniken und Analysetechniken der Sozialwissenschaften
Informationswissenschaft
interaktive, elektronische Medien
Freie Schlagwörter
Algorithm auditing; data collection; search engine audits; user modelling
Sprache Dokument
Englisch
Publikationsjahr
2022
Seitenangabe
S. 1-16
Zeitschriftentitel
Journal of Information Science (2022)
DOI
https://doi.org/10.1177/01655515221093029
ISSN
1741-6485
Status
Veröffentlichungsversion; begutachtet (peer reviewed)
Lizenz
Creative Commons - Namensnennung 4.0
FörderungGefördert durch die Deutsche Forschungsgemeinschaft (DFG) - Projektnummer 491156185 / Funded by the German Research Foundation (DFG) - Project number 491156185