The accuracy trap or How to build a phony classifier

Abstract

This guide explains, in four steps, how to build a phony text classifier using supervised machine learning - a classifier that is absolutely unreliable but looks outwardly sophisticated and attractive. You might enjoy this text if one or more of the following statements apply to you: You are interested in the automated identification of hate speech or related content in online discussions, as long as it looks good; you want to do something with machine learning to impress your peer group, but you do not have the nerve to dig deep into this field as well; you are either a somewhat sneaky or a humorous person. Of course, however, if you are a good and decent researcher, you might also take hints from this text on how not to step into the accuracy trap and how not to fall for the tricks of phony classification.