Online hatred of women in the Incels.me forum
Linguistic analysis and automatic detection
Sylvia Jaki | University of Hildesheim
Tom De Smedt | University of Antwerp
Maja Gwóźdź | University of Munich
Rudresh Panchal | Columbia University
Alexander Rossa | University of Hull
Guy De Pauw | Textgain
This paper presents a study of the (now suspended) online discussion forum Incels.me and its users, involuntary celibates or incels, a virtual community of isolated men without a sexual life, who see women as the cause of their problems and often use the forum for misogynistic hate speech and other forms of incitement. Involuntary celibates have attracted media attention and concern, after a killing spree in April 2018 in Toronto, Canada. The aim of this study is to shed light on the group dynamics of the incel community, by applying mixed-methods quantitative and qualitative approaches to analyze how the users of the forum create in-group identity and how they construct major out-groups, particularly women. We investigate the vernacular used by incels, apply automatic profiling techniques to determine who they are, discuss the hate speech posted in the forum, and propose a Deep Learning system that is able to detect instances of misogyny, homophobia, and racism, with approximately 95% accuracy.
Keywords: misogyny, hate speech, social media, forensic linguistics, text analytics, text profiling
Published online: 08 July 2019
https://doi.org/10.1075/jlac.00026.jak
https://doi.org/10.1075/jlac.00026.jak
References
References
Argamon, Shlomo, Moshe Koppel, James W. Pennebaker, and Jonathan Schler
2009 “Automatically Profiling the Author of an Anonymous Text.” Communications of the ACM 52(2):119–23. 
Berger, J. M.
2017 “Extremist Construction of Identity: How Escalating Demands for Legitimacy Shape and Define In-Group and Out-Group Dynamics.” ICCT Research Paper. https://icct.nl/wp-content/uploads/2017/04/ICCT-Berger-Extremist-Construction-of-Identity-April-2017-2.pdf.
Blommaert, Jan
2017 “Online-offline Modes of Identity and Community: Elliot Rodger’s Twisted World of Masculine Victimhood.” Tilburg Papers in Culture Studies 200. https://biblio.ugent.be/publication/8551305/file/8551306.pdf
Bou-Franch, Patricia, and Pilar Garcés-Conejos Blitvich
2016 “Gender Ideology and Social Identity Processes in Online Language Aggression against Women.” In Exploring Language Aggression against Women, edited by Patricia Bou-Franch, 59–81. Amsterdam/Philadelphia: John Benjamins.
Brennan, Michael, Zadia Afroz, and Rachel Greenstadt
2012 “Adversarial Stylometry: Circumventing Authorship Recognition to Preserve Privacy and Anonymity.” ACM Transactions on Information and System Security 15(3): Art. 12.
Brown, Alexander
2017 “What is Hate Speech? Part 1: The Myth of Hate.” Law and Philosophy 36: 419–468.
Brown, Rachel
2016 Defusing Hate: A Strategic Communication Guide to Counteract Dangerous Speech. https://www.ushmm.org/m/pdfs/20160229-Defusing-Hate-Guide.pdf
Brown, Rupert, and Hanna Zagefka
2005 “Ingroup Affiliations and Prejudice.” In On the Nature of Prejudice, edited by John F. Dovidio, Peter Glick, and Laurie A. Rudman, 54–70. Maldon, MA, etc.: Blackwell.
Chollet, François
Colleoni, Elanor, Alessandro Rozza, and Adam Arvidsson
2014 “Echo Chamber or Public Sphere? Predicting Political Orientation and Measuring Political Homophily in Twitter Using Big Data.” Journal of Communication 64(2):317–32.
Collins, Michael
2002 “Discriminative training methods for hidden markov models: Theory and experiments with perceptron algorithms.” Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing 10: 1–8.
De Smedt, Tom, and Walter Daelemans
2012 “Pattern for Python.” Journal of Machine Learning Research 13: 2063–2067. http://www.jmlr.org/papers/volume13/desmedt12a/desmedt12a.pdf
De Smedt, Tom, Guy De Pauw, and Pieter Van Ostaeyen
2018 “Automatic Detection of Online Jihadist Hate Speech.” CLiPS Technical Report Series 7: 1–30. https://www.uantwerpen.be/images/uantwerpen/container2712/files/hate-speech-detection.pdf
Dholakia, Utpal M., and Richard P. Bagozzi
2004 “Motivational Antecedents, Constituents and Consequents of Virtual Community Identity.” In Virtual and Collaborative Teams: Process, Technologies, and Practice, edited by Susan H. Godar and Sharmila P. Ferris, 252–67. Hershey/London: Idea Group Publishing.
DiMauro, Dina
2008 “Reluctant Virginity: The Relationship between Sexual Status and Self-esteem.” Theses and Dissertations 717. http://rdw.rowan.edu/etd/717
Donnelly, Denise, Elisabeth Burgess, Sally Anderson, Regina Davis, and Joy Dillard
2001 “Involuntary Celibacy: A Life Course Analysis.” The Journal of Sex Research 38(2):159–69.
Douglas, Karen M.
2009 “Psychology, Discrimination and Hate Groups Online.” In The Oxford Handbook of Internet Psychology, edited by Adam N. Joinson, Katelyn Y. A. McKenna, Tom Postmes, and Ulf-Dietrich Reips, 155–63. Oxford/New York: Oxford University Press.
Gilmore, David D.
2001 Misogyny: The Male Malady. Philadelphia: University of Pennsylvania Press.
Ging, Debbie
2017 “Alphas, Betas, and Incels: Theorizing the Masculinities of the Manosphere.” Men and Masculinities.
Haines, Russel, Jill Hough, Lan Cao, and Douglas Haines
2014 “Anonymity in Computer-mediated Communication: More Contrarian Ideas with Less Influence.” Group Decision and Negotiation 23(4):765–86.
Hamm, Mark S., and Ramón F. J. Spaij
2017 The Age of Lone Wolf Terrorism. New York/Chichester: Columbia University Press.
Haythornthwaite, Caroline
2009 “Social Networks and Online Community.” In The Oxford Handbook of Internet Psychology, edited by Adam N. Joinson, Katelyn Y. A. McKenna, Tom Postmes, and Ulf-Dietrich Reips, 121–137. Oxford/New York: Oxford University Press.
Jane, Emma A.
2014a “Back to the Kitchen, cunt”: Speaking the Unspeakable about Online Misogyny. Continuum: Journal of Media and Cultural Studies 28(4):558–70.
2014b “ ‘You’re a Ugly, Whorish, Slut’. Understanding E-bile.” Feminist Media Studies 14(4):431–46.
2017 “Systemic Misogyny Exposed: Translating Rapeglish from the Manosphere with a Random Rape Threat Generator.” International Journal of Cultural Studies 21(6):661–80.
Kämper, Andreas
2011 [r]echte Kerle. Zur Kumpanei der MännerRECHTSbewegung. Münster: UNRAST.
Kim, Yoon
2014 “Convolutional Neural Networks for Sentence Classification.” arXiv: 1408.5882, https://arxiv.org/pdf/1408.5882.pdf
Kleinke, Sonja, and Birte Bös
2015 “Intergroup Rudeness and the Metapragmatics of its Negotiation in Online Discussion Fora.” Pragmatics 25(1):47–71.
Langman, Peter
2016 “Elliot Rodger: An Analysis.” https://schoolshooters.info/sites/default/files/rodger_analysis_2.0.pdf
Larkin, Ralph W.
2018 “Learning to be a Rampage Shooter: The Case of Elliot Rodger.” In The Wiley Handbook on Violence in Education: Forms, Factors, and Preventions, edited by Harvey Shapiro, 69–84. Hoboken, NJ: Wiley & Sons.
Liu, Bing
2012 Sentiment Analysis and Opinion Mining. San Rafael, CA: Morgan & Claypool.
Mantilla, Karla
2013 “Gendertrolling: Misogyny Adapts to New Media.” Feminist Studies 39(2):563–70. https://www.jstor.org/stable/23719068
ed. 2015 Gendertrolling: How Misogyny Went Viral. Santa Barbara, CA: Praeger Press.
Marx, Konstanze
2018 “Cybermobbing aus sprachwissenschaftlicher Perspektive.” Sprachreport 1/18: 1–9. https://ids-pub.bsz-bw.de/frontdoor/index/index/docId/7209
Musolff, Andreas
2012 “The Study of Metaphor as Part of Critical Discourse Analysis.” Critical Discourse Studies 9(3):301–10.
Newman, Matthew L., Carla J. Groom, Lori D. Handelman, and James W. Pennebaker
2008 “Gender Differences in Language Use: An Analysis of 14,000 Text Samples.” Discourse Processes 45: 211–36.
Neuman, Yair, Dan Assaf, Yochai Cohen, and James L. Knoll
2015 “Profiling School Shooters: Automatic Text-based Analysis.” Frontiers in Psychiatry 6: Art. 86.
Nockleby, John T.
2000 “Hate Speech.” In Encyclopedia of the American Constitution. Vol. 3, 2nd ed., edited by Leonard W. Levy and Kenneth L. Karst, 1277–1279. Detroit: Macmillan Reference US.
Pennebaker, James W.
2011 The Secret Life of Pronouns. What our Words Say about us. New York: Bloomsbury.
Pennebaker, James W., Martha E. Francis, and Roger J. Booth
2001 Linguistic Inquiry and Word Count: LIWC 2001. Mahwah, NJ: Lawrence Erlbaum Associates.
Plummer, Ken
2005 “Male Sexualities.” In Handbook of Studies on Men & Masculinities, edited by Michael S. Kimmel, Jeff Hearn, and R. W. Connell, 178–195. Thousand Oaks/London/New Delhi: Sage.
Rego, Richard
2018 “Changing Forms and Platforms of Misogyny: Sexual Harassment of Women Journalists on Twitter.” Media Watch 9(3):472–485.
Salter, Anastasia, and Bridget Blodgett
2012 “Hypermasculinity & Dickwolves: The Contentious Role of Women in the New Gaming Public.” Journal of Broadcasting & Electronic Media 56(3):401–16.
Schmidt, Anna, and Michael Wiegand
2017 “A Survey on Hate Speech Detection Using Natural Language Processing.” Proceedings of the Fifth Workshop on Natural Language Processing for Social Media: 1–10.
Suler, John
2004 “The Online Disinhibition Effect.” CyberPsychology & Behavior 7(3):321–6.
Turnage, Anna K.
2007 “Email Flaming Behaviors and Organizational Conflict.” Computer-Mediated Communication 13(1):43–59.
Van Valkenburgh, Shawn P.
2018 “Digesting the Red Pill: Masculinity and Neoliberalism in the Manosphere.” Men and Masculinities.
Waltman, Michael, and John Haas
2011 The Communication of Hate. New York: Peter Lang.
Zimmerman, Shannon, Luisa Ryan, and David Duriesmith
2018 “Recognizing the Violent Extremist Ideology of ‘Incels’.” Women in International Security policybrief 09/2018: 1–5. http://www.academia.edu/37532571/Recognizing_the_Violent_Extremist_Ideology_of_Incels
Cited by
Cited by 30 other publications
No author info given
2021.
In The Incel Rebellion: The Rise of the Manosphere and the Virtual War Against Women,
► pp. 129 ff.
Atari, Mohammad, Aida Mostafazadeh Davani, Drew Kogon, Brendan Kennedy, Nripsuta Ani Saxena, Ian Anderson & Morteza Dehghani
2021. Morally Homogeneous Networks and Radicalism.
Social Psychological and Personality Science ► pp. 194855062110593 ff.
Baele, Stephane J., Lewys Brace & Travis G. Coan
2020. The ‘tarrant effect’: what impact did far-right attacks have on the 8chan forum?.
Behavioral Sciences of Terrorism and Political Aggression ► pp. 1 ff.
Beran, Ondřej
2021. To Each Incel According to His Needs?.
SATS 22:2 ► pp. 191 ff.
Chang, Winnie
2020. The monstrous-feminine in the incel imagination: investigating the representation of women as “femoids” on /r/Braincels.
Feminist Media Studies ► pp. 1 ff.
DeCook, Julia R. & Megan Kelly
2021. Interrogating the “incel menace”: assessing the threat of male supremacy in terrorism studies.
Critical Studies on Terrorism ► pp. 1 ff.
Dynel, Marta
2020. Vigilante disparaging humour at r/IncelTears: Humour as critique of incel ideology.
Language & Communication 74 ► pp. 1 ff.
Dynel, Marta & Fabio I. M. Poppi
2020. Arcana imperii*.
Journal of Language Aggression and Conflict 8:1 ► pp. 57 ff.
Fowler, Kurt
2021. From Chads to Blackpills, a Discursive Analysis of the Incel’s Gendered Spectrum of Political Agency.
Deviant Behavior ► pp. 1 ff.
Gill, Paul, Zoe Marchment, Sanaz Zolghadriha, Nadine Salman, Bettina Rottweiler, Caitlin Clemmow & Isabelle Van Der Vegt
2020.
In Radicalization and Counter-Radicalization [Sociology of Crime, Law and Deviance, ],
► pp. 55 ff.
Hajarian, Mohammad & Zahra Khanbabaloo
2021.
In 2021 7th International Conference on Web Research (ICWR),
► pp. 169 ff.
Heritage, Frazer & Veronika Koller
2020. Incels, in-groups, and ideologies.
Journal of Language and Sexuality 9:2 ► pp. 152 ff.
Jones, Callum, Verity Trott & Scott Wright
2020. Sluts and soyboys: MGTOW and the production of misogynistic online harassment.
New Media & Society 22:10 ► pp. 1903 ff.
Kerr, Selina E.M.
2021.
In Mitigating Mass Violence and Managing Threats in Contemporary Society [Advances in Psychology, Mental Health, and Behavioral Studies, ],
► pp. 321 ff.
Menzie, Lauren
2020. Stacys, Beckys, and Chads: the construction of femininity and hegemonic masculinity within incel rhetoric.
Psychology & Sexuality ► pp. 1 ff.
Modha, Sandip, Prasenjit Majumder, Thomas Mandl & Chintak Mandalia
2020. Detecting and visualizing hate speech in social media: A cyber Watchdog for surveillance.
Expert Systems with Applications 161 ► pp. 113725 ff.
Modha, Sandip, Prasenjit Majumder & Thomas Mandl
2021. An empirical evaluation of text representation schemes to filter the social media stream.
Journal of Experimental & Theoretical Artificial Intelligence ► pp. 1 ff.
Palomino-Manjón, Patricia
2020. Feminist activism on Twitter.
Journal of Language Aggression and Conflict
Pelzer, Björn, Lisa Kaati, Katie Cohen & Johan Fernquist
2021. Toxic language in online incel communities.
SN Social Sciences 1:8
Prażmo, Ewelina
2020.
Foids are worse than animals. A cognitive linguistics analysis of dehumanizing metaphors in online discourse.
Topics in Linguistics 21:2 ► pp. 16 ff.
Preston, Kayla, Michael Halpin & Finlay Maguire
2021. The Black Pill: New Technology and the Male Supremacy of Involuntarily Celibate Men.
Men and Masculinities 24:5 ► pp. 823 ff.
Sang, Yisi & Jeffrey Stanton
2020.
In International Conference on Social Media and Society,
► pp. 5 ff.
Saragoza, Philip A., Stephen G. White & Victor Hong
2021.
In College Psychiatry [Psychiatry Update, 1],
► pp. 123 ff.
Schwartz, Martin D.
2021. Masculinities, Sport, and Violence Against Women: The Contribution of Male Peer Support Theory.
Violence Against Women 27:5 ► pp. 688 ff.
Sear, Rebecca
2021. The male breadwinner nuclear family is not the ‘traditional’ human family, and promotion of this myth may have adverse health consequences.
Philosophical Transactions of the Royal Society B: Biological Sciences 376:1827
Sugiura, Lisa
2021.
In The Palgrave Handbook of Gendered Violence and Technology,
► pp. 355 ff.
Trott, Verity Anne
2020. ‘Gillette: The best a beta can get’: Networking hegemonic masculinity in the digital sphere.
New Media & Society ► pp. 146144482097829 ff.
van der Vegt, Isabelle, Maximilian Mozes, Bennett Kleinberg & Paul Gill
2021. The Grievance Dictionary: Understanding threatening language use.
Behavior Research Methods 53:5 ► pp. 2105 ff.
Williams, D. J., Michael Arntfield, Kaleigh Schaal & Jolene Vincent
2021. Wanting sex and willing to kill: Examining demographic and cognitive characteristics of violent “involuntary celibates”.
Behavioral Sciences & the Law 39:4 ► pp. 386 ff.
Witt, Taisto
2020. ‘If i cannot have it, i will do everything i can to destroy it.' the canonization of Elliot Rodger: ‘Incel’ masculinities, secular sainthood, and justifications of ideological violence.
Social Identities 26:5 ► pp. 675 ff.
This list is based on CrossRef data as of 10 january 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.