Article published in:

From Text to Political Positions: Text analysis across disciplines
Edited by Bertie Kaal, Isa Maks and Annemarie van Elfrinkhof
[Discourse Approaches to Politics, Society and Culture 55] 2014
► pp. 93–116

Text to Ideology or Text to Party Status?

Graeme Hirst | Department of Computer Science, University of Toronto

Yaroslav Riabinin | Department of Computer Science, University of Toronto

Jory Graham | Department of Computer Science, University of Toronto

Magali Boizot-Roche | Department of Computer Science, University of Toronto

Colin Morris | Department of Computer Science, University of Toronto

Recent papers have used support-vector machines with word features to classify political texts by ideology. Our own work on this topic led us to hypothesize that such classifiers are sensitive not to expressions of ideology but rather to expressions of attack and defense, opposition and government. We test this hypothesis by training on one set of parliamentary speeches and testing on another in which party roles have been interchanged, and we find that the performance of the classifier completely disintegrates. Moreover, some features that are indicative of each party ‘swap sides’ with the change of government. Our results suggest that the language of attack and defense, of government and opposition, will dominate and confound any sensitivity to ideology in these kinds of classifiers.

Published online: 07 May 2014
https://doi.org/10.1075/dapsac.55.05hir

References

DEFT

(2009) Actes de l’atelier de clôture de la cinquième édition du DÉfi Fouille de Textes, Paris 22 June 2009.

Diermeier, D., J.-F. Godbout, B. Yu and S. Kaufmann

2007 Language and ideology in Congress. Annual Meeting of the Midwest Political Science Association.

Forest, D., A. van Hoeydonck, D. Létourneau and M. Bélanger

2009 Impacts de la variation du nombre de traits discriminants sur la catégorisation des documents. Actes de l’atelier de clôture de la cinquième édition du DÉfi Fouille de Textes, Paris, 22 June 2009, pp. 75–88.

Germann, U

2001 Aligned Hansards of the 36th Parliament of Canada. Available at www.isi.edu/natural-language/download/hansard/

Greene, S

2007 Spin: Lexical semantics, transitivity, and the identification of implicit sentiment. PhD thesis, University of Maryland, College Park.

Grouin, C., B. Arnulphy, J.-B. Berthelin, S. El Ayari, A. Garcia-Fernandez, A. Grappy, M. Hurault-Plantet, P. Paroubek, I. Robba and P. Zweigenbaum

2009 Présentation de l’édition du D’Efi Fouille de Textes (DEFT’09). Actes de l’atelier de clôture de la cinquième édition du DÉfi Fouille de Textes, Paris, 22 June 2009, pp. 35–50.

Hix, S., A.G. Noury and G. Roland

2007 Democratic Politics in the European Parliament. Cambridge: Cambridge University Press.

Jiang, M. and S. Argamon

2008 Political leaning categorization by exploring subjectivities in political blogs. Proceedings, 4th International Conference on Data Mining (DMIN 2008), pp. 647–653.

Lin, W., T. Wilson, J. Wiebe and A. Hauptmann

2006 Which side are you on? Identifying perspectives at the document and sentence levels. Proceedings of the 10th Conference on Natural Language Learning (CoNLL-X), pp. 109–116.

Marx, M. and A. Schuth

2010 DutchParl: The parliamentary documents in Dutch. Proceedings of the 7th Conference on Language Resources and Evaluation (LREC 2010), Malta.

Mullen, T. and R. Malouf

2006 A preliminary investigation into sentiment analysis of informal political discourse. Proceedings of the AAAI Symposium on Computational Approaches to Analysing Weblogs (AAAI-CAAW), pp. 159–162.

Pennebaker, J.W

2008 The meaning of words: Obama versus McCain. Weblog 12 October 2008 Available at wordwatchers.wordpress.com/2008/10/12/

Pennebaker, J.W., C.K. Chung, M. Ireland, A. Gonzales and R.J. Booth

2007a The Development and Psychometric Properties of LIWC2007. Available at www.liwc.net/LIWC2007LanguageManual.pdf

2007b Linguistic inquiry and word count (LIWC2007). Available at www.liwc.net

Riabinin, Y

2009 Computational identification of ideology in text: A study of Canadian parliamentary debates. MSc paper, Department of Computer Science, University of Toronto, January 2009 Available at www.cs.toronto.edu/compling/publications.html

Thomas, M., B. Pang and L. Lee

2006 Get out the vote: Determining support or opposition from congressional floor-debate transcripts. Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), pp. 327–335.

Yu, B., S. Kaufmann and D. Diermeier

2008 Classifying party affiliation from political speech. Journal of Information Technology in Politics 5(1), pp. 33–48.

Cited by

Cited by 8 other publications

Baden, Christian, Neta Kligler-Vilenchik & Moran Yarchi

2020. Hybrid Content Analysis: Toward a Strategy for the Theory-driven, Computer-assisted Classification of Large Text Corpora. Communication Methods and Measures 14:3 ► pp. 165 ff.

Baden, Christian, Christian Pipal, Martijn Schoonvelde & Mariken A. C. G van der Velden

2022. Three Gaps in Computational Text Analysis Methods for Social Sciences: A Research Agenda. Communication Methods and Measures 16:1 ► pp. 1 ff.

Cochrane, Christopher, Ludovic Rheault, Jean-François Godbout, Tanya Whyte, Michael W.-C. Wong & Sophie Borwein

2022. The Automatic Analysis of Emotion in Political Speech Based on Transcripts. Political Communication 39:1 ► pp. 98 ff.

Dallmann, Alexander, Florian Lemmerich, Daniel Zoller & Andreas Hotho

2015. In Proceedings of the 26th ACM Conference on Hypertext & Social Media - HT '15, ► pp. 133 ff.

Dehghani, Mostafa, Hosein Azarbonyad, Jaap Kamps & Maarten Marx

2016. In Experimental IR Meets Multilinguality, Multimodality, and Interaction [Lecture Notes in Computer Science, 9822], ► pp. 69 ff.

Dehghani, Mostafa, Hosein Azarbonyad, Jaap Kamps & Maarten Marx

2016. In Proceedings of the 2016 ACM International Conference on the Theory of Information Retrieval, ► pp. 185 ff.

Rheault, Ludovic & Christopher Cochrane

2020. Word Embeddings for the Analysis of Ideological Placement in Parliamentary Corpora. Political Analysis 28:1 ► pp. 112 ff.

Zuba, Krzysztof

2021. From the poster boy of Europeanization to the sick man of Europe: thirty years (1990–2019) of Poland’s European policy. European Politics and Society 22:3 ► pp. 394 ff.

This list is based on CrossRef data as of 14 may 2022. Please note that it may not be complete. Sources presented here have been supplied by the respective publishers. Any errors therein should be reported to them.