Classification Accuracy as a Substantive Quantity of Interest: Measuring Polarization in Westminster Systems

View Researcher's Other Codes

Disclaimer: The provided code links for this paper are external links. Science Nest has no responsibility for the accuracy, legality or content of these links. Also, by downloading this code(s), you agree to comply with the terms of use as set out by the author(s) of the code(s).

Authors Andrew Peterson, Arthur Spirling
Journal/Conference Name POLITICAL ANALYSIS
Paper Category
Paper Abstract Measuring the polarization of legislators and parties is a key step in understanding how politics develops over time. But in parliamentary systems—where ideological positions estimated from roll calls may not be informative—producing valid estimates is extremely challenging. We suggest a new measurement strategy, that makes innovative use of the ‘accuracy’ of machine classifiers, i.e. the number of correct predictions made as a proportion of all predictions. In our case, the ‘labels’ are the party identifications of the members of parliament, predicted from their speeches, along with some information on debate subjects. Intuitively, when the learner is able to discriminate members in the two main Westminster parties well, we claim we are in a period of ‘high’ polarization. By contrast, when the classifier has low accuracy—and makes a relatively large number of mistakes in terms of allocating members to parties based on the data—we argue parliament is in an era of ‘low’ polarization. This approach is fast and substantively valid, and we demonstrate its merits with simulations, and by comparing the estimates from 78 years of House of Commons speeches with qualitative and quantitative historical accounts of the same. As a headline finding, we note that contemporary British politics is approximately as polarized as it was in the mid1960s—that is, in the middle of the ‘post-war consensus’. More broadly, we show that the technical performance of supervised learning algorithms can be directly informative about substantive matters in social science. Word count: 3136 (excluding abstract and Online Appendices) ∗We are grateful to Niels Goet, Justin Grimmer and Ben Lauderdale for comments an earlier draft. Audiences at the European Political Science Association meeting and the American Political Science Association meeting provided helpful feedback. Comments from two anonymous referees and the editor at Political Analysis improved our manuscript considerably. Our replication materials for this paper may be found here: †Postdoctoral Researcher, University of Geneva. ‡Associate Professor of Politics and Data Science, New York University.
Date of publication 2018
Code Programming Language Python

Copyright Researcher 2022