Prediction, Proxies, and Power

View Researcher's Other Codes

Disclaimer: The provided code links for this paper are external links. Science Nest has no responsibility for the accuracy, legality or content of these links. Also, by downloading this code(s), you agree to comply with the terms of use as set out by the author(s) of the code(s).

Authors Robert J. Carroll, Brenton Kenkel
Paper Category
Paper Abstract Many enduring questions in international relations theory focus on power relations between states, so it is important that scholars have a good measure of relative power. But the standard measure of relative military power, the capability ratio, is barely better than random guessing at predicting military dispute outcomes. We use machine learning tools to build a superior proxy, the Dispute Outcome Expectations score, from the same underlying data. Our measure is an order of magnitude better than the capability ratio at predicting dispute outcomes. In replications of 18 recent empirical studies in international relations, we find that replacing the standard measure with DOE scores usually improves both in-sample and out-of-sample goodness of fit. More broadly, we argue that scholars should focus on out-of-sample predictive power when constructing proxies for important concepts in political science. Our approach illustrates how machine learning tools can automate this process. ∗Florida State University. Email: †Vanderbilt University. Email: For all its progress—more nuanced arguments, more useful theories, bigger data and more systematic ways to analyze them—international relations remains, in many ways, a study of power. This is best reflected in the questions that have endured. Is the world safer when power is concentrated in a few states or broadly distributed (Waltz 1979)? How does the balance of power between states, or shifts thereof, affect the likelihood of war (Organski and Kugler 1980; Powell 1999, 2006)? Do international organizations allow states to gain benefits they would not receive from power politics alone (Keohane and Nye 2001)? But without good measures of power, we cannot provide good empirical answers to these fundamental questions. Consequently, the importance of measuring power to the study of international politics cannot be overstated. Like many other important concepts in political science—say, ideology or democracy—power cannot be measured directly. Indeed, measurement problems in political science often entail the construction of proxies. Recent advances in computing and modeling have allowed political scientists to build sophisticated, data-driven proxies for variables as diverse as legislator ideology (Clinton, Jackman and Rivers 2004), judicial independence (Linzer and Staton 2014), and country regime types (Jackman and Treier 2008). But despite the centrality of power to many important hypotheses in international relations, its measurement has seen far less innovation.1 In this article, we devise a new approach to measuring power—specifically, the balance of material power between a pair of countries. Our focus on the contributions of material capabilities follows the example set by most existing efforts to measure power in the international sphere, starting with the work by Singer, Bremer and Stuckey (1972). In contrast with previous approaches, ours is data-driven: we aim to learn what combination of observable material capability variables best predicts international dispute outcomes. We show that the standard measure, the ratio of Composite Index of National Capability scores (Singer, Bremer and Stuckey 1972), predicts militarized dispute outcomes terribly—only 1 percent better than a null model with random guessing. Our new proxy, the Dispute Outcome Expectations score, is much better, providing a 20 percent predictive improvement. Before constructing a new proxy for relative material power, we first consider what makes a good proxy more generally. Despite the innovations in measurement in various fields, political scientists have not reached a consen1A recent exception is Arena (2012).
Date of publication 2019
Code Programming Language R

Copyright Researcher 2022