Hostname: page-component-848d4c4894-wg55d Total loading time: 0 Render date: 2024-05-16T12:19:02.714Z Has data issue: false hasContentIssue false
  • English
  • Français

Measuring the Significance of Policy Outputs with Positive Unlabeled Learning

Published online by Cambridge University Press:  19 October 2020

RADOSLAW ZUBEK Open the ORCID record for RADOSLAW ZUBEK [Opens in a new window] ,
ABHISHEK DASGUPTA Open the ORCID record for ABHISHEK DASGUPTA [Opens in a new window]  and
DAVID DOYLE Open the ORCID record for DAVID DOYLE [Opens in a new window]
RADOSLAW ZUBEK*
Affiliation:
University of Oxford
ABHISHEK DASGUPTA*
Affiliation:
University of Oxford
DAVID DOYLE*
Affiliation:
University of Oxford
*
Radoslaw Zubek, Associate Professor, Department of Politics and International Relations, University of Oxford, radoslaw.zubek@politics.ox.ac.uk.
Abhishek Dasgupta, Research Software Engineer, Department of Computer Science, University of Oxford, abhishek.dasgupta@cs.ox.ac.uk.
David Doyle Associate Professor, Department of Politics and International Relations, University of Oxford, david.doyle@politics.ox.ac.uk.
Get access Rights & Permissions [Opens in a new window]

Abstract

Identifying important policy outputs has long been of interest to political scientists. In this work, we propose a novel approach to the classification of policies. Instead of obtaining and aggregating expert evaluations of significance for a finite set of policy outputs, we use experts to identify a small set of significant outputs and then employ positive unlabeled (PU) learning to search for other similar examples in a large unlabeled set. We further propose to automate the first step by harvesting “seed” sets of significant outputs from web data. We offer an application of the new approach by classifying over 9,000 government regulations in the United Kingdom. The obtained estimates are successfully validated against human experts, by forecasting web citations, and with a construct validity test.

Type
Letter
Information
American Political Science Review , Volume 115 , Issue 1 , February 2021 , pp. 339 - 346
Copyright
© The Author(s), 2020. Published by Cambridge University Press on behalf of the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank Ken Benoit, Shaun Bevan, Jack Blumenau, Ruth Dixon, Andy Eggers, Mikolaj Firlej, Diego Garzia, Felix Krawatzek, Ben Lauderdale, Innar Liiv, Marcin Matczak, Nolan McCarty, Sharon Meraz, Scot Peterson, Ulrich Sieberer, Jayne Sunley, Mariusz Zubek, and three anonymous reviewers for helpful comments and suggestions. We acknowledge the financial support from BA/Leverhulme Small Research Grant (SG152293) and John Fell OUP Research Fund (151/025). Abhishek Dasgupta acknowledges support from the Centre for Technology and Global Affairs. David Doyle acknowledges support from the Jackie Lambert Research Fund at St Hugh’s College. Our thanks to Kwok Cheung, Tom Fleming, Vladimir Mikulik, Paul Moore, Edward Percarpio, Hugh Thomas, Zoe Harrison, Neerja Gurnani, Atrayee De, and Jacob Yu for excellent research assistance. Replication files are available at the American Political Science Review Dataverse: https://doi.org/10.7910/DVN/1XXDMW.

References

Angelova, Mariyana, Hanna, Bäck, Wolfgang C., Müller, and Daniel, Strobl. 2018. “Veto Player Theory and Reform Making in Western Europe.” European Journal of Political Research 57 (2): 282–307.CrossRefGoogle ScholarPubMed
Becher, Michael. 2010. “Constraining Ministerial Power: The Impact of Veto Players on Labor Market Reforms in Industrial Democracies, 1973-2000.” Comparative Political Studies 43 (1): 33–60.CrossRefGoogle Scholar
Chang, Yin-Wen, and Lin, Chih-Jen. 2008. “Feature Ranking Using Linear SVM.” In Proceedings of the Workshop on Causation and Prediction Challenge at WCCI 3: 53–64.Google Scholar
Clinton, Joshua D., and Lapinski, John S.. 2006. “Measuring Legislative Accomplishment, 1877–1994.” American Journal of Political Science 50 (1): 232–249.CrossRefGoogle Scholar
Conley, Richard S., and Bekafigo, Marija A.. 2010. “No Irish Need Apply? Veto Players and Legislative Productivity in the Republic of Ireland, 1949-2000.” Comparative Political Studies 43 (1): 91–118.CrossRefGoogle Scholar
Döring, Herbert. 2001. “Parliamentary Agenda Control and Legislative Outcomes in Western Europe.” Legislative Studies Quarterly 26 (1): 145–165.CrossRefGoogle Scholar
Epstein, Lee, and Segal, Jeffrey A.. 2000. “Measuring Issue Salience.” American Journal of Political Science 44 (1): 66–83.CrossRefGoogle Scholar
Greentarget. 2017. State of Digital & Content Marketing Survey. https://greentarget.com/wp-content/uploads/2019/07/2017-State-of-Digital-Content-Marketing-Survey-FINAL.pdf.Google Scholar
Howell, William, Scott, Adler, Charles, Cameron, and Charles, Riemann. 2000. “Divided Government and the Legislative Productivity of Congress, 1945-94.” Legislative Studies Quarterly 25 (2): 285–312.CrossRefGoogle Scholar
Huber, John D., and Shipan, Charles R.. 2002. Deliberate Discretion? The Institutional Foundations of Bureaucratic Autonomy. New York: Cambridge University Press.CrossRefGoogle Scholar
Junge, Dirk, Koenig, Thomas, and Luig, Bernd. 2015. “Legislative Gridlock and Bureaucratic Politics in the European Union.” British Journal of Political Science 45 (4): 777–797.CrossRefGoogle Scholar
Li, Xiaoli, and Liu, Bing. 2003. “Learning to Classify Texts Using Positive and Unlabeled Data.” In Proceedings of the 18th International Conference on Artificial intelligence, 587–592.Google Scholar
Liu, Bing. 2011. Web Data Mining. New York: Springer.CrossRefGoogle Scholar
Mayhew, David R. 1991. Divided We Govern. New Haven, CT: Yale University Press.Google Scholar
Page, Edward C. 2001. Governing by Numbers: Delegated Legislation and Everyday Policymaking. Oxford: Hart Publishing.Google Scholar
Schultz, Mike, Doerr, John E., and Frederiksen, Lee. 2013. Professional Services Marketing: How the Best Firms Build Premier Brands, Thriving Lead Generation Engines, and Cultures of Business Development Success. Hoboken, NJ: John Wiley & Sons.Google Scholar
Seely, Antony. 2020. The Budget and Annual Finance Bill. Briefing Paper 813. Westminister: House of Commons Library.Google Scholar
Tsebelis, George. 1999. “Veto Players and Law Production in Parliamentary Democracies: An Empirical Analysis.” American Political Science Review 93 (3): 591–608.Google Scholar
Warber, Adam L., Ouyang, Yu, and Waterman, Richard W.. 2018. “Landmark Executive Orders.” Presidential Studies Quarterly 48 (1): 110–126.CrossRefGoogle Scholar
Zhang, Bangzuo, and Zuo, Wanli. 2008. “Learning from Positive and Unlabeled Examples: A Survey.” In 2008 International Symposiums on Information Processing, 650–654.Google Scholar
Supplementary material: Link

Zubek et al. Dataset

Dataset

https://doi.org/10.7910/DVN/1XXDMW
Link
Supplementary material: PDF

Zubek et al. supplementary material

Online Appendix

Download Zubek et al. supplementary material(PDF)
PDF 6.7 MB