Skip to main content Accessibility help
×
Home
Hostname: page-component-684899dbb8-5dd2w Total loading time: 0.27 Render date: 2022-05-18T14:33:20.483Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true }

Measuring the Significance of Policy Outputs with Positive Unlabeled Learning

Published online by Cambridge University Press:  19 October 2020

RADOSLAW ZUBEK*
Affiliation:
University of Oxford
ABHISHEK DASGUPTA*
Affiliation:
University of Oxford
DAVID DOYLE*
Affiliation:
University of Oxford
*
Radoslaw Zubek, Associate Professor, Department of Politics and International Relations, University of Oxford, radoslaw.zubek@politics.ox.ac.uk.
Abhishek Dasgupta, Research Software Engineer, Department of Computer Science, University of Oxford, abhishek.dasgupta@cs.ox.ac.uk.
David Doyle Associate Professor, Department of Politics and International Relations, University of Oxford, david.doyle@politics.ox.ac.uk.

Abstract

Identifying important policy outputs has long been of interest to political scientists. In this work, we propose a novel approach to the classification of policies. Instead of obtaining and aggregating expert evaluations of significance for a finite set of policy outputs, we use experts to identify a small set of significant outputs and then employ positive unlabeled (PU) learning to search for other similar examples in a large unlabeled set. We further propose to automate the first step by harvesting “seed” sets of significant outputs from web data. We offer an application of the new approach by classifying over 9,000 government regulations in the United Kingdom. The obtained estimates are successfully validated against human experts, by forecasting web citations, and with a construct validity test.

Type
Letter
Copyright
© The Author(s), 2020. Published by Cambridge University Press on behalf of the American Political Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank Ken Benoit, Shaun Bevan, Jack Blumenau, Ruth Dixon, Andy Eggers, Mikolaj Firlej, Diego Garzia, Felix Krawatzek, Ben Lauderdale, Innar Liiv, Marcin Matczak, Nolan McCarty, Sharon Meraz, Scot Peterson, Ulrich Sieberer, Jayne Sunley, Mariusz Zubek, and three anonymous reviewers for helpful comments and suggestions. We acknowledge the financial support from BA/Leverhulme Small Research Grant (SG152293) and John Fell OUP Research Fund (151/025). Abhishek Dasgupta acknowledges support from the Centre for Technology and Global Affairs. David Doyle acknowledges support from the Jackie Lambert Research Fund at St Hugh’s College. Our thanks to Kwok Cheung, Tom Fleming, Vladimir Mikulik, Paul Moore, Edward Percarpio, Hugh Thomas, Zoe Harrison, Neerja Gurnani, Atrayee De, and Jacob Yu for excellent research assistance. Replication files are available at the American Political Science Review Dataverse: https://doi.org/10.7910/DVN/1XXDMW.

References

Angelova, Mariyana, Hanna, Bäck, Wolfgang C., Müller, and Daniel, Strobl. 2018. “Veto Player Theory and Reform Making in Western Europe.” European Journal of Political Research 57 (2): 282307.CrossRefGoogle ScholarPubMed
Becher, Michael. 2010. “Constraining Ministerial Power: The Impact of Veto Players on Labor Market Reforms in Industrial Democracies, 1973-2000.” Comparative Political Studies 43 (1): 3360.CrossRefGoogle Scholar
Chang, Yin-Wen, and Lin, Chih-Jen. 2008. “Feature Ranking Using Linear SVM.” In Proceedings of the Workshop on Causation and Prediction Challenge at WCCI 3: 5364.Google Scholar
Clinton, Joshua D., and Lapinski, John S.. 2006. “Measuring Legislative Accomplishment, 1877–1994.” American Journal of Political Science 50 (1): 232249.CrossRefGoogle Scholar
Conley, Richard S., and Bekafigo, Marija A.. 2010. “No Irish Need Apply? Veto Players and Legislative Productivity in the Republic of Ireland, 1949-2000.” Comparative Political Studies 43 (1): 91118.CrossRefGoogle Scholar
Döring, Herbert. 2001. “Parliamentary Agenda Control and Legislative Outcomes in Western Europe.” Legislative Studies Quarterly 26 (1): 145165.CrossRefGoogle Scholar
Epstein, Lee, and Segal, Jeffrey A.. 2000. “Measuring Issue Salience.” American Journal of Political Science 44 (1): 6683.CrossRefGoogle Scholar
Howell, William, Scott, Adler, Charles, Cameron, and Charles, Riemann. 2000. “Divided Government and the Legislative Productivity of Congress, 1945-94.” Legislative Studies Quarterly 25 (2): 285312.CrossRefGoogle Scholar
Huber, John D., and Shipan, Charles R.. 2002. Deliberate Discretion? The Institutional Foundations of Bureaucratic Autonomy. New York: Cambridge University Press.CrossRefGoogle Scholar
Junge, Dirk, Koenig, Thomas, and Luig, Bernd. 2015. “Legislative Gridlock and Bureaucratic Politics in the European Union.” British Journal of Political Science 45 (4): 777797.CrossRefGoogle Scholar
Li, Xiaoli, and Liu, Bing. 2003. “Learning to Classify Texts Using Positive and Unlabeled Data.” In Proceedings of the 18th International Conference on Artificial intelligence, 587–592.Google Scholar
Liu, Bing. 2011. Web Data Mining. New York: Springer.CrossRefGoogle Scholar
Mayhew, David R. 1991. Divided We Govern. New Haven, CT: Yale University Press.Google Scholar
Page, Edward C. 2001. Governing by Numbers: Delegated Legislation and Everyday Policymaking. Oxford: Hart Publishing.Google Scholar
Schultz, Mike, Doerr, John E., and Frederiksen, Lee. 2013. Professional Services Marketing: How the Best Firms Build Premier Brands, Thriving Lead Generation Engines, and Cultures of Business Development Success. Hoboken, NJ: John Wiley & Sons.Google Scholar
Seely, Antony. 2020. The Budget and Annual Finance Bill. Briefing Paper 813. Westminister: House of Commons Library.Google Scholar
Tsebelis, George. 1999. “Veto Players and Law Production in Parliamentary Democracies: An Empirical Analysis.” American Political Science Review 93 (3): 591608.Google Scholar
Warber, Adam L., Ouyang, Yu, and Waterman, Richard W.. 2018. “Landmark Executive Orders.” Presidential Studies Quarterly 48 (1): 110126.CrossRefGoogle Scholar
Zhang, Bangzuo, and Zuo, Wanli. 2008. “Learning from Positive and Unlabeled Examples: A Survey.” In 2008 International Symposiums on Information Processing, 650–654.Google Scholar
Supplementary material: Link

Zubek et al. Dataset

Link
Supplementary material: PDF

Zubek et al. supplementary material

Online Appendix

Download Zubek et al. supplementary material(PDF)
PDF 7 MB

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Measuring the Significance of Policy Outputs with Positive Unlabeled Learning
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

Measuring the Significance of Policy Outputs with Positive Unlabeled Learning
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

Measuring the Significance of Policy Outputs with Positive Unlabeled Learning
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *