Skip to main content Accessibility help
×
Home

Machine Learning Human Rights and Wrongs: How the Successes and Failures of Supervised Learning Algorithms Can Inform the Debate About Information Effects

  • Kevin T. Greene (a1), Baekkwan Park (a1) and Michael Colaresi (a1)

Abstract

There is an ongoing debate about whether human rights standards have changed over the last 30 years. The evidence for or against this shift relies upon indicators created by human coders reading the texts of human rights reports. To help resolve this debate, we suggest translating the question of changing standards into a supervised learning problem. From this perspective, the application of consistent standards over time implies a time-constant mapping from the textual features in reports to the human coded scores. Alternatively, if the meaning of abuses have evolved over time, then the same textual features will be labeled with different numerical scores at distinct times. Of course, while the mapping from natural language to numerical human rights score is a highly complicated function, we show that these two distinct data generation processes imply divergent overall patterns of accuracy when we train a wide variety of algorithms on older versus newer sets of observations to learn how to automatically label texts with scores. Our results are consistent with the expectation that standards of human rights have changed over time.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Machine Learning Human Rights and Wrongs: How the Successes and Failures of Supervised Learning Algorithms Can Inform the Debate About Information Effects
      Available formats
      ×

      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Machine Learning Human Rights and Wrongs: How the Successes and Failures of Supervised Learning Algorithms Can Inform the Debate About Information Effects
      Available formats
      ×

      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Machine Learning Human Rights and Wrongs: How the Successes and Failures of Supervised Learning Algorithms Can Inform the Debate About Information Effects
      Available formats
      ×

Copyright

This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits noncommercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.

Corresponding author

Footnotes

Hide All

Authors’ note: This material is based upon work supported by the National Science Foundation under Grant No. 1753528.

Contributing Editor: Justin Grimmer

Footnotes

References

Hide All
Bagozzi, B., and Berliner, D.. 2016. The politics of scrutiny in human rights monitoring: Evidence from structural topic models of US State Department human rights reports. Political Science Research and Methods: 1–17. doi:10.1017/psrm.2016.44.
Clark, A. M., and Sikkink, K.. 2013. Information effects and human rights data: Is the good news about increased human rights information bad news for human rights measures? Human Rights Quarterly 35(3):539568.
Fariss, C. J. 2014. Respect for human rights has improved over time: Modeling the changing standard of accountability. American Political Science Review 108(2):297318.
Fariss, C. J., Linder, F. J., Jones, Z. M., Crabtree, C. D., Biek, M. A., Ross, A.-S. M., Kaur, T., and Tsai, M.. 2015. Human rights texts: Converting human rights primary source documents into data. PLOS ONE 10(9): e0138935.
Flach, P. 2012. Machine learning: The art and science of algorithms that make sense of data . New York: Cambridge.
Grimmer, J., and Stewart, B. M.. 2013. Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Political Analysis 21(3):267297.
Keck, M., and Sikkink, K.. 1998. Activists beyond borders: Advocacy networks in international politics . Itaca, NY: Cornell University Press.
Kotsiantis, S. B. 2007. Supervised machine learning: A review of classification techniques. Informatica 31(1):249268.
Mitchell, T. 1998. Machine learning . New York: McGraw-Hill.
Monroe, B. L., Colaresi, M. P., and Quinn, K. M.. 2008. Fightin’words: Lexical feature selection and evaluation for identifying the content of political conflict. Political Analysis 16(4):372403.
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., and Radev, D. R.. 2010. How to analyze political attention with minimal assumptions and costs. American Journal of Political Science 54(1):209228.
Richards, D. L. 2012. “Rhetoric and Reality” Revisited. Journal of Human Rights 11(3):337343.
Richards, D. L.2015. The myth of information effects: A reply to Clark and Sikkink. Working paper, University of Connecticut.
Ward, M. 2016. Can we predict politics? Toward what end? Journal of Global Security Studies 1(1):8091.
MathJax
MathJax is a JavaScript display engine for mathematics. For more information see http://www.mathjax.org.

Keywords

Type Description Title
UNKNOWN
Supplementary materials

Greene et al. supplementary material
Greene et al. supplementary material 1

 Unknown (423 KB)
423 KB

Metrics

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed