Hostname: page-component-76fb5796d-r6qrq Total loading time: 0 Render date: 2024-04-29T15:10:24.269Z Has data issue: false hasContentIssue false
  • English
  • Français

Automated Coding of Political Campaign Advertisement Videos: An Empirical Validation Study

Published online by Cambridge University Press:  10 November 2022

Alexander Tarr Open the ORCID record for Alexander Tarr [Opens in a new window] ,
June Hwang  and
Kosuke Imai Open the ORCID record for Kosuke Imai [Opens in a new window]
Alexander Tarr
Affiliation:
Graduate Student, Department of Electrical Engineering, Princeton University, Princeton, NJ 08544, USA. E-mail: atarr3@gmail.com
June Hwang
Affiliation:
Advisor, Consulate General of the Republic of Korea, Honolulu, HI 96817, USA. E-mail: wjhwangusa@gmail.com
Kosuke Imai*
Affiliation:
Professor, Department of Government and Department of Statistics, Harvard University, Cambridge, MA 02138, USA. E-mail: imai@harvard.edu, URL: https://imai.fas.harvard.edu
*
Corresponding author Kosuke Imai
Get access Rights & Permissions [Opens in a new window]

Abstract

Video advertisements, either through television or the Internet, play an essential role in modern political campaigns. For over two decades, researchers have studied television video ads by analyzing the hand-coded data from the Wisconsin Advertising Project and its successor, the Wesleyan Media Project (WMP). Unfortunately, manually coding more than a hundred of variables, such as issue mentions, opponent appearance, and negativity, for many videos is a laborious and expensive process. We propose to automatically code campaign advertisement videos. Applying state-of-the-art machine learning methods, we extract various audio and image features from each video file. We show that our machine coding is comparable to human coding for many variables of the WMP datasets. Since many candidates make their advertisement videos available on the Internet, automated coding can dramatically improve the efficiency and scope of campaign advertisement research. Open-source software package is available for implementing the proposed methodology.

Keywords

audio datacomputer visionimage datamachine learningtext datavideo data
Type
Article
Information
Political Analysis , Volume 31 , Issue 4 , October 2023 , pp. 554 - 574
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of the Society for Political Methodology

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Edited by Jeff Gill

References

Banda, K. K. 2015. “Competition and the Dynamics of Issue Convergence.” American Politics Research 43: 821–845.CrossRefGoogle Scholar
Baskurt, K. B., and Samet, R.. 2019. “Video Synopsis: A Survey.” Computer Vision and Image Understanding 181: 26–38.CrossRefGoogle Scholar
Cano, P., Batlle, E., Kalker, T., and Haitsma, J.. 2005. “A Review of Audio Fingerprinting.” Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology 41: 271–284.CrossRefGoogle Scholar
Chakraborty, S., Tickoo, O., and Iyer, R.. 2015. “Adaptive Keyframe Selection for Video Summarization.” In 2015 IEEE Winter Conference on Applications of Computer Vision (WACV), 702–709. IEEE.CrossRefGoogle Scholar
Cohn, J. F., Ambadar, Z., and Ekman, P.. 2007. “Observer-Based Measurement of Facial Expression with the Facial Action Coding System.” In Handbook of Emotion Elicitation and Assessment, edited by Coan, J. A., and Allen, J. J. B., 203–221. New York: Oxford University Press.Google Scholar
Dietrich, B. J. 2021. “Using Motion Detection to Measure Social Polarization in the U.S. House of Representatives.” Political Analysis 29: 250–259.CrossRefGoogle Scholar
Dietrich, B. J., Enos, R. D., and Sen, M.. 2018. “Emotional Arousal Predicts Votes on the Supreme Court.” Political Analysis 27: 237–243.CrossRefGoogle Scholar
Downie, J. S. 2008. “The Music Information Retrieval Evaluation Exchange (2005–2007): A Window into Music Information Retrieval Research.” Acoustical Science and Technology 29: 247–255.CrossRefGoogle Scholar
Fridkin, K. L., and Kenney, P.. 2011. “Variability in Citizens’ Reactions to Different Types of Negative Campaigns.” American Journal of Political Science 55: 307–325.CrossRefGoogle Scholar
Gerber, A. S., Gimpel, J. G., Green, D. P., and Shaw, D. R.. 2011. “How Large and Long-Lasting Are the Persuasive Effects of Televised Campaign Ads? Results from a Randomized Field Experiment.” American Political Science Review 105: 135–150.CrossRefGoogle Scholar
Grabe, M. E., and Bucy, E. P.. 2009. Image Bite Politics: News and the Visual Framing of Elections. New York: Oxford University Press.CrossRefGoogle Scholar
Grother, P. J., Ngan, M. L., Hanaoka, K. K., et al. 2019. “Face Recognition Vendor Test Part 3: Demographic Effects.” NIST Interagency/Internal Report (NISTIR) No. 8280, National Institute of Standards and Technology.CrossRefGoogle Scholar
Haitsma, J., and Kalker, T.. 2002. “A Highly Robust Audio Fingerprinting System.” In ISMIR 2002, 107–115.Google Scholar
Jin, X., and Tan, X.. 2017. “Face Alignment in-the-Wild: A Survey.” Computer Vision and Image Understanding 162: 1–22.10.1016/j.cviu.2017.08.008CrossRefGoogle Scholar
Joo, J., Bucy, E. P., and Seidel, C.. 2019. “Coding of Televised Leader Displays: Detecting Nonverbal Political Behavior with Computer Vision and Deep Learning.” International Journal of Communication 13: 4044–4066.Google Scholar
Kang, T., Fowler, E. F., Franz, M. M., and Ridout, T. N.. 2017. “Issue Consistency? Comparing Television Advertising, Tweets, and E-Mail in the 2014 Senate Campaigns.” Political Communication 35: 32–49.CrossRefGoogle Scholar
Kaplan, N., Park, D. K., and Ridout, T. N.. 2006. “Dialogue in American Political Campaigns? An Examination of Issue Convergence in Candidate Television Advertising.” American Journal of Political Science 50: 724–736.CrossRefGoogle Scholar
Kazemi, V., and Sullivan, J.. 2014. “One Millisecond Face Alignment with an Ensemble of Regression Trees.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1867–1874.Google Scholar
King, D. E. 2009. “Dlib-ml: A Machine Learning Toolkit.” Journal of Machine Learning Research 10: 1755–1758.Google Scholar
Knox, D., and Lucas, C.. 2021. “A Dynamic Model of Speech for the Social Sciences.” American Political Science Review 115: 649–666.CrossRefGoogle Scholar
Krupnikov, Y. 2011. “When Does Negativity Demobilize? Tracing the Conditional Effect of Negative Campaigning on Voter Turnout.” American Journal of Political Science 55: 797–813.CrossRefGoogle Scholar
Luo, W., Xing, J., Milan, A., Zhang, X., Liu, W., Zhao, X., and Kim, T.-K.. 2021. “Multiple Object Tracking: A Literature Review.” Artificial Intelligence 293: 103448.CrossRefGoogle Scholar
Matz, S. C., Segalin, C., Stillwell, D., Müller, S. R., and Bos, M. W.. 2019. “Predicting the Personal Appeal of Marketing Images Using Computational Methods.” Journal of Consumer Psychology 29: 370–390.CrossRefGoogle Scholar
Mehr, S. A., et al. 2019. “Universality and Diversity in Human Song.” Science 366: eaax0868.CrossRefGoogle ScholarPubMed
Meirick, P. C., et al. 2018. “To Tell the Truth: Ad Watch Coverage, Ad Tone, and the Accuracy of Political Advertising.” Political Communication 35: 450–469.CrossRefGoogle Scholar
Prabhavalkar, R., Rao, K., Sainath, T. N., Li, B., Johnson, L., and Jaitly, N.. 2017. “A Comparison of Sequence-to-Sequence Models for Speech Recognition.” Interspeech 2017: 939–943.CrossRefGoogle Scholar
Proksch, S.-O., Wratil, C., and Wäckerle, J.. 2019. “Testing the Validity of Automatic Speech Recognition for Political Text Analysis.” Political Analysis 27: 339–359.CrossRefGoogle Scholar
Ren, J.-M., Ming-Ju, W., and Jang, J.-S. R.. 2015. “Automatic Music Mood Classification Based on Timbre and Modulation Features.” IEEE Transactions on Affective Computing 6: 236–246.CrossRefGoogle Scholar
Schaffner, B. F. 2005. “Priming Gender: Campaigning on Women’s Issues in U.S. Senate Elections.” American Journal of Political Science 49: 803–817.CrossRefGoogle Scholar
Schroff, F., Kalenichenko, D., and Philbin, J.. 2015. “FaceNet: A Unified Embedding for Face Recognition and Clustering.” In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 815–823.Google Scholar
Schwemmer, C., Knight, C., Bello-Pardo, E. D., Oklobdzija, S., Schoonvelde, M., and Lockhart, J. W.. 2020. “Diagnosing Gender Bias in Image Recognition Systems.” Socius 6: 1–17.CrossRefGoogle ScholarPubMed
Sides, J., and Karch, A.. 2008. “Messages that Mobilize? Issue Publics and the Content of Campaign Advertising.” Journal of Politics 70: 466–476.CrossRefGoogle Scholar
Soltau, H., Liao, H., and Sak, H.. 2016. “Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition.” Proc. Interspeech 2017, pp. 3707–3711.Google Scholar
Tarr, A., Imai, K.. and Hwang, J.. 2022. “Replication Data for: Automated Coding of Political Campaign Advertisement Videos: An Empirical Validation Study.” Harvard Dataverse, V1. https://doi.org/10.7910/DVN/6SWKPR CrossRefGoogle Scholar
Torres, M. 2018. “Understanding Visual Messages: Visual Framing and the Bag of Visual Words.” Technical Report. Applied Statistics Workshop, Institute of Quantitative Social Science, Harvard University.Google Scholar
Torres, M. and Cantu, F.. 2021. “Learning to See: Convolutional Neural Networks for the Analysis of Social Science Data.” Political Analysis 30: 113–131.CrossRefGoogle Scholar
Wang, A. et al. 2003. “An Industrial Strength Audio Search Algorithm.” In ISMIR 2003, 7–13. Washington.Google Scholar
Wang, M., and Deng, W.. 2018. “Deep Face Recognition: A Survey.” Neurocomputing 429: 215–244.CrossRefGoogle Scholar
Wesleyan Media Project. 2017. Political Advertising in 2014 Codebook, ver 1.0. Technical Report.Google Scholar
Williams, N. W., Casas, A., and Wilkerson, J. D.. 2020. Images as Data for Social Science Research: An Introduction to Convolutional Neural Nets for Image Classification. New York: Cambridge University Press.CrossRefGoogle Scholar
Xi, N., Ma, D., Liou, M., Steinert-Threlkeld, Z. C., Anastasopoulos, J., and Joo, J.. 2020. “Understanding the Political Ideology of Legislators from Social Media Images.” In Proceedings of the International AAAI Conference on Web and Social Media, vol. 14: 726–737.CrossRefGoogle Scholar
Xiong, W., et al. 2016. “Achieving Human Parity in Conversational Speech Recognition.” Preprint, arXiv:1610.05256.Google Scholar
Ye, Q., and Doermann, D.. 2015. “Text Detection and Recognition in Imagery: A Survey.” IEEE Transactions on Pattern Analysis and Machine Intelligence 37: 1480–1500.CrossRefGoogle ScholarPubMed
Young, L., and Soroka, S.. 2012. “Affective News: The Automated Coding of Sentiment in Political Texts.” Political Communication 29: 205–231.CrossRefGoogle Scholar
Zhu, Y., Yao, C., and Bai, X.. 2016. “Scene Text Detection and Recognition: Recent Advances and Future Trends.” Frontiers of Computer Science 10: 19–36.CrossRefGoogle Scholar
Supplementary material: PDF

Tarr et al. supplementary material

Tarr et al. supplementary material

Download Tarr et al. supplementary material(PDF)
PDF 2.5 MB
Supplementary material: Link

Tarr et al. Dataset

Dataset

https://doi.org/10.7910/DVN/6SWKPR
Link