Hostname: page-component-5db58dd55d-m58mf Total loading time: 0 Render date: 2026-06-03T12:13:16.342Z Has data issue: false hasContentIssue false
  • English
  • Français

Estimating Interest Groups’ Policy Positions through Content Analysis: A Discussion of Automated and Human-Coding Text Analysis Techniques Applied to Studies of EU Lobbying

Published online by Cambridge University Press:  01 January 2026

Adriana Bunea  and
Raimondas Ibenskas
Adriana Bunea*
Affiliation:
Department of Political Science, University College London, 31 Tavistock Square, London WC1H 9QU, UK
Raimondas Ibenskas*
Affiliation:
Department of Politics and International Relations, University of Southampton, Highfield Campus, University Road, Southampton SO17 1BJ, UK
*
E-mail: a.bunea@ucl.ac.uk
E-mail: R.Ibenskas@soton.ac.uk
Get access Rights & Permissions [Opens in a new window]

Abstract

The promises and pitfalls of automated (computer-assisted) and human-coding content analysis techniques applied to political science research have been extensively discussed in the scholarship on party politics and legislative studies. This study presents a similar comparative analysis outlining the pay-offs and trade-offs of these two methods of content analysis applied to research on EU lobbying. The empirical focus is on estimating interest groups’ positions based on their formally submitted policy position documents in the context of EU policymaking. We identify the defining characteristics of these documents and argue that the choice for a method of content analysis should be informed by a concern for addressing the specificities of the research topic covered, of the research question asked and of the data sources employed. We discuss the key analytical assumptions and methodological requirements of automated and human-coding text analysis and the degree to which they match the identified text characteristics. We critically assess the most relevant methodological challenges research designs face when these requirements need to be complied with and how these challenges might affect measurement validity. We also compare the two approaches in terms of their reliability and resource intensity. The article concludes with recommendations and issues for future research.

Keywords

EU lobbyingpolicy position estimatestext analysisautomated and human-coding techniques

Information

Type
Symposium
Information
European Political Science , Volume 16 , Issue 3 , September 2017 , pp. 337 - 351
Copyright
Copyright © 2016 European Consortium for Political Research

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

*

Corresponding author.

References

Alexa, M. and Zuell, C. (2000) ‘Text analysis software: Commonalities, differences and limitations: The results of a review’, Quality and Quantity 34(3): 299-321 CrossRefGoogle Scholar
Benoit, K. and Laver, M. (2006) Party Policy in Modern Democracies, London: Routledge.CrossRefGoogle Scholar
Benoit, K. and Laver, M. (2007) ‘Estimating party policy positions: Comparing expert surveys and hand-coded content analysis’, Electoral Studies 26(1): 90-107 10.1016/j.electstud.2006.04.008CrossRefGoogle Scholar
Benoit, K. and Laver, M. (2012) ‘The Dimensionality of Political Space: Epistemological and Methodological Considerations’, European Union Politics 13(2): 194-218 10.1177/1465116511434618CrossRefGoogle Scholar
Boräng, F., Eising, R., Klüver, H., Mahoney, C., Naurin, D., Rasch, D. and Rozbicka, P. (2014) ‘Identifying frame: A comparison of research methods’, Interest Groups and Advocacy 3(2): 188-201 CrossRefGoogle Scholar
Budge, I. and Pennings, P. (2007) ‘Do they work? Validating computerized word frequency estimates against policy series’, Electoral Studies 26(1): 121-129 10.1016/j.electstud.2006.04.002CrossRefGoogle Scholar
Bunea, A. (2013) ‘Issues, preferences and ties: Determinants of EU interest groups’ preference attainment in the environmental policy area’, Journal of European Public Policy 20(4): 552-570 10.1080/13501763.2012.726467CrossRefGoogle Scholar
Bunea, A. (2014) ‘Explaining interest groups’ articulation of policy preferences in the European commission’s open consultations: An analysis of the environmental policy area’, Journal of Common Market Studies 52(6): 1224-1241 10.1111/jcms.12151CrossRefGoogle Scholar
Bunea, A. and Baumgartner, F.R. (2014) ‘The state of the discipline: Authorship, research designs, and citation patterns in studies of EU interest groups and lobbying’, Journal of European Public Policy 21(10): 1412-1434 CrossRefGoogle Scholar
Bunea, A. and Ibenskas, R. (2015) ‘Quantitative text analysis and the study of EU lobbying and interest groups’, European Union Politics 16(3): 429-455 10.1177/1465116515577821CrossRefGoogle Scholar
Caldeira, G.A. and Wright, J.R. (1990) ‘Amici Curiae before the supreme court: Who participates, when and how much?’, Journal of Politics 52(3): 782-806 CrossRefGoogle Scholar
Diermeier, D., Godbout, J.F., Bei, Y. and Kaufmann, S. (2012) ‘Language and ideology in congress’, British Journal of Political Science 42(1): 31-55 CrossRefGoogle Scholar
Eising, R. (2016, this issue of European Political Science) ‘Studying interest groups: Methodological challenges and tools’, European Political Science, doi: 10.1057/eps.2016.14.CrossRefGoogle Scholar
Eising, R., Rasch, D. and Rozbicka, P. (2015) ‘Institutions, policies and arguments: Framing activities in the EU multi-level system’, Journal of European Public Policy 22(4): 516-533 10.1080/13501763.2015.1008552CrossRefGoogle Scholar
Evans, D. (1996) ‘Before the roll-call: Interest group lobbying and public policy outcomes in house’, Political Research Quarterly 49(2): 287-304 CrossRefGoogle Scholar
Grimmer, J. and Stewart, B.M. (2013) ‘Text as data: The promise and pitfalls of automatic content analysis methods for political texts’, Political Analysis 21(3): 267-297 CrossRefGoogle Scholar
Hayes, A.F. and Krippendorff, K. (2007) ‘Answering the call for a standard reliability measure for coding data’, Communication Methods and Measures 1(1): 77-89 CrossRefGoogle Scholar
Klüver, H. (2009) ‘Measuring interest group influence using quantitative text analysis’, European Union Politics 10(4): 535-549 CrossRefGoogle Scholar
Klüver, H. (2013) Lobbying in the European Union: Interest Groups, Lobbying Coalitions and Policy Change, Oxford: Oxford University Press.10.1093/acprof:oso/9780199657445.001.0001CrossRefGoogle Scholar
Klüver, H. and Mahoney, C. (2015) ‘Measuring interest group framing strategies in public policy debates’, Journal of Public Policy 35(2): 223-244 10.1017/S0143814X14000294CrossRefGoogle Scholar
Klüver, H., Mahoney, C. and Opper, M. (2015) ‘Framing in context: How interest groups employ framing to lobby the European commission’, Journal of European Public Policy 22(4): 481-498 CrossRefGoogle Scholar
Krippendorff, K. (2004) Content Analysis: An Introduction to Its Methodology, Thousand Oaks, CA; London: SAGE Publications.Google Scholar
Laver, M., Benoit, K. and Garry, J. (2003) ‘Extracting policy positions from political texts using words as data’, American Political Science Review 97(2): 311-331 10.1017/S0003055403000698CrossRefGoogle Scholar
Lowe, W. and Benoit, K. (2013) ‘Validating estimates of latent traits from textual data using human judgment as a benchmark’, Political Analysis 21(3): 298-313 10.1093/pan/mpt002CrossRefGoogle Scholar
Mikhaylov, S., Laver, M. and Benoit, K. (2012) ‘Coder reliability and misclassification in the human coding of party manifestos’, Political Analysis 20(1): 78-91 10.1093/pan/mpr047CrossRefGoogle Scholar
Nelson, D. and Yackee, S.W. (2012) ‘Lobbying coalitions and government policy change: An analysis of federal rulemaking’, Journal of Politics 74(2): 339-553 10.1017/S0022381611001599CrossRefGoogle Scholar
Neuendorf, K.A. (2002) The Content Analysis Guidebook, London: SAGE Publications.Google Scholar
Proksch, S., Lowe, W. and Soroka, S. (2015) ‘Automated multilingual content analysis: A new approach to estimating conflict from parliamentary speeches across languages’, Paper presented at the fifth annual conference of the European Political Science Association, Vienna, 25–27 June.Google Scholar
Proksch, S.O. and Slapin, J.B. (2012) ‘Institutional foundations of legislative speech’, American Journal of Political Science 56(3): 520-537 CrossRefGoogle Scholar
Rasmussen, A. (2015) ‘Participation in written government consultations in Denmark and the United Kingdom: System and actor-level effect’, Government and Opposition 50(2): 271-299 10.1017/gov.2014.16CrossRefGoogle Scholar
Rasmussen, A. and Alexandrova, P. (2012) ‘Foreign interests lobbying Brussels: Participation of non-EU members in commission consultations’, Journal of Common Market Studies 50(4): 614-631 10.1111/j.1468-5965.2012.02251.xCrossRefGoogle Scholar
Rasmussen, A. and Carroll, B.J. (2014) ‘Determinants of upper-class dominance in the heavenly chorus: Lessons from European union online consultations’, British Journal of Political Science 44(2): 445-459 10.1017/S0007123412000750CrossRefGoogle Scholar
Roberts, C.W. (2000) ‘A conceptual framework for quantitative text analysis on joining probabilities and substantive inferences about texts’, Quality and Quantity 34(3): 259-274 10.1023/A:1004780007748CrossRefGoogle Scholar
Skalski, P.D. (2002) ‘Computer Content Analysis Software’, in Neuendorf, K.A. (ed.) The Content Analysis Guidebook London: SAGE Publications, pp. 225-235.Google Scholar
Slapin, J.B. and Proksch, S.O. (2008) ‘A scaling model of for estimating time-series party positions from texts’, American Journal of Political Science 52(3): 705-722 10.1111/j.1540-5907.2008.00338.xCrossRefGoogle Scholar
Slapin, J.B. and Proksch, S.O. (2014) ‘Words as Data: Content Analysis in Legislative Studies’, in Martin, S., Saalfeld, T. and Strom, K.W. (eds.) Oxford Handbook of Legislative Studies Oxford: Oxford University Press, pp. 126-144.Google Scholar
Vannoni, M. (2016, this issue of European Political Science) ‘Studying preference attainment using spatial models’, European Political Science, doi: 10.1057/eps.2016.13.CrossRefGoogle Scholar
Voltolini, B. (2016, this issue of European Political Science) ‘Framing processes and lobbying in EU foreign policy: Case study and process tracing methods’, European Political Science, doi: 10.1057/eps.2016.18.CrossRefGoogle Scholar
Yackee, J.W. and Yackee, S.W. (2006) ‘A bias towards business? Assessing interest group influence on the US bureaucracy’, Journal of Politics 68(1): 128-139 10.1111/j.1468-2508.2006.00375.xCrossRefGoogle Scholar