Algorithms as Co-Researchers

doi:10.1017/9781009106436.018

Chapter 14 - Algorithms as Co-Researchers

Exploring Meaning and Bias in Qualitative Research

from Part III - Illustrative Examples and Emergent Issues

Published online by Cambridge University Press: 08 June 2023

Wendy Arianne Günther ,

Mark Thompson ,

Mayur P. Joshi and

Stavros Polykarpou

Edited by

Boyka Simeonova and

Robert D. Galliers

Show author details

Boyka Simeonova: Affiliation:
University of Leicester
Robert D. Galliers: Affiliation:
Bentley University, Massachusetts and Warwick Business School

Book contents

Get access

Summary

This chapter discusses the reflexive relationship between qualitative researchers and the process of selecting, forming, processing and interpreting data in algorithmic qualitative research. Drawing on Heidegger’s ideas, it argues that such research is necessarily synthetic – even creative – in that these activities inflect, and are in turn inflected by, the data itself. Thus, methodological transparency is key to understanding how different types of meanings become infused in the process of algorithmic qualitative research. While algorithmic research practices provide multiple opportunities for creating transparent meaning, researchers are urged to consider how such practices can also introduce and reinforce human and algorithmic bias in the form of unacknowledged introduction of perspectives into the data. The chapter demonstrates this reflexive dance of meaning and bias using an illustrative case of topic modelling. It closes by offering some recommendations for engaging actively with the domain, considering a multi-disciplinary approach, and adopting complementary methods that could potentially help researchers in fostering transparency and meaning.

Keywords

algorithmic qualitative research algorithms transparency meaning opacity bias topic modelling methodology data

Information

Type: Chapter
Information: Cambridge Handbook of Qualitative Digital Research , pp. 211 - 228

DOI: https://doi.org/10.1017/9781009106436.018 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Abbasi, A., Sarker, S. and Chiang, R. (2016). Big data research in information systems: Toward an inclusive research agenda. Journal of the Association for Information Systems, 17(2), i–xxxii.Google Scholar

Agrawal, A., Fu, W. and Menzies, T. (2018). What is wrong with topic modeling? And how to fix it using search-based software engineering. Information and Software Technology, 98, 74–88.Google Scholar

Anderson, C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired, www.wired.com/2008/06/pb-theory/.Google Scholar

Avgerou, C. (2019). Contextual explanation: Alternative approaches and persistent challenges. MIS Quarterly, 43(3), 977–1006.Google Scholar

Bao, Y. and Datta, A. (2014). Simultaneously discovering and quantifying risk types from textual risk disclosures. Management Science, 60(6), 1371–1391.CrossRef Google Scholar

Bapna, S., Benner, M. J. and Qiu, L. (2019). Nurturing online communities: An empirical investigation. MIS Quarterly, 43(2), 425–452.CrossRef Google Scholar

Barthes, R. (1972). The Structuralist Activity: Critical Essays. Evanston, IL: Northwestern University Press.Google Scholar

Barrett, M. and Oborn, E. (2018). Bridging the research-practice divide: Harnessing expertise collaboration in making a wider set of contributions. Information and Organization, 28(1), 44–51.Google Scholar

Belford, M., Mac Namee, B. and Greene, D. (2018). Stability of topic modeling via matrix factorization. Expert Systems with Applications, 91, 159–169.Google Scholar

Berry, M. W., Browne, M., Langville, A. N., Pauca, B. P. and Plemmons, R. J. (2007). Algorithms and applications for approximate nonnegative matrix factorization. Computational Statistics & Data Analytics, 52(1), 155–173.Google Scholar

Binkley, D., Heinz, D., Lawrie, D. and Overfelt, J. (2014). Understanding LDA in source code analysis. Proceedings of the International Conference on Program Comprehension, Hyderabad, India.Google Scholar

Bird, S., Klein, E. and Loper, E. (2009). Natural Language Processing with Python: Analyzing Text with the Natural Language Toolkit. Sebastopol, CA: O’Reilly Media, Inc.Google Scholar

Blei, D. M. (2012). Probabilistic topic models. Communications of the ACM, 55(4), 77–84.CrossRef Google Scholar

Blei, D. M. and Lafferty, J. D. (2009). Topic models. In Srivastava, A. N. and Sahami, M. (eds), Text Mining: Classification, Clustering, and Applications. London: Chapman & Hall, pp. 71–94.Google Scholar

Brynjolfsson, E. and McAfee, A. (2017). The business of artificial intelligence. Harvard Business Review, 7, 1–20.Google Scholar

Busa, R. (1990). Informatics and the new philology. Computers and the Humanities, 24(5–6), 339–343.Google Scholar

Chang, J., Gerrish, S., Wang, C., Boyd-Graber, J. L. and Blei, D. M. (2009). Reading tea leaves: How humans interpret topic models. Advances in Neural Information Processing Systems, 22, 288–296.Google Scholar

Chen, L., Baird, A. and Straub, D. W. (2019). An analysis of the evolving intellectual structure of health information systems research in the information systems discipline. Journal of the Association for Information Systems, 20(8), 1023–1074.Google Scholar

Christin, A. (2020). Algorithmic ethnography, during and after COVID-19. Communication and the Public, 5(3–4), 108–111.Google Scholar

Creswell, J. (2003). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 2nd edition. Los Angeles, CA: Sage.Google Scholar

Croidieu, G. and Kim, P. H. (2018). Labor of love: Amateurs and lay-expertise legitimation in the early U.S. radio field. Administrative Science Quarterly, 63(1), 1–42.Google Scholar

Dong, W., Liao, S. and Zhang, Z. (2018). Leveraging financial social media data for corporate fraud detection. Journal of Management Information Systems, 35(2), 461–487.Google Scholar

Dreyfus, H. (1993). What Computers Still Can’t Do: A Critique of Artificial Reason, 2nd edition. Cambridge, MA: MIT Press.Google Scholar

Dubé, L. and Paré, G. (2003). Rigor in information systems positivist case research: Current practices, trends, and recommendations. MIS Quarterly, 27(4), 597–636.CrossRef Google Scholar

Dwivedi, Y. K., Hughes, L., Ismagilova, E., Aarts, G., Coombs, C., Crick, T. et al. (2021). Artificial intelligence (AI): Multidisciplinary perspectives on emerging challenges, opportunities, and agenda for research, practice and policy. International Journal of Information Management, 57, 1–47.Google Scholar

Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550.Google Scholar

Evangelopoulos, N. (2016). Thematic orientation of the ISJ within a semantic space of IS research. Information Systems Journal, 26(1), 39–46.Google Scholar

Evangelopoulos, N., Zhang, X. and Prybutok, V. R. (2012). Latent semantic analysis: five methodological recommendations. European Journal of Information Systems, 21(1), 70–86.CrossRef Google Scholar

Galdas, P. (2017). Revisiting bias in qualitative research: Reflections on its relationship with funding and impact. International Journal of Qualitative Methods, 16(1), 1–2.Google Scholar

George, G., Haas, M. R. and Pentland, A. (2014). Big data and management. Academy of Management Journal, 57(2), 321–326.Google Scholar

Geva, H., Oestreicher-Singer, G. and Saar-Tsechansky, M. (2019). Using retweets when shaping our online persona: Topic modeling approach. MIS Quarterly, 43(2), 501–524.Google Scholar

Giorgi, S. and Weber, K. (2015). Marks of distinction: Framing and audience appreciation in the context of investment advice. Administrative Science Quarterly, 60(2), 333–367.Google Scholar

Giorgi, S., Maoret, M. and Zajac, E. J. (2019). On the relationship between firms and their legal environment: The role of cultural consonance. Organization Science, 30(4), 803–830.Google Scholar

Glaser, V. L., Pollock, N. and D’Adderio, L. (2021). The biography of an algorithm: Performing algorithmic technologies in organizations. Organization Theory, 2(2), 1–27.Google Scholar

Goes, P. B. (2014). Editor’s comments: Big data and IS research. MIS Quarterly, 38(3), iii–viii.Google Scholar

Gong, J., Abhishek, V. and Li, B. (2018). Examining the impact of keyword ambiguity on search advertising performance: A topic model approach. MIS Quarterly, 42(3), 805–829.CrossRef Google Scholar

Günther, W. and Joshi, M. (2020). Algorithmic intelligence in research: Prevalent topic modeling practices and implications for rigor in IS and management research. International Conference on Information Systems, virtual conference.Google Scholar

Haans, R. F. (2019). What’s the value of being different when everyone is? The effects of distinctiveness on performance in homogeneous versus heterogeneous categories. Strategic Management Journal, 40(1), 3–27.Google Scholar

Hannigan, T., Haans, R. F., Vakili, K., Tchalian, H., Glaser, V., Wang, M. et al. (2019). Topic modeling in management research: Rendering new theory from textual data. Academy of Management Annals, 13(2), 586–632.Google Scholar

Heidegger, M. (1962). Being and Time. Oxford, UK and Cambridge, MA: Blackwell.Google Scholar

Heidegger, M. (1977). The question concerning technology. In Krell, D. F. (ed.), Basic Writings. New York, NY: HarperCollins, pp. 213–238.Google Scholar

Hickman, L., Thapa, S., Tay, L., Cao, M. and Srinivasan, P. (2022). Text preprocessing for text mining in organizational research: Review and recommendations. Organizational Research Methods, 25(1), 114–146.Google Scholar

Huang, A. H., Lehavy, R., Zang, A. Y. and Zheng, R. (2018). Analyst information discovery and interpretation roles: A topic modeling approach. Management Science, 64(6), 2833–2855.Google Scholar

Introna, L. and Ilharco, F. (2011). Phenomenology, screens, and screenness: Returning to the world itself. In Galliers, R. D. and Currie, W. (eds), The Oxford Handbook of Management Information Systems: Critical Perspectives and New Directions. Oxford: Oxford University Press.Google Scholar

Jarzabkowski, P., Bednarek, R. and Lê, J. K. (2014). Producing persuasive findings: Demystifying ethnographic textwork in strategy and organization research, Strategic Organization, 12(4), 274–287.Google Scholar

Jones, M. (2019). What we talk about when we talk about (big) data. Journal of Strategic Information Systems, 28(1), 3–16.Google Scholar

Jones, S. (2016). Roberto Busa, S. J., and the Emergence of Humanities Computing: The Priest and the Punched Cards. London: Routledge.Google Scholar

Kaplan, S. and Vakili, K. (2015). The double‐edged sword of recombination in breakthrough innovation. Strategic Management Journal, 36(10), 1435–1457.Google Scholar

Karanović, J., Berends, H. and Engel, Y. (2020). Regulated dependence: Platform workers’ responses to new forms of organizing. Journal of Management Studies, 58(4), 1070–1106.Google Scholar

Kitchin, R. (2014). Big data, new epistemologies and paradigm shifts. Big Data Society, 1(1), 1–12.Google Scholar

Klein, H. K. and Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23(1), 67–93.Google Scholar

Larsen, K. R. and Bong, C. H. (2016). A tool for addressing construct identity in literature reviews and meta-analyses. MIS Quarterly, 40(3), 529–551.Google Scholar

Lash, M. T. and Zhao, K. (2016). Early predictions of movie success: The who, what, and when of profitability. Journal of Management Information Systems, 33(3), 874–903.Google Scholar

Latour, B. (1990). Technology is society made durable. Sociological Review, 38(Supp. 1), 103–131.Google Scholar

Lee, G. M., Qiu, L. and Whinston, A. B. (2016). A friend like me: Modeling network formation in a location-based social network. Journal of Management Information Systems, 33(4), 1008–1033.Google Scholar

Mackenzie, A. (2017). Machine Learners: Archaeology of a Data Practice. Cambridge, MA: MIT Press.Google Scholar

Madsen, A. K. (2015). Between technical features and analytic capabilities: Charting a relational affordance space for digital social analytics. Big Data & Society, 2(1), 1–15.Google Scholar

Manovich, L (2011). Trending: The promises and the challenges of big social data. Manovich, http://manovich.net/index.php/projects/trending-the-promises-and-the-challenges-of-big-social-data.Google Scholar

McCarty, W. (2003). Humanities computing, www.mccarty.org.uk/essays/McCarty,%20Humanities%20computing.pdf.Google Scholar

Newell, S. and Marabelli, M. (2015). Strategic opportunities (and challenges) of algorithmic decision-making: A call for action on the long-term societal effects of ‘datification’. Journal of Strategic Information Systems, 24(1), 3–14.Google Scholar

Nielsen, M. W. and Börjeson, L. (2019). Gender diversity in the management field: Does it matter for research outcomes? Research Policy, 48(7), 1617–1632.Google Scholar

Pääkkönen, J. and Ylikoski, P. (2021). Humanistic interpretation and machine learning. Synthese, 199(1), 1461–1497.Google Scholar

Patton, M. Q. (2002). Qualitative Research & Evaluation Methods, 3rd edition. Thousand Oaks, CA: Sage.Google Scholar

Ramsay, S. (2011). Reading Machines: Toward an Algorithmic criticism. Champaign, IL: University of Illinois Press.Google Scholar

Ribes, D. and Jackson, S. J. (2013). Data bite man: The work of sustaining long-term study. In Gitelman, L. (ed.), ‘Raw Data’ Is an Oxymoron. Cambridge, MA: MIT Press, pp. 147–166.Google Scholar

Roberts, M. E., Stewart, B. M., Tingley, D. and Airoldi, E. M. (2013). The structural topic model and applied social science. Advances in Neural Information Processing Systems Workshop on Topic Models: Computation, Application, and Evaluation, Lake Tahoe, NV.Google Scholar

Rockwell, G. and Sinclair, S. (2016). Computer-Assisted Interpretation in the Humanities. Cambridge, MA: MIT Press.Google Scholar

Salge, T. O., Antons, D., Barrett, M., Kohli, R., Oborn, E. and Polykarpou, S. (2022). How IT investments help hospitals gain and sustain reputation in the media: The role of signaling and framing. Information Systems Research, 33(1), 110–130.Google Scholar

Samtani, S., Chinn, R., Chen, H. and NunamakerJr, J. F. (2017). Exploring emerging hacker assets and key hackers for proactive cyber threat intelligence. Journal of Management Information Systems, 34(4), 1023–1053.Google Scholar

Schmiedel, T., Müller, O. and vom Brocke, J. (2019). Topic modeling as a strategy of inquiry in organizational research: A tutorial with an application example on organizational culture. Organizational Research Methods, 22(4), 941–968.Google Scholar

Schultze, U. and Avital, M. (2011). Designing interviews to generate rich data for information systems research. Information and Organization, 21(1), 1–16.CrossRef Google Scholar

Shadish, W. R., Cook, T. D. and Campbell, D. T. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston, MA: Houghton, Mifflin & Co., Vol. XXI.Google Scholar

Shi, Z., Lee, G. M. and Whinston, A. B. (2016). Toward a better measure of business proximity: Topic modeling for industry intelligence. MIS Quarterly, 40(4), 1035–1056.Google Scholar

Shi, D., Guan, J., Zurada, J. and Manikas, A. (2017). A data-mining approach to identification of risk factors in safety management systems. Journal of Management Information Systems, 34(4), 1054–1081.Google Scholar

Sidorova, A., Evangelopoulos, N., Valacich, J. S. and Ramakrishnan, T. (2008). Uncovering the intellectual core of the information systems discipline. MIS Quarterly, 32(3), 467–482.CrossRef Google Scholar

Suchman, L. (2006). Human-Machine Reconfigurations. Cambridge: Cambridge University Press.Google Scholar

Suchman, L. (1987). Plans and Situated Actions: The Problem of Human-Machine Communication (Learning in Doing: Social, Cognitive and Computational Perspectives). Cambridge: Cambridge University Press.Google Scholar

Tashakkori, A. and Creswell, J. W. (2007). Exploring the nature of research questions in mixed methods research. Journal of Mixed Methods Research, 1(3), 207–211.Google Scholar

Tracy, S. J. (2013). Qualitative Research Methods: Collecting Evidence, Crafting Analysis, Communicating Impact. Chichester, UK: Wiley-Blackwell.Google Scholar

Tully, J. (1994). Philosophy in an Age of Pluralism. Cambridge: Cambridge University Press.Google Scholar

Van Maanen, J. (2011). Ethnography as work: Some rules of engagement. Journal of Management Studies, 48(1), 218–234.Google Scholar

Van Maanen, J. and de Rond, M. (2017). The making of a classic ethnography: Notes on Alice Goffman’s On the Run. Academy of Management Review, 42(2), 396–406.Google Scholar

Weber, M. (2003). The Protestant Ethic and the Spirit of Capitalism. New York, NY: Courier Dover.Google Scholar

Wendt, T. (2013). Designing for transparency and the myth of the modern interface. Ux Magazine, https://uxmag.com/articles/designing-for-transparency-and-the-myth-of-the-modern-interface.Google Scholar

Winner, M. (1999). Do artifacts have politics? In MacKenzie, D. A. and Wajcman, J. (eds), The Social Shaping of Technology, 2nd edition. London: Open University Press, pp. 28–40. (Originally printed in Daedalus, 109(1), winter 1980.)Google Scholar

Winograd, T. and Flores, F. (1986). Understanding Computers and Cognition: A New Foundation for Design. Norwood, NJ: Ablex.Google Scholar

Yin, R. K. (1981). The case study crisis: Some answers. Administrative Science Quarterly, 26(1), 58–65.Google Scholar

Yue, W. T., Wang, Q. H. and Hui, K. L. (2019). See no evil, hear no evil? Dissecting the impact of online hacker forums. MIS Quarterly, 43(1), 73–95.Google Scholar

Zammuto, R. F., Griffith, T. L., Majchrzak, A., Dougherty, D. J. and Faraj, S. (2007). Information technology and the changing fabric of organization. Organization Science, 18(5), 749–762.Google Scholar

Accessibility standard: Unknown

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the HTML of this chapter is currently unknown and may be updated in the future.