I (the first author) remember my first position as an assistant professor. Although I trained in a prominent organizational psychology doctoral program in the United States, the only job interviews and offers I received were from business schools. When I got to my new job, I was able to get a courtesy appointment in my university's psychology department (then ranked the second best in the country). Upon meeting one of the senior faculty in industrial–organizational (I-O) psychology, I was asked about the kind of research I did. After discussing the theoretical topics I examined, I noted that most of my research was qualitative. Her response was short and to the point: “Well someone has to do that kind of research.” The tone was comparable to the judge in Caddyshack who tells an aspiring lawyer who cannot afford law school, “The world needs ditch diggers, too.”
Now, nearly 20 years after that incident, the use of qualitative research has become more prominent among psychologists. Indeed, in a historical review of qualitative research in psychology, Wertz (Reference Wertz2014, p. 14) suggests that “its position has become less polemic and more integrated with the field.” Similarly, a recent article opens with the following:
In recent years, an active coalition of psychologists exploring vistas in qualitative inquiry has spearheaded the development of the Society for Qualitative Inquiry in Psychology. Importantly, the society has now become a full-fledged participant in the American Psychological Association (APA), prominently situated within Division 5. The previous name of the division—Evaluation, Measurement, & Statistics—will be replaced with the Division of Quantitative and Qualitative Methods. Adding further weight to these ventures, a new APA journal—Qualitative Psychology—is in its first year of publication. (Gergen, Josselson, & Freeman, Reference Gergen, Josselson and Freeman2015, p. 1)
The article further notes that the qualitative research section of the British Psychological Society has grown at such a fast rate that, in less than 10 years, it has becomes its largest section (Gergen et al., Reference Gergen, Josselson and Freeman2015). However, interest in qualitative methods has been uneven. In this article, we examine the prevalence (or lack thereof) of qualitative research in I-O psychology. We also review some of the myths that continue to plague the method, and we provide some avenues to incorporate more qualitative research into I-O psychology. Throughout this article, we wish to encourage thoughtful discussion of the reasons why qualitative research has been underappreciated in our area—and what can be done to rectify this situation, if rectification is indeed needed. We believe it is.
Maps: The What, Why, and Where of Qualitative Research
What
Qualitative research, broadly defined, refers to “any type of research that produces findings not arrived at by statistical procedures or other means of quantification” (Strauss & Corbin, Reference Strauss and Corbin1998, pp. 10–11). In the Handbook of Research Methods in Industrial and Organizational Psychology, Locke and Golden-Biddle (Reference Locke, Golden-Biddle and Goldberg2002) further suggest that qualitative research often is conducted in the field, where the subjects of one's studies are working. It also often refers to methods that involve qualitative data (e.g., text) as well as results that are communicated in a nonquantitative fashion. Examples of qualitative methods include, but are not limited to, ethnography, grounded theory, case studies, phenomenology, narrative analysis, hermeneutics, and some types of content analysis. Qualitative research also tends to be inductive in nature, emphasizing the building or elaborating of theory rather than theory testing (cf. Bitektine, Reference Bitektine2008). As such, the purpose of inductive qualitative research is to better understand the worldview of the people one is studying and then to translate that understanding to an academic audience. This approach stands in contrast to traditional deductive quantitative research, wherein the researcher wishes to determine whether the participants conform to his or her theory(-ies)—a very top down approach. In this respect, the relationship between the researcher and the researched in inductive qualitative research is somewhat reversed from its typical role: The researcher depends on the researched to share their understanding of the world rather than imposing his or her worldview on the researched. It is for this reason that qualitative researchers often refer to those they study as “informants” rather than “respondents” (surveys) or “subjects/participants” (experimental studies). The focus of our article will be on inductive qualitative research.
According to McGrath's (Reference McGrath1981) discussion of the “three-horned dilemma” that outlines the strengths and limitations of different research methods, qualitative methods tend to excel at realism at the expense of generalizability and precision. It may be due to these particular weaknesses that psychologists in general—and I-O psychologists in particular—have tended to eschew qualitative methods. With regard to precision, some have argued that our theories advance by becoming more precise in terms of their boundary conditions (Edwards, Reference Edwards2010). Inductive qualitative methods, by contrast, tend to build and expand theory. Moreover, qualitative methods, because they often have small sample sizes, are poor with regard to statistical generalizability. However, as we note below, they can have some naturalistic and analytic generalizability. Despite its shortcomings, there are a variety of unique features to inductive qualitative research that should be attractive to an I-O psychologist.
Why
The choice of which method(s) to employ should follow the research question one chooses to pose. Below, we discuss several reasons why I-O psychologists may wish to engage in research questions that would benefit from qualitative methods.
1. Organizational changes create new research questions. To begin, some might argue that such trends as globalization, demographic changes in the workforce, and the increasing influence of social media and other technological advancements means that how we organize, how we work, and how we relate to one another in the workplace is changing. Given these trends, I-O psychology scholars share the collective responsibility to investigate if, how, and why these trends influence our theorizing about organizations, the work conducted therein, and the relationships that permeate them (e.g., leader–follower interaction, teamwork, conflict management). Shifts in research questions and research areas in response to broader societal and organizational changes is not a new insight. As noted by Rousseau and Fried (Reference Rousseau and Fried2001),
Research is a product of its time. In their review of organizational behavior research since World War II, Goodman and Whetten (1998) observe that researchers’ focus shifts depending upon the issues of the day. Individual productivity dominated in the aftermath of the war, with a focus on the rebuilding of civilian economies. Over time, workers became more skilled, affluent, and educated. Research topics shifted to reflect the rise of team structures and growing concern over quality of work life. . . . With the passage of time, the meaning of various constructs and the factors giving rise to them can change . . . [and] it is important to acknowledge that our constructs themselves can change meanings over time. By this we mean that the meaning of the terms can shift, often unannounced. Thus, when we think of organizational commitment today, one has to ask what exactly the “organization” is that either the individual or the researcher has in mind. (p. 5)
We argue that inductive qualitative research is ideally positioned to understand changes that are affecting organizations, with the end result being theory elaboration or the emergence of new theories. Such an influx of new theories and theorizing may be welcome because many of our major theories have remained relatively unchanged for 40 years or more. Indeed as Suddaby, Hardy, and Huy (Reference Suddaby, Hardy and Huy2011, p. 236) note, “most of the theories of organization used by contemporary management researchers were formulated several decades ago, largely in the 1960s and 1970s, and these theories have persisted, mostly intact, since that time.” To be sure, this quotation refers to traditional management theories; however, the same sentiment applies to the field of I-O psychology to some extent as well. As argued eloquently by Lee, Mitchell, and Harman (Reference Lee, Mitchell, Harman and Zedeck2011, p. 82), “qualitative research provides a different and enriching window for observing behavioral phenomenon and can be invaluable for providing a different perspective on topics that are in need of some renovation and creative new thinking.”
2. Understanding “how” and “why” rather than how many. Quantitative research excels at answering questions about quantity—how much or how many. For example, if I implement X, what percentage of people will respond with Y (or Z)? Inductive qualitative research is designed to ask different but no less important questions: “how” (process) and “why” (for similar arguments, see Bluhm, Harman, Lee, & Mitchell, Reference Bluhm, Harman, Lee and Mitchell2011; Lee et al., Reference Lee, Mitchell, Harman and Zedeck2011). In this way, it can advance our thinking even in areas where research is relatively mature. To illustrate, we know a fair amount about socialization practices and its impact on members. Research has suggested that socialization can change how people think about themselves. Divestiture tactics (Van Maanen & Schein, Reference Van Maanen, Schein and Staw1979), in particular, are designed to strip away a person's sense of who they are and replace it with another identity (e.g., military socialization). However, how this happens was left unspecified until a series of inductive studies helped to illuminate the process and “elaborate” theory by “filling in” what we do not yet know (Ibarra, Reference Ibarra1988; Pratt, Rockmann, & Kaufmann, Reference Pratt, Rockmann and Kaufmann2006).
We should be clear in noting that qualitative methods are not the only means of answering these types of questions, just that they are well tailored to answer them. For example, quantitative researchers often try to get at the why with mediation models and the identification of mechanisms. However, organizational systems are complex, and this complexity is sometimes at odds with the necessarily distilled processes studied in mediation models. Moreover, quantitative methods may be tied more closely to extant theories given the largely theory-testing focus of our field. As a result, insights and “mechanisms” that may emerge from study participants are less likely to be noted in quantitative research, even when these insights would allow for greater explanatory power in that context. Mirroring our earlier remarks, inductive qualitative research gets at the “how” and the “why” from the perspective of those we study. To the degree that one's research questions are attempting to get at how or why organizational members think, feel, and act in the way that they do, qualitative research may be appropriate.
3. Contextualizes our findings. Qualitative research methods may also be applicable when one's research questions are geared to better understand how particular contexts influence how people think, feel, and act. As noted by McGrath (Reference McGrath1981), qualitative research excels at realism and emphasizes the role of context in our research. One might expect that the choice of being I-O psychologists, rather than social psychologists for example, is motivated, at least in part, by our belief that there is something about organizations that may influence our understandings of phenomena and our theorizing. This point was made forcefully by Heath and Sitkin (Reference Heath and Sitkin2001) who have argued for emphasizing the “O” in organizational behavior (and we would argue, in I-O research) and not focusing on decontextualized behavior (“Big B” research) or on behaviors not exclusive to organizational contexts (“Contextualized B”). Big “O” research takes into account the organizing process—and thus what makes our research uniquely organizational (see also Johns, Reference Johns2006). More recently, in their chapter on qualitative research published in the latest APA handbook of I-O psychology, Lee et al. (Reference Lee, Mitchell, Harman and Zedeck2011) noted of the importance of understanding context,
We submit that knowledge of qualitative methods offers additional tools to all psychologists with which to understand not only context as a substantive or control variable but many other phenomena as well. Qualitative research will not, and should not, replace quantitative methods, but it has an important role in contributing to, and supplementing researchers’ understanding of, behavior in organizations. (p. 73)
Thus, a method that is highly sensitive to context and processes, such as qualitative methods, would seem uniquely positioned to produce Big “O” research.
4. Facilitates impact. One additional reason for why a researcher may want to use qualitative methods has less to do with the kinds of questions one might ask than with what one wants to do with the results of one's research. Qualitative research facilitates impact in two ways. First, because its goal is to understand the worldviews of those we study, we have the ability to translate findings to people in that organization (and potentially to other practitioners) more easily. This ease of translation, of course, may vary depending on the type of qualitative methodology used. For example, ethnographic research (Spradley, Reference Spradley1979) or some forms of insider–outsider research (Bartunek & Louis, Reference Bartunek and Louis1996), where a member of the organization studied joins the scholarly research team, may be easier to communicate than a hermeneutic analysis (“the theory or philosophy of the interpretation of meaning,” particularly in text data; Bleicher, Reference Bleicher1980, p. 1).
There is also some evidence, albeit from our “sister” field of organizational behavior, that inductive qualitative research may be more impactful with regard to academic scholarship as well. To illustrate, in 2006, the Academy of Management Journal (AMJ) polled its board to see what were the “most interesting management articles written in the past 100 years” (Bartunek, Rynes, & Ireland, Reference Bartunek, Rynes and Ireland2006). Although this list was notable in that there was very little consensus (perhaps not surprisingly), from the little consensus that did appear, it was clear that papers utilizing qualitative research or mixed methods were overrepresented. More recently, Bansal and Corley (Reference Bansal and Corley2011) pointed out that six of the last eight papers that had been awarded AMJ’s Best Paper Award were qualitative. The reasons why qualitative articles may be successful are, in part, because of the richness of the stories we tell through qualitative data (cf. Pollock & Bono, Reference Pollock and Bono2013) and the theoretical insights they impart about how individuals in organizations operate. Unfortunately, as we note below, given the paucity of qualitative research in top I-O psychology journals, the potential impact of qualitative research in I-O psychology remains an empirical issue.
Where
Despite various potential reasons for its use, qualitative research has traditionally had a hard time being accepted in I-O psychology and organizational behavior/human resources management (hereafter, OBHR) research. In fact, Eby, Hurst, and Butts (Reference Eby, Hurst, Butts, Lance and Vandenberg2009) sought to determine whether qualitative research was perceived as a “redheaded stepchild in organizational and social science research” (p. 219). As part of their analysis, they calculated the publication rate for qualitative and mixed-method papers in the top three journals in three disciplines (applied psychology, management, and social psychology). The picture they painted was rather bleak—roughly 3% of the articles published in these journals contained mixed methods, and only 1% were pure qualitative papers. Their analysis spanned the years 1990 to 2005.
Inspired by their work, we wondered whether the state of our science had changed since 2005. We replicated their analyses for the 2006–2013 period. Given the readership of this journal, we chose to focus only on the top journals for I-O psychology and OBHR. We list the journals in Table 1. We classified articles according to their method—purely qualitative or mixing both qualitative and quantitative traditions. See Appendix A for additional information about our methodology.
Table 1. Publication Rate in Top Journals in I-O Psychology and OBHR (2006–2013)
Note. The total of articles published excludes editorials and book reviews. A total of 3,528 articles were published in all journals. Percentages reflect within-journal ratios of qualitative or mixed-method papers. I-O = industrial–organizational; OBHR = organizational behavior/human resources.
Our searches revealed interesting findings. First, we were surprised (and dismayed) at the number of classification errors found in PsycINFO. The search results exposed errors of commission and errors of omission. Specifically, errors of commission were those when articles were listed as containing qualitative methods when they did not. Errors of omission occurred when a mixed-methods paper was only listed as qualitative or was not listed at all but should have been listed as either qualitative or mixed methods. In short, identifying the method used in articles with the “additional limits” strategy was unreliable at best. However, this strategy was not useless. Although there was substantial overlap between the limits and the Boolean searches, each search strategy also identified articles that the other search strategy did not.
We can compare the numbers reported in Table 1 with those in Table 9.1 of Eby et al. (Reference Eby, Hurst, Butts, Lance and Vandenberg2009). We were pleased to see that some journals have a greater proportion of qualitative and mixed-method work during the 2006–2013 period than they did between 1990–2005, the period analyzed by Eby and colleagues. This is the case for AMJ, Administrative Science Quarterly (ASQ), and Human Resource Management (HRM). It should be noted that AMJ, in particular, began appointing an associate editor in 2007 to specifically handle qualitative submissions and has moved from one to three qualitative editors since that time. In addition, Organization Science (OS), a journal that also appoints qualitative editors, had a very healthy proportion of qualitative and mixed-method work. Other journals did not fare as well. In particular, the publication rate of qualitative and mixed-method pieces in Journal of Applied Psychology (JAP) remained stable (and low) since 1990. Although Personnel Psychology (PPsych), Organizational Behavior and Human Decision Processes (OBHDP), and Journal of Management (JOM) were not analyzed by Eby and colleagues, we suspect the inclusion of qualitative and mixed-methods work in those outlets would have been low as well. These are not journals known for their publication of qualitative work.
We supplemented this journal content analysis with a study of the editorial statements for these journals (see Kidd, Reference Kidd2002, for a similar analysis for general psychology). Our goal was to determine the extent to which the statements or their aims and scopes referred specifically to qualitative methods or mixed methods. We also coded the statements for catchall phrases such as “all methods are welcome.” The statements varied in length and content, but almost all referred to being open to qualitative research in general or mentioned specific approaches (e.g., case studies). Of the three journals that did not specifically mention qualitative research (OBHDP, JOM, and PPsych), OBHDP stated being particularly open to multiple study articles that have complementary methods. PPsych and JOM did not mention qualitative methods but did not specifically mention quantitative methods either.
A few findings struck us as worth mentioning. First, two journals were particularly inclusive of qualitative methods in their editorial statements. ASQ indicated that “beginning with a special issue on qualitative research in 1979, ASQ set the standard for excellence in qualitative research.” (“About,” 2000). AMJ was equally welcoming to qualitative authors, referring to the contributions of inductive qualitative work to theory building and relaxing their 40-page limit for papers presenting qualitative data (“Information for Contributors,” n.d.). It is therefore not surprising that AMJ and ASQ have two of the highest rates of qualitative papers of the journals studied.
Another journal's editorial statement is worth discussing. JAP indicated being open to “rigorously conducted qualitative research on phenomena that are difficult to capture with quantitative methods, or on phenomena that warrant inductive theory building” but went on to state that the journal welcomes “data (quantitative or qualitative) [that] are analyzed with elegant or simple statistics” (“Description,” n.d.). Although this certainly opens the door to qualitative research, it may not be as inviting to qualitative researchers as one might expect. Adding “phenomena that are difficult to capture with quantitative methods” appears to reinforce the myth that we discuss below that inductive research is only useful when exploring phenomena that have not been widely addressed using quantitative data. Furthermore, the quantification of qualitative data is appropriate only in some types of qualitative studies but certainly not in all of them (Pratt, Reference Pratt2008). We were, however, glad that the phrase “or on phenomena that warrant inductive theory building” was added to the editorial statement while this article was in the review process. Furthermore, a recent editorial (Chen, Reference Chen2015) also reinforced the journal's openness to diverse research traditions, including qualitative methods and inductive approaches. Given this openness, the historically lower publication rate for qualitative papers in JAP may improve in the future.
Finally, we were interested in perusing Organizational Research Methods (ORM) given its methodological and analytical focus. We were pleased to find a strong representation of qualitative work in this journal. Of the over 250 articles (excluding editorials and book reviews) published in ORM between 2006 and 2013, about 20% of these dealt with qualitative research. Topics ranged from an entire special issue dedicated to ethnography (Cunliffe, Reference Cunliffe2010), special features on quality assessment in qualitative methods (Esterby-Smith, Golden-Biddle, & Locke, Reference Esterby-Smith, Golden-Biddle and Locke2008), and reviews of special software (e.g., Pollach, Reference Pollach2011) to treatments of different approaches in the context of the organizational sciences, such as grounded theory (e.g., O'Reilly, Paper, & Marx, Reference O'Reilly, Paper and Marx2012), content analysis (e.g., Sonpar & Golden-Biddle, Reference Sonpar and Golden-Biddle2008), case study (e.g., Piekkari, Welch, & Paaavilainen, Reference Piekkari, Welch and Paaavilainen2009), and mixed-methods designs (e.g., Molina-Azorin, Reference Molina-Azorin2012). Several tutorials are also available, such as how to ensure rigor in qualitative work (Gibbert & Ruigrok, Reference Gibbert and Ruigrok2010; Gioia, Corley, & Hamilton, Reference Gioia, Corley and Hamilton2013) and how to conduct member checking (Locke & Velamuri, Reference Locke and Velamuri2009). Finally, some articles discussed tensions between qualitative and quantitative traditions, such as in the review process (Pratt, Reference Pratt2008).
Thus, we can conclude that interest in qualitative research is increasing, at least in some parts of the academy. However, it is also interesting to know that the only two journals in our list with “psychology” in their title, PPsych and JAP, both had acceptance rates of qualitative research below 1%. Because of this finding, we compared the publication rates for qualitative research across types of journals, broadly defined as management and I-O psychology. As a reviewer suggested, it may be that as psychologists, the I-O community (and its journals) are more likely to use quantitative methods (though this would not explain why there seems to be growth in qualitative methods in other domains in psychology, as noted in our introduction). Although it is true that journals like JAP and PPsych had lower rates of qualitative and mixed-methods publications than ASQ and OS (the former two being more closely aligned with I-O psychology and the latter two with management), it is also true that journals catering to both groups (e.g., JOM) had equally low publication rates as JAP and PPsych.
Ultimately, however, our review cannot fully assess why there are such low publication rates in I-O journals. However, one straightforward argument for low acceptance rates is that they represent the low base rate of submissions. In other words, if authors are simply not submitting qualitative work to the journals, is it fair to blame the journals and the reviewing process? Indeed, the submission rate of qualitative work is rather low for some of these journals (Hemingway, Reference Hemingway2001). Even in journals where there is a fair amount of qualitative research published, most submissions are quantitative. The first author contacted the current and past AMJ editors, and both suggest that submission rates for qualitative papers is about 13% of the total manuscripts submitted. However, the submission base rate may be symptomatic of greater problems. If few qualitative papers are submitted to our top-tier journals, is it because few researchers are using qualitative methods? This begs the question of why. Like others (Eby et al., Reference Eby, Hurst, Butts, Lance and Vandenberg2009), we believe that qualitative methods and those who employ them are plagued by several myths.
Myths: Why Aren't Qualitative Methods More Prominent in I-O Psychology Journals?
We believe that the two prominent barriers to publishing qualitative research, especially in psychology-oriented journals, are a lack of training in qualitative methods (Hemingway, Reference Hemingway2001) and common misunderstandings or “myths” related to what qualitative methods are and what they are not.
The latest version of the Guidelines for Education and Training in Industrial–Organizational Psychology includes the following about research methods:
The specific areas encompassed by research methods include the scientific method (with attention to issues in the philosophy of science); inductive and deductive reasoning, the generation and articulation of problem statements, research questions, and hypotheses; literature review and critique, the nature and definition of constructs; study designs (experimental, quasi-experimental, and non-experimental); and psychometrics. At an operational level, research methods includes, but is not limited to, the manipulation of variables (in experimental research), the concepts underlying and methods used for the assessment of the reliability and validity of measures, the administration of various measures (questionnaires, interviews, observations of behavior, projective measures, etc.), the use of various sampling procedures (probability- and nonprobability-based) especially as applied to survey research, the conduct of research in the laboratory and the field with various strategies (experiment, survey, simulation, case study, etc.), the use of statistical methods to establish relationships between variables, causality, and the formulation of research-based conclusions. Specific knowledge about relative strengths and weaknesses of different research strategies, an understanding of qualitative research methods, and an appreciation of the benefits of alternative strategies must be developed. (Society for Industrial and Organizational Psychology [SIOP], 2016)
Although not ignoring qualitative approaches completely, the emphasis is clearly on quantitative methods. We were pleased that the phrase “a tolerant appreciation of the benefits of alternative strategies must be developed” (emphasis added) found in the 1999 version of the Guidelines had been modified (SIOP, 1999). Still, it should perhaps come as no surprise that a perusal of the websites, course catalogs, and graduate handbooks of top I-O psychology programs in North America revealed that none required a stand-alone qualitative methods course (the methodology and the list of schools appear in Appendix B). Although it is possible that qualitative methods are covered in one or more sessions of a specific research methods course, it is impossible to master even one qualitative approach in 3 to 6 hours of seminars and a handful of assigned readings. It is also possible that students can take a seminar on qualitative methods in a different department on campus, but our search did not find clear endorsements of qualitative methods courses offered by other departments on the programs’ websites and their attendant documents.
It is interesting to note that two well-known interdisciplinary doctoral programs that include an I-O psychology component did incorporate qualitative methods training in their curriculum (see Appendix B for course titles). Furthermore, these programs listed qualitative methods in the list of skills students would acquire in the program and in their program values. It may be that the interdisciplinarity inherent in these programs fosters an appreciation for a variety of methodological approaches.
In the first author's experience as an editor for qualitative research at three journals, two of which are listed in Table 1, as well as a writer and presenter of qualitative research, several misconceptions of qualitative research exist:
1. It is not rigorous. There are various spins on this particular critique. For example, there are some who believe that using qualitative methods is the same thing as simply gathering qualitative data, such as interview data. Some even equate qualitative research with just “talking with people and getting quotes.” At the University of Illinois, we used to refer to this type of research as “small ‘q’” research, where q stood for “quappy” or bad qualitative research. This type of research was often seen in people claiming to do “mixed-methods” research but equating the qualitative part of their methodology with “talking with some leaders and organizational members” before doing a survey or other quantitative assessment. Put another way, gathering qualitative data is not the same as utilizing qualitative methods.
In this vein, the qualitative researcher is also viewed as being something like a reporter; however, like all research, qualitative research is about protecting the researched, not exposing them as a reporter might. In addition, as we discuss below, the methods in qualitative research are used to build and extend theory, not simply tell a story. This myth about the lack of rigor is also related to the myth that qualitative research is “easy” or at least “easier” than utilizing other methodologies. That would likely be the case if it just involved talking to people and taking the choicest quotations. But when done rigorously, design, data collection, and analysis are very difficult and time-consuming—qualities true of any methodology conducted rigorously. What each of these critiques miss, however, is what might be called the Brutus misperception. Just as the fault lies not in the stars but in ourselves, rigor lies not in the method but in the methodologist. All research, qualitative and quantitative, can be rigorous or not. It depends on the researcher.
But what makes for rigorous qualitative research? This is an issue that could be a paper in itself. In general, it should be noted that there is not an agreed on standard for evaluating qualitative research (see Pratt, Reference Pratt2008). Indeed, there are at least two competing “camps” in this regard. In one camp, there is the belief that there should be qualitative analogs to criteria used to evaluate quantitative research. Yin (Reference Yin2003), for example, suggests that construct validity in case research can be approximated by triangulating via multiple sources of evidence and having key informants review the draft of one's paper. Reliability can be strengthened by using protocols (e.g., interview or observation) and developing a case study database. Finally, he outlines a replication logic for multiple case studies that can strengthen external validity. Others suggest that given that quantitative and qualitative research are sometimes based on different ontological and epistemological assumptions (see Lincoln & Guba, Reference Lincoln, Guba, Denzin and Lincoln2000), trying to generate analogous criteria is perhaps not possible. Instead, researchers have looked at how qualitative research, such as ethnographies, “convinces” (or does not convince) readers regarding the credibility of their findings. Golden-Biddle and Locke (Reference Golden-Biddle and Locke2007), for example, suggest three dimensions for enhancing such credibility: “authenticity” (was the author true to the experience he or she had in the field?), “plausibility” (does the contribution of the article make sense), and “criticality” (does the research make you rethink taken-for-granted assumptions?). These different criteria mirror the different ontologies and epistemologies of those who use qualitative research. Indeed, although some qualitative researchers are highly interpretivist, others tend more toward positivism. In sum, qualitative research, like all research, can be rigorously done. What counts as “rigorous,” however, varies, at least in part, by one's ontological and epistemological assumptions.
2. It is just pre-science. Qualitative methods have a long history in psychology, and they have been utilized by such luminaries as William James, whom some regard as the “father” of American psychology. Perhaps because of its use in the distant past, it is viewed as something that has preceded, and perhaps must only precede, more positivistic hypodeductive designs. Put another way, there are those who think that qualitative research should always end in hypotheses that will later be tested quantitatively. However, as noted by Locke and Golden-Biddle (Reference Locke, Golden-Biddle and Goldberg2002), people with different paradigms will approach qualitative research differently. Thus, researchers with a “modernist” paradigm might agree that qualitative research is ultimately useful as part of a hypodeductive method. However, researchers with other paradigms (or we might argue, different ontologies and epistemologies), such as an interpretivist paradigm, would not.
Depending on your perspective, there is also disagreement about what topics for which qualitative methods are most appropriate. Indeed, both authors have been told that some research areas are too “mature” to benefit from qualitative inquiry. In other words, what would a qualitative study uncover that is not already known? Hemingway (Reference Hemingway2001) echoes this sentiment. On the basis of interviews of editors of journals commonly read by I-O psychologists, most of her informants noted that
qualitative techniques [are] useful in early stages of scientific investigation (e.g., obtaining a general understanding of a topic, identifying variables or constructs, developing theory, and generating testable hypotheses) rather than the hypothesis-testing phase. Editors generally felt that more objective and precise methods (typically quantitative) of investigation should take over after the initial qualitative work has been done. (Hemingway, Reference Hemingway2001, p. 47)
But even in a mature area like team research, qualitative investigations can still add value and be published in our leading I-O psychology journals (e.g., Behfar, Peterson, Mannix, & Trochim, Reference Behfar, Peterson, Mannix and Trochim2008).
A variant on this myth that qualitative research is pre-scientific (or is perhaps only useful early in the scientific research process) is that qualitative research is not empirical. This may occur when people conflate the empirical nature of research with positivistic, quantitative research. However, empirical means “originating in or based on observation or experience” (http://www.merriam-webster.com/dictionary/empirical). Thus, in many ways, methods like ethnography, which involves extended participant observation, are inherently empirical as they provide both an observational and an experiential basis for learning.
Similarly, some may also equate empirical with objective and replicable. But it is not clear that these latter criteria clearly differentiate qualitative and quantitative work either. As noted by Hemingway (Reference Hemingway2001) “quantitative research is not synonymous with objectivity and qualitative research with subjectivity. Both research approaches (quantitative and qualitative) have a degree of subjectivity because both are influenced by human decisions.” To illustrate, the influence of human decisions in quantitative research is clearly demonstrated in the many judgment calls required of meta-analysts (Aguinis, Dalton, Bosco, Pierce, & Dalton, Reference Aguinis, Dalton, Bosco, Pierce and Dalton2011). Moreover, both qualitative (Yin, Reference Yin2003) and quantitative studies (Carey, Reference Carey2015) face challenges regarding replicability.
Issues of (pre-)science, empiricism, and objectivity raise the issue of how qualitative research relates to the scientific method. As we indicate above, whether and how qualitative methods relate to the scientific method depend, in part, on the ontology and epistemology—or the paradigm (Locke & Golden-Biddle, Reference Locke, Golden-Biddle and Goldberg2002)—of the researcher. For example, Eby et al. (Reference Eby, Hurst, Butts, Lance and Vandenberg2009) indicate that it is erroneous to believe that qualitative research is never based on the scientific method. Indeed, qualitative research involves observation and description (of data) used to induce meaning and generate theories. The same is the case in quantitative research. Although the focus on hypothesis formulation and testing is not as prevalent in qualitative research as it is in quantitative research, Eby and colleagues still found evidence of it in the articles they reviewed. That said, qualitative researchers from a more interpretivist or social constructionist perspective may not view their research as following the scientific method. Thus, qualitative methods sometimes will and sometimes will not utilize the scientific method.
Taking a step back, it is important not to tie specific methodologies exclusively to the scientific method. As Abraham Kaplan (Reference Kaplan1964) admonished in his classic, The Conduct of Inquiry: Methodology for Behavioral Science,
It is less important to draw a fine line between what is “scientific” and what is not than to cherish every opportunity for scientific growth. There is no need for behavioral science to tighten its immigration laws against subversive aliens. Scientific institutions are not so easily overthrown. The more realistic danger is that some preferred set of techniques will come to be identified with scientific method as such. (p. 28)
3. You cannot learn from such a small n/It's not generalizable. In terms of statistical generalizability, the latter is certainly true. However, qualitative research can use naturalistic generalizability: the ability to make comparisons to like others (Stake, Reference Stake, Denzin and Lincoln2000). As a student of the first author once noted, the essence of this type of generalizability can be found in the Chinese proverb, “The sparrow is small but all its vital organs are there.” In other words, if you really understand one type of organization, you might be able to make some assertions about similar types of organizations, just as learning about a maple may give you some insights into an oak. Inductive qualitative research can also lead to analytic generalizability where researchers generalize their findings to theories, and theories in turn may be applied beyond a specific context (Yin, Reference Yin2003).
4. Researchers go in “blank slate”/they only find what they are looking for. Qualitative research seems to be hammered by two competing myths. On the one hand, they are depicted as going in with a blank slate and as not knowing the literature. As a result, they end up “recreating the wheel.” On the other hand, others believe that qualitative researchers go in with so many preconceptions that that they simply “find what they are looking for.” With regard to the former, qualitative researchers do not go in “blank slate.” Heading into the field with no knowledge of the literature does not make sense for at least two reasons. First, without at least a theoretical arena in which you are interested, it would be difficult to know what to observe and what general types of questions to ask. Second, if you want to build or elaborate theory, you need to know what theories exist, as well as those areas where theoretical knowledge is limited. Thus, qualitative researchers often have to read a lot of research before entering the field (and after leaving the field) in order to better understand what is known and what is not.
With regard to the latter, rigorous qualitative research is not an exercise in confirming one's own implicit hypotheses (i.e., finding what you are looking for). As noted, the purpose of inductive qualitative research is to better understand the mindsets of the individuals you are studying. It is for this reason that qualitative researchers are often “surprised” by what they find and often shift their research focus to better understand these new insights. There are a variety of techniques that a qualitative researcher can use to help overcome “finding what you are looking for.” At a basic level, a researcher needs to ask nonleading, open-ended questions that allow informants to teach him or her more about how they see the world. Similarly, member checking, which involves verifying your findings with those whom you study, is also helpful. As one of the first author's dissertation committee members, Martha Feldman, noted, another practice a qualitative researcher can adopt is conducting throughout one's study a mental exercise whereby you consciously think of what you would need to see to disconfirm what you think you know or believe.
Moving Forward: Integrating Qualitative Research Into I-O Psychology
To this point, the nature of our argument has been threefold. First, we have illustrated the lack of qualitative research in I-O psychology journals, despite an increasing interest in other areas of psychology and in a related discipline, OBHR. Second, we have discussed why important research questions in I-O psychology would benefit from the use of qualitative methods. Third, we have pointed out some myths that may be hindering the use of qualitative methods in addressing these questions. We end our discussion by examining what could be done to better integrate qualitative methods into I-O psychology.
To begin this discussion, we reiterate and extend a stream of arguments we have already presented. Although qualitative research appears to have found some acceptance in top-tier organizational behavior journals, such as AMJ, ASQ, and OS—as well as HRM—its presence in I-O psychology and I-O-OBHR journals remains exceedingly low. There are a few reasons for this. As we have noted, there is a base-rate issue: Journals cannot publish qualitative papers if people do not submit them in the first place. However, individuals are not likely to submit to journals that have few scholars on the editor team, or the editorial board, who do not conduct qualitative research themselves. It is likely not an accident that AMJ, ASQ, and OS each have editors who specialize in qualitative research and have members of the editorial boards who are well-known qualitative scholars. Continuing this chain of logic, journals cannot have qualitative researchers on their board or editor teams if they are not well versed in the methodology, which is difficult to do if researchers are not trained in it. Further to the training argument, there is the mindset argument. Are doctoral students and tenure-track faculty members discouraged from tackling qualitative projects? Are these projects perceived as being more risky than their quantitative counterparts (Hemingway, Reference Hemingway2001)? Sometimes it feels like this is the generally accepted mindset. The first author, for example, was originally discouraged from doing a qualitative dissertation and then was encouraged to supplement his qualitative research with an experimental study. From the perspective of a youngish (albeit tenured) scholar squarely trained in a quantitative tradition, the second author can attest at how daunting learning qualitative techniques can be on one's own. Whereas specific quantitative approaches (e.g., time series analyses, cluster analysis, hierarchical linear models) appear well defined to the uninitiated scholar, qualitative approaches can appear more abstract. Workshops and written resources seem to abound for the former but are less prevalent for the latter. Trusted colleagues are within reach for the former but unlikely to be in our network for the latter. As a result, it is often easier to stay in the comfort zone created by one's academic training than it is to venture into unknown territory. Although, as the second author can attest, plunging into qualitative methods can be rewarding and exciting, we do understand the concerns. However, to continue to discourage qualitative research because there are not enough editors or reviewers on journals, or because there are high start-up costs, could easily mean that change might never happen. Indeed, the barriers we note build off each other (e.g., little doctoral training means high start-up costs, and likely fewer qualified people for editorial boards), so it is difficult to see how change might happen without intervention of some kind.
If we are to move forward, there are no quick fixes. Below, however, are some ideas and resources that can be implemented in a relatively short time horizon.
1. More representation of qualitative researchers as editors and editorial board members of major I-O psychology journals. As noted, this trend is already happening in organizational behavior. Qualitative I-O psychologists are out there, and putting qualitative editors and editorial board members in I-O psychology journals would go a long way to communicate that these methods are indeed welcome. Furthermore, because some editors who usually deal with quantitatively based papers may not be comfortable with adjudicating qualitative submissions (Hemingway, Reference Hemingway2001), instituting a qualitative researcher as an editor or associate editor—or even temporarily as a guest editor for specific manuscripts—will help ensure that those submissions are thoroughly and fairly evaluated.
2. Intensive training classes and professional development workshops. Although it would be difficult to immediately implement qualitative research courses for doctoral students for a variety of reasons (e.g., having the personnel, financial, and other resources needed to implement such a course), there are some avenues for researchers. For example, the Center for the Advancement of Research Methods and Analysis (CARMA) has intensive 2½ day workshops on a variety of qualitative topics, including ethnography, grounded theory, and qualitative interviewing. CARMA also has a library of webcasts, which is freely accessible to faculty and students of partner institutions. There are currently 10 webcasts focused on qualitative methods.
A less intensive option would be to encourage more multihour professional development workshops at our major conferences. Although qualitative methods workshops have been a part of the Academy of Management for many years, there have been only a handful of qualitative methods workshops at SIOP. Indeed, perusing a decade of conference programming reveals that only four Friday seminars, preconference workshops, or master's tutorials were devoted to qualitative methods. Although these are perhaps less beneficial than the multiday training courses, they can provide some basic knowledge about qualitative methods, as well as introduce participants to resources for further education and instruction. For example, such workshops can recommend books on ethnography (e.g., Spradley, Reference Spradley1979), grounded theory (e.g., Charmaz, Reference Charmaz2006; Glaser & Strauss, Reference Glaser and Strauss1967; Strauss & Corbin, Reference Strauss and Corbin1998), and case studies (Stake, Reference Stake, Denzin and Lincoln2000; Yin, Reference Yin2003). These workshops can go a long way to help attendees increase their appreciation of the use and utility of qualitative methods in our field. General psychology conferences such as those held by the Association for Psychological Science also offer workshops, and qualitative methods are occasionally covered therein.
3. Resource sharing across universities. In areas where multiple colleges and universities exist, there may be opportunities to share resources. For example, students at Boston College, Boston University, Harvard Business School, and Massachusetts Institute of Technology will often attend qualitative research method classes at neighboring schools. Similarly, these four schools have joined others in the area to host a semiannual Field Researchers Conference where local doctoral students and faculty meet to discuss issues such as framing qualitative research or publishing qualitative research (see http://www.bostonfieldresearchers.org/). Four universities in Montréal, Canada (McGill University, Concordia University, the École des Hautes Études Commerciales, and the Université du Québec à Montréal), have also joined forces to offer a joint PhD in Administration. Qualitative methods courses are offered yearly in English and French. Furthermore, their general research methods courses discuss both quantitative and qualitative approaches.
4. Special issue of journals. We encourage editors of our leading journals to call for special issues featuring qualitative or mixed methods. The resulting publications would then serve as exemplars to follow for future submissions. In addition to simply doing a special issue featuring only qualitative papers, another approach could be to publish pairs of papers focusing on the same substantive topic but using different approaches. This pairing would illustrate the benefits of using qualitative approaches in our field as well as show the complementarity of qualitative and quantitative methods. Guest editors can be brought on board for this initiative.
5. Journal guidelines. AMJ has published several editorials on qualitative methods, all of which are freely available in the authors’ resources section of their website (http://aom.org/Publications/AMJ/Author-Resources.aspx). There are currently eight such editorials, ranging in topics from grounded theory (Suddaby, Reference Suddaby2006), to case research (Eisenhardt & Graebner, Reference Eisenhardt and Graebner2007; Siggelkow, Reference Siggelkow2007), richness (Weick, Reference Weick2007), and writing suggestions (Pratt, Reference Pratt2009). The Journal of Occupational and Organizational Psychology (“Qualitative Guidelines,” n.d.) has also published specific guidelines for qualitative work in their authors’ guidelines to aid in the writing and reviewing of qualitative submissions. Although guidelines can be helpful, as long as they do not become edicts, we prefer the AMJ editorial model, which allows for richer and more nuanced discussion of the various types of qualitative methods. As we have noted, there are different kinds of qualitative methods. Hence, one needs to be careful to not impose “one size fits all” guidelines. In general, we encourage other journals to provide resources for qualitative researchers, in addition to relaxing page limits for qualitative work as done by AMJ and ASQ. The latter allows authors whose work is based on qualitative data to show their data through thick descriptions and quotations, both of which aid in establishing the credibility of their results.
Concluding Remarks
Our goal with this focal article is to generate a discussion on the place of qualitative methods in the field of I-O psychology. To be sure, we do not argue that researchers in our field use qualitative methods for the sake of doing qualitative research. Instead, it is our contention that if researchers are not aware of the possibilities afforded by qualitative methods, they are not in a position of considering a qualitative design when determining how to best answer their research questions. As always, the research question determines the choice of methods.
With this in mind, we have argued that a greater appreciation of qualitative methods and its resulting research can help our field advance in new directions by expanding or revising extant theories and generating new theories. We hope to have shed light on common myths surrounding qualitative methods and on the many advantages associated with the use of these methods. We hope to stimulate constructive discussion and debates on the relative merits of qualitative methods in our field. Finally, we hope to spur changes that will facilitate greater use of qualitative research. In particular, we hope to have convinced doctoral program directors to incorporate more opportunities for graduate students to learn qualitative methods throughout their training. In this sense, we echo Aguinis, Pierce, Bosco, and Muslin's (Reference Aguinis, Pierce, Bosco and Muslin2009) call for more doctoral education in qualitative methods for students in our field. With the recent revisions of the SIOP Guidelines for Education and Training, now is the time to broaden our methodological horizons.
Appendix A
We based our journal list on four criteria: presence in the Eby et al.’s (2009) Applied Psychology and Management lists, impact factor, presence on the Financial Times 45 list, and presence among journals ranked highly by Zickar and Highhouse (Reference Zickar and Highhouse2001). Like Eby and colleagues, we discarded Academy of Management Review, given its scope. We also did not include Management Information Systems Quarterly as it is not a widely read journal in I-O psychology.
We approached the literature search in two ways. We first conducted a search in PsycINFO for each journal using the “additional limits” function and searching for “1600 qualitative study.” We then cross-checked the results of this search with a keyword-driven search strategy. Like Eby and colleagues (2009), we conducted a Boolean search using a broad key word (qualitative) and several narrower terms denoting research philosophies and data analysis techniques (e.g., ethnography, grounded theory, content analysis).
Appendix B
We developed our list of top I-O psychology doctoral programs in North America by comparing several sources. Because there are many ways to determine the rankings of top doctoral programs, we elected to peruse the most commonly used lists in our field. First, we looked at the current version of the U.S. News & World Report program ranking for I-O psychology, which listed four schools: Michigan State University and the University of Minnesota (tied for first place), Bowling Green State University, and the University of South Florida (“Industrial and Organizational Psychology,” 2013). We also looked at Tables 1, 2, and 3 in Gibby, Reeve, Grauer, Mohr, and Zickar (Reference Gibby, Reeve, Grauer, Mohr and Zickar2002). This article also reported the 2001 U.S. News & World Report rankings list in its Table 1. Furthermore, we took into consideration the data reported in Table 1 by Oliver, Blair, Gorman, and Woehr (Reference Oliver, Blair, Gorman and Woehr2005) and the data reported in Tables 1, 2, 3, and 5 of Beiler, Zimmerman, Doerr, and Clark (Reference Beiler, Zimmerman, Doerr and Clark2014). We perused the website of each program that was listed at least once in the top 10 positions of these reports or articles. These programs are listed below.
For each program, we located the main page for its I-O psychology doctoral program. We then searched the course listings, published program structure, course catalog, and graduate student handbook for the sequence of research methods and analysis courses. Although not all programs contained each of these documents, we were able to find sufficient information on the course sequence required of doctoral students for each program. We found ample information on quantitative and statistics requirements. We believe it is reasonable to assume that any qualitative methods requirements would have been listed in the methods/analysis sequence.
Three additional doctoral programs were listed in the ranking articles we looked at. These are not typical I-O psychology programs. Indeed, these three programs stand out as being interdisciplinary by design. These are the University of Michigan (Personality and Social Contexts), University of North Carolina at Charlotte (UNCC; Organizational Science), and Teacher's College, Columbia University (Social–Organizational Psychology). The University of Michigan program did not appear to explicitly require any qualitative training of its students. However, we were pleased to see that the other two had a healthy representation of qualitative methods in their mix of research methods courses.
The Organizational Science program at UNCC lists two qualitative methods courses (Qualitative Research Methods and Advanced Qualitative Data Analysis). Furthermore, the program values listed on the website explicitly refer to qualitative research (UNCC Organizational Science, n.d.). The Social–Organizational Psychology program at Columbia University lists three courses in qualitative methods on the main department website (Qualitative Research Methods in Organizations: Data Collection, Qualitative Research Methods in Organizations: Data Analysis Design, and Methods of Case Study and Analysis). Furthermore, the website explicitly lists qualitative research methods as a skill students will acquire throughout their doctoral program.
