I agree with and expand on Pratt and Bonaccio's (Reference Pratt and Bonaccio2016) view that in order to facilitate the use of qualitative research in our field, we need more guidance for industrial–organizational (I-O) researchers in terms of training, best practice articles, and criteria for evaluating research quality. Fortunately, the interest in and openness to qualitative research is slowly but steadily growing in our field, something that is reflected in small but increasing numbers of publications and best practice articles (Bluhm, Harman, Lee, & Mitchell, Reference Bluhm, Harman, Lee and Mitchell2011; Smith, Madden, & Ashmos Plowman, Reference Smith, Madden, Ashmos Plowman, Lance and Vanderberg2015). To give an example, the Journal of Applied Psychology (JAP), a journal not well known for publishing qualitative research, has recently published a qualitative article written by my colleagues and me (Wilhelmy, Kleinmann, König, Melchers, & Truxillo, Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016).
Pratt and Bonaccio propose that one way to further facilitate qualitative research could be to provide specific journal guidelines on qualitative research (as has been done by the Journal of Occupational and Organizational Psychology; JOOP). My focus in this commentary is to elaborate on this proposition and to point out several important risks and opportunities that journals will have to think through if they consider providing guidelines on qualitative research. I believe that the development and the application of such journal guidelines are balancing acts, but they are well worth tackling by our field.
Risks of Developing Journal Guidelines for Qualitative Research
As Pratt and Bonaccio point out, a risk of providing guidelines is that qualitative manuscripts are evaluated on the basis of inadequate criteria that discourage qualitative research. I agree, but I believe that it is central to differentiate between two different kinds of criteria that might discourage qualitative research: (a) criteria that discourage qualitative research because they contradict the fundamentals of qualitative research and are thus destructive and inadequate versus (b) criteria that may seem discouraging because they are hard to accomplish but are in fact constructive and valuable for supporting qualitative research in our field.
Avoiding Criteria That Contradict the Fundamentals of Qualitative Research
An important first step is to acknowledge the unique features of qualitative research. When teaching qualitative research methods—whether it is to undergrads, graduate students, or faculty members—I like to illustrate the deductive, bottom-up approach of qualitative research by comparing it with scuba diving: When you scuba dive, you are still on the same planet, and you still breath air, but you find yourself in a world that works in a different way, that has its own principles. You actively dive deep down to explore the phenomenon from up close. You go where it seems promising to gain new and illuminative insights. You let yourself be surprised by what you happen to discover. If you lack knowledge and experience, it is a scary and risky endeavor.
This metaphor stresses the core characteristics of qualitative research that need to be considered when thinking about journal guidelines, for example, subjectivity. Regarding subjectivity, Pratt and Bonaccio (p. 705) cite Hemingway's (Reference Hemingway2001) statement that “quantitative research is not synonymous with objectivity and qualitative research with subjectivity. Both approaches (quantitative and qualitative) have a degree of subjectivity because both are influenced by human decisions.” I agree, but it is important to emphasize that in qualitative research, subjectivity is not an inevitable bias or source of error but is in fact part of its nature. Qualitative research is—and should be—subjective.
It is important to point out that there are at least two ways in which qualitative research is characterized by subjectivity, and both should be acknowledged by journal guidelines. First, as also outlined by Pratt and Bonaccio, qualitative research is about understanding how individuals (i.e., study informants) make sense of the phenomenon under investigation. Second, in many qualitative approaches, the researcher's subjective views and insights are considered to be valuable. The researcher is actively involved and engages in interpretations.
The subjectivity of qualitative research is often contrasted with positivist criteria, such as objectivity and reliability—traditionally used to assess scientific rigor—which require that the researcher remains detached from the study participants and the phenomenon under investigation (Willig, Reference Willig2013). If one demands objectivity and reliability in their strict sense, qualitative research cannot deliver because objectivity and reliability are largely incompatible with its bottom-up approach (Cassell & Symon, Reference Cassell and Symon2011). In other words, when applying positivist criteria as rules of the game, qualitative research is inevitably faulted.
In contrast, appropriate criteria for evaluating qualitative research can encourage and reward subjectivity at its best and most valuable, as outlined above. For example, the journal guidelines used by JOOP are exemplary in the way that they consider the subjectivity of the informants: “the subjective understanding or sense-making process of individuals in relation to their work situation” (“Qualitative Guidelines,” n.d.). However, the JOOP guidelines do not directly refer to researchers’ subjectivity: It is only noted that the researchers’ subjective views and influences should be reflected and discussed, which is called reflectivity.
Promoting Criteria That Set High Standards for Qualitative Research
In addition, there are criteria for evaluating qualitative research that I would describe as useful for the progress of our field but that are hard to achieve because they may seem contradictory to the fundamentals of qualitative research. For example, in JOOP’s qualitative research guidelines, there are keywords like “rationale,” “justified,” and “explanation” that ask for a well thought out, systematic approach and report of the study. We want to see qualitative papers published that are reproducible and straightforward so that future research can build on the work presented (Bluhm et al., Reference Bluhm, Harman, Lee and Mitchell2011); however, ironically, good qualitative research is characterized by also being creative and adaptive. For example, the original focus of the study can be narrowed or extended if new insights are gained as the study proceeds, and new data can be collected if it helps to better answer the research question (e.g., Wilhelmy et al., Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016). As Pratt and Bonaccio describe it, “qualitative researchers are often ‘surprised’ by what they find and often shift their research focus to better understand these new insights” (p. 707). Thus, instead of a straight, predictable path, qualitative research oftentimes resembles a twisting and winding trail full of unpredictable detours.
Thus, is qualitative research only perceived as being risky because it is hard to publish (Eby, Hurst, & Butts, Reference Eby, Hurst, Butts, Lance and Vandenberg2009) or also because, when conducting qualitative research, one is faced with this seemingly irresolvable trade-off between reproducibility and creativity? In my opinion, this challenge stresses the importance of training I-O researchers in qualitative research because it enables them to understand and address these complex, nuanced criteria for evaluating qualitative research. For example, one trick is to use a series of memos throughout the research process to constantly document what is done and changed, and for what reason (e.g., Wilhelmy et al., Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016). This documentation helps to provide rationales about the context and consequences of any adaptations and can actually help to realize that the research process needs to be adapted because, through the documentation routines, the researcher becomes more sensitive to the nuances of phenomenon under study.
Promoting Criteria That Stimulate Diversity in Qualitative Research
Another risk of journal guidelines for qualitative research is to endanger the diversity in qualitative research methods (cf. Bansal & Corley, Reference Bansal and Corley2011). In fact, approaches to qualitative research are so diverse that it is disputable whether we should talk about qualitative methods in general terms at all (Willig, Reference Willig2013). Each qualitative approach (e.g., grounded theory, ethnography, action research) reflects different epistemological roots and thus has its own definition of rigor. In addition, qualitative approaches can often be combined with different qualitative techniques for data collection (e.g., interviews, focus groups, observation) and data analyses (e.g., coding, excerption). For this reason, guidelines may focus on those features that all qualitative approaches have in common. For example, many would agree that no matter which specific qualitative approach is chosen, there are three questions that should ideally all be answered with a “yes”: (a) Is the qualitative approach chosen appropriate to answer the research question(s)? (b) Is the qualitative approach applied in a creative and flexible way so as to best answer the research question(s)? (c) Is the qualitative approach applied in a stringent way?
In addition, it may be good to specify whether criteria for research rigor only apply to certain research approaches. For instance, in the JOOP guidelines, specifications such as “in some qualitative studies, where appropriate” can be found (“Qualitative Guidelines,” n.d.). Furthermore, I agree with Pratt and Bonaccio who state that “Guidelines can be helpful, as long as they do not become edicts” (p. 710). A good strategy can be to state a criterion for research rigor but leave its implementation open to the researcher. In this vein, the JOOP guidelines state, “Epistemological integrity: Does the paper take a consistent approach towards epistemology, ontology, and methods?” without determining how this has to be achieved (“Qualitative Guidelines,” n.d.). We need to leave enough freedom and flexibility for different frameworks of qualitative research and styles of reporting results. For example, when applying a grounded theory approach, it is important to make it transparent and traceable as to how the specific methodology of the study enhances the study's rigor in the sense of that particular grounded theory's approach (e.g., Wilhelmy et al., Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016).
Opportunities in Developing Journal Guidelines for Qualitative Research
An important opportunity of journal guidelines is their potential to fight against persisting false beliefs regarding qualitative research. Despite an ongoing debate around the value of criteria for assessing qualitative research (Cassell & Symon, Reference Cassell and Symon2011), different collections of quality criteria have been suggested, but they often do not get the appreciation that they deserve (e.g., Elliott, Fischer, & Rennie, Reference Elliott, Fischer and Rennie1999; Henwood & Pidgeon, Reference Henwood and Pidgeon1992). Providing the essence of these criteria in the form of journal guidelines would make important characteristics of qualitative research more visible to authors, reviewers, and readers and could thus diminish the tenacious myths described by Pratt and Bonaccio.
Another advantage in providing journal guidelines for qualitative research would be to encourage researchers to use qualitative research methodologies more frequently because clear guidance is provided. As reported by Cassell and Symon (Reference Cassell and Symon2011), many I-O researchers perceive qualitative approaches as dangerous and risky and believe that it is “best to avoid it when the stakes are high, for example, with clients or dissertations” (p. 639). Providing clear information on what is expected by a certain journal should enhance the chances of the work being published, decrease the risk involved, and thus make qualitative approaches less daunting. Put differently, journal guidelines could serve as “guardrails” that make one more likely to safely reach the destination in terms of getting one's qualitative work published while ideally leaving enough space to be creative and flexible during the research process.
In addition, providing journal guidelines could help reviewers, who may have relatively little training in or exposure to qualitative methods, assess the quality and acceptability of qualitative research. As Pratt and Bonaccio mention, there are often not enough reviewers available who are able to evaluate qualitative research, which is a problem for journal editors who are trying to have qualitative studies reviewed. Providing guidelines could address this short supply by broadening the circle of reviewers beyond those who have extensive experience with qualitative research.
In summary, the opportunities that journal guidelines for qualitative research provide counterbalance the potential risks, as long as the guidelines are carefully developed, articulated, and applied. Nonetheless, I agree with Pratt and Bonaccio that journal guidelines are not the only way to facilitate the use of qualitative approaches and promote discussions in our field. In addition to training in doctoral programs and at conferences, editorials and best practice articles will remain important formats to provide ideas on how a particular qualitative research method can be put into action and reported in a rigorous way. No matter what format or model is used, I am convinced that the constructive discussions arising about qualitative research will move our field forward. Reverting once more to the metaphor of scuba diving, not every I-O researcher needs to become an enthusiastic scuba diver, but it would be ideal if everyone had a basic diver's license or, at least, gave snorkeling a try. If we want to know our range of options for understanding organizational phenomena, and if we want to be able to effectively evaluate and critique the approaches taken by other researchers, we need to be familiar with both the quantitative and the qualitative worlds of research.
I agree with and expand on Pratt and Bonaccio's (Reference Pratt and Bonaccio2016) view that in order to facilitate the use of qualitative research in our field, we need more guidance for industrial–organizational (I-O) researchers in terms of training, best practice articles, and criteria for evaluating research quality. Fortunately, the interest in and openness to qualitative research is slowly but steadily growing in our field, something that is reflected in small but increasing numbers of publications and best practice articles (Bluhm, Harman, Lee, & Mitchell, Reference Bluhm, Harman, Lee and Mitchell2011; Smith, Madden, & Ashmos Plowman, Reference Smith, Madden, Ashmos Plowman, Lance and Vanderberg2015). To give an example, the Journal of Applied Psychology (JAP), a journal not well known for publishing qualitative research, has recently published a qualitative article written by my colleagues and me (Wilhelmy, Kleinmann, König, Melchers, & Truxillo, Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016).
Pratt and Bonaccio propose that one way to further facilitate qualitative research could be to provide specific journal guidelines on qualitative research (as has been done by the Journal of Occupational and Organizational Psychology; JOOP). My focus in this commentary is to elaborate on this proposition and to point out several important risks and opportunities that journals will have to think through if they consider providing guidelines on qualitative research. I believe that the development and the application of such journal guidelines are balancing acts, but they are well worth tackling by our field.
Risks of Developing Journal Guidelines for Qualitative Research
As Pratt and Bonaccio point out, a risk of providing guidelines is that qualitative manuscripts are evaluated on the basis of inadequate criteria that discourage qualitative research. I agree, but I believe that it is central to differentiate between two different kinds of criteria that might discourage qualitative research: (a) criteria that discourage qualitative research because they contradict the fundamentals of qualitative research and are thus destructive and inadequate versus (b) criteria that may seem discouraging because they are hard to accomplish but are in fact constructive and valuable for supporting qualitative research in our field.
Avoiding Criteria That Contradict the Fundamentals of Qualitative Research
An important first step is to acknowledge the unique features of qualitative research. When teaching qualitative research methods—whether it is to undergrads, graduate students, or faculty members—I like to illustrate the deductive, bottom-up approach of qualitative research by comparing it with scuba diving: When you scuba dive, you are still on the same planet, and you still breath air, but you find yourself in a world that works in a different way, that has its own principles. You actively dive deep down to explore the phenomenon from up close. You go where it seems promising to gain new and illuminative insights. You let yourself be surprised by what you happen to discover. If you lack knowledge and experience, it is a scary and risky endeavor.
This metaphor stresses the core characteristics of qualitative research that need to be considered when thinking about journal guidelines, for example, subjectivity. Regarding subjectivity, Pratt and Bonaccio (p. 705) cite Hemingway's (Reference Hemingway2001) statement that “quantitative research is not synonymous with objectivity and qualitative research with subjectivity. Both approaches (quantitative and qualitative) have a degree of subjectivity because both are influenced by human decisions.” I agree, but it is important to emphasize that in qualitative research, subjectivity is not an inevitable bias or source of error but is in fact part of its nature. Qualitative research is—and should be—subjective.
It is important to point out that there are at least two ways in which qualitative research is characterized by subjectivity, and both should be acknowledged by journal guidelines. First, as also outlined by Pratt and Bonaccio, qualitative research is about understanding how individuals (i.e., study informants) make sense of the phenomenon under investigation. Second, in many qualitative approaches, the researcher's subjective views and insights are considered to be valuable. The researcher is actively involved and engages in interpretations.
The subjectivity of qualitative research is often contrasted with positivist criteria, such as objectivity and reliability—traditionally used to assess scientific rigor—which require that the researcher remains detached from the study participants and the phenomenon under investigation (Willig, Reference Willig2013). If one demands objectivity and reliability in their strict sense, qualitative research cannot deliver because objectivity and reliability are largely incompatible with its bottom-up approach (Cassell & Symon, Reference Cassell and Symon2011). In other words, when applying positivist criteria as rules of the game, qualitative research is inevitably faulted.
In contrast, appropriate criteria for evaluating qualitative research can encourage and reward subjectivity at its best and most valuable, as outlined above. For example, the journal guidelines used by JOOP are exemplary in the way that they consider the subjectivity of the informants: “the subjective understanding or sense-making process of individuals in relation to their work situation” (“Qualitative Guidelines,” n.d.). However, the JOOP guidelines do not directly refer to researchers’ subjectivity: It is only noted that the researchers’ subjective views and influences should be reflected and discussed, which is called reflectivity.
Promoting Criteria That Set High Standards for Qualitative Research
In addition, there are criteria for evaluating qualitative research that I would describe as useful for the progress of our field but that are hard to achieve because they may seem contradictory to the fundamentals of qualitative research. For example, in JOOP’s qualitative research guidelines, there are keywords like “rationale,” “justified,” and “explanation” that ask for a well thought out, systematic approach and report of the study. We want to see qualitative papers published that are reproducible and straightforward so that future research can build on the work presented (Bluhm et al., Reference Bluhm, Harman, Lee and Mitchell2011); however, ironically, good qualitative research is characterized by also being creative and adaptive. For example, the original focus of the study can be narrowed or extended if new insights are gained as the study proceeds, and new data can be collected if it helps to better answer the research question (e.g., Wilhelmy et al., Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016). As Pratt and Bonaccio describe it, “qualitative researchers are often ‘surprised’ by what they find and often shift their research focus to better understand these new insights” (p. 707). Thus, instead of a straight, predictable path, qualitative research oftentimes resembles a twisting and winding trail full of unpredictable detours.
Thus, is qualitative research only perceived as being risky because it is hard to publish (Eby, Hurst, & Butts, Reference Eby, Hurst, Butts, Lance and Vandenberg2009) or also because, when conducting qualitative research, one is faced with this seemingly irresolvable trade-off between reproducibility and creativity? In my opinion, this challenge stresses the importance of training I-O researchers in qualitative research because it enables them to understand and address these complex, nuanced criteria for evaluating qualitative research. For example, one trick is to use a series of memos throughout the research process to constantly document what is done and changed, and for what reason (e.g., Wilhelmy et al., Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016). This documentation helps to provide rationales about the context and consequences of any adaptations and can actually help to realize that the research process needs to be adapted because, through the documentation routines, the researcher becomes more sensitive to the nuances of phenomenon under study.
Promoting Criteria That Stimulate Diversity in Qualitative Research
Another risk of journal guidelines for qualitative research is to endanger the diversity in qualitative research methods (cf. Bansal & Corley, Reference Bansal and Corley2011). In fact, approaches to qualitative research are so diverse that it is disputable whether we should talk about qualitative methods in general terms at all (Willig, Reference Willig2013). Each qualitative approach (e.g., grounded theory, ethnography, action research) reflects different epistemological roots and thus has its own definition of rigor. In addition, qualitative approaches can often be combined with different qualitative techniques for data collection (e.g., interviews, focus groups, observation) and data analyses (e.g., coding, excerption). For this reason, guidelines may focus on those features that all qualitative approaches have in common. For example, many would agree that no matter which specific qualitative approach is chosen, there are three questions that should ideally all be answered with a “yes”: (a) Is the qualitative approach chosen appropriate to answer the research question(s)? (b) Is the qualitative approach applied in a creative and flexible way so as to best answer the research question(s)? (c) Is the qualitative approach applied in a stringent way?
In addition, it may be good to specify whether criteria for research rigor only apply to certain research approaches. For instance, in the JOOP guidelines, specifications such as “in some qualitative studies, where appropriate” can be found (“Qualitative Guidelines,” n.d.). Furthermore, I agree with Pratt and Bonaccio who state that “Guidelines can be helpful, as long as they do not become edicts” (p. 710). A good strategy can be to state a criterion for research rigor but leave its implementation open to the researcher. In this vein, the JOOP guidelines state, “Epistemological integrity: Does the paper take a consistent approach towards epistemology, ontology, and methods?” without determining how this has to be achieved (“Qualitative Guidelines,” n.d.). We need to leave enough freedom and flexibility for different frameworks of qualitative research and styles of reporting results. For example, when applying a grounded theory approach, it is important to make it transparent and traceable as to how the specific methodology of the study enhances the study's rigor in the sense of that particular grounded theory's approach (e.g., Wilhelmy et al., Reference Wilhelmy, Kleinmann, König, Melchers and Truxillo2016).
Opportunities in Developing Journal Guidelines for Qualitative Research
An important opportunity of journal guidelines is their potential to fight against persisting false beliefs regarding qualitative research. Despite an ongoing debate around the value of criteria for assessing qualitative research (Cassell & Symon, Reference Cassell and Symon2011), different collections of quality criteria have been suggested, but they often do not get the appreciation that they deserve (e.g., Elliott, Fischer, & Rennie, Reference Elliott, Fischer and Rennie1999; Henwood & Pidgeon, Reference Henwood and Pidgeon1992). Providing the essence of these criteria in the form of journal guidelines would make important characteristics of qualitative research more visible to authors, reviewers, and readers and could thus diminish the tenacious myths described by Pratt and Bonaccio.
Another advantage in providing journal guidelines for qualitative research would be to encourage researchers to use qualitative research methodologies more frequently because clear guidance is provided. As reported by Cassell and Symon (Reference Cassell and Symon2011), many I-O researchers perceive qualitative approaches as dangerous and risky and believe that it is “best to avoid it when the stakes are high, for example, with clients or dissertations” (p. 639). Providing clear information on what is expected by a certain journal should enhance the chances of the work being published, decrease the risk involved, and thus make qualitative approaches less daunting. Put differently, journal guidelines could serve as “guardrails” that make one more likely to safely reach the destination in terms of getting one's qualitative work published while ideally leaving enough space to be creative and flexible during the research process.
In addition, providing journal guidelines could help reviewers, who may have relatively little training in or exposure to qualitative methods, assess the quality and acceptability of qualitative research. As Pratt and Bonaccio mention, there are often not enough reviewers available who are able to evaluate qualitative research, which is a problem for journal editors who are trying to have qualitative studies reviewed. Providing guidelines could address this short supply by broadening the circle of reviewers beyond those who have extensive experience with qualitative research.
In summary, the opportunities that journal guidelines for qualitative research provide counterbalance the potential risks, as long as the guidelines are carefully developed, articulated, and applied. Nonetheless, I agree with Pratt and Bonaccio that journal guidelines are not the only way to facilitate the use of qualitative approaches and promote discussions in our field. In addition to training in doctoral programs and at conferences, editorials and best practice articles will remain important formats to provide ideas on how a particular qualitative research method can be put into action and reported in a rigorous way. No matter what format or model is used, I am convinced that the constructive discussions arising about qualitative research will move our field forward. Reverting once more to the metaphor of scuba diving, not every I-O researcher needs to become an enthusiastic scuba diver, but it would be ideal if everyone had a basic diver's license or, at least, gave snorkeling a try. If we want to know our range of options for understanding organizational phenomena, and if we want to be able to effectively evaluate and critique the approaches taken by other researchers, we need to be familiar with both the quantitative and the qualitative worlds of research.