Introduction
Focus group research is a useful methodology as a source of core or supplementary data, but advice on using digital tools in certain stages of focus group research is relatively scarce. This article draws on experience with two projects in order to outline how researchers can harness technologies for focus group recruitment and data analysis, particularly for research focused on marginalized communities. These techniques are particularly useful in the context of virtual focus groups, which provide a potentially more affordable and inclusive alternative to in-person groups and may also help to decrease the carbon footprint of research by reducing in-person travel. The experience with COVID-19 has demonstrated not only that situations arise which necessitate virtual work but also that technologies such as Zoom can transition quickly into widespread use. Geared to new and existing focus group users, the article identifies fruitful ways to harness a wider range of technologies for focus group research while maintaining consistency with established practices.
This article is based on our combined experience in two projects: one on measuring sex and gender in survey research (Bittner and Goodyear-Grant, Reference Bittner and Goodyear-Grant2017a, Reference Bittner and Goodyear-Grant2017b; Bittner et al., Reference Bittner, Goodyear-Grant and Wallace2018) and one with the Canadian Municipal Election Study (CMES) (Wallace et al., “Immigrants and Participation,” Reference Wallace, McGregor and Tolley2019). Both projects used focus groups in mixed-methods studies of political behaviour and are particularly attuned to engaging with marginalized populations.
In this article, we argue that harnessing digital technologies is useful for focus group research, and we turn our attention in particular to the benefits it can bring to enhancing (a) recruitment and participation in focus groups and (b) the analysis of focus group data. While focus group methodologies have existed for quite some time, many scholars in the positivist tradition are unaccustomed to this type of research, and we seek to relieve a bit of the “fear of the unknown” in this article. Further, we hope that our contribution will help to better equip peer reviewers to assess focus group research.
Focus Group Methods
A focus group is “a group of individuals selected and assembled by researchers to discuss and comment on, from personal experience, the topic that is the subject of the research” (Gibbs, Reference Gibbs1997: 499). These discussions generally centre on broad, open-ended questions or prompts that inquire about participants’ attitudes, priorities and understanding of a topic (Kitzinger, Reference Kitzinger1994; Merton, Reference Merton1987). A crucial element of analytical focus is interaction, or the “social event” (Goss and Leinbach, Reference Goss and Leinbach1996), in which “collective sense is made, meanings negotiated, and identities elaborated through the process of social interaction between people” (Wilkinson, Reference Wilkinson1999: 225; see also Morgan, Reference Morgan1988; Stewart et al., Reference Stewart, Shamdasani and Rook1990; Kitzinger, Reference Kitzinger1994). As Cyr (Reference Cyr2019: 98) argues, one of the unique features of focus group research is that it generates data at three levels of analysis: “the individual, the group, and interactive levels.” A focus on the individual level entails analyzing participants’ individual interventions; a focus on the group level “centers on the conversation as a whole,” often looking at the session in broad strokes; a focus on the interactive level pays attention to “the conversational journey,” looking at exchanges, contention and deliberation (Cyr, Reference Cyr2019: 98). A research team will choose which level(s) on which to focus, depending on their research questions.
Focus groups can be used as a stand-alone methodology or in combination with other methods, and they are often used in pilot phases of research (Bloor et al., Reference Bloor, Frankland, Thomas and Robson2001). Nyumba et al. (Reference Nyumba, Wilson, Derrick and Mukherjee2018) recommend the use of focus groups within a mixed-methods strategy, especially grounded theory (Charmaz, Reference Charmaz2006; Strauss, Reference Strauss1987), content analysis (Morgan, Reference Morgan1988), discourse analysis (Potter and Wetherell, Reference Potter and Wetherell1987) and survey methods (Bloor et al., Reference Bloor, Frankland, Thomas and Robson2001). These authors argue that more nuanced knowledge is attained by a mixed approach, which is less likely to result in blind spots or missed information.
The literature offers valuable resources on focus group methodologies and data analysis. Among the most comprehensive are Cyr's (Reference Cyr2019) Focus Groups for the Social Science Researcher and Krueger and Casey's (Reference Krueger and Casey2015) Focus Groups: A Practical Guide for Applied Research. Existing focus group texts do not provide a lot of advice, however, regarding participant identification and recruitment, compared to the emphasis placed on project design, group composition, moderating focus groups, and so on. This is a missed opportunity to guide the research community, as recruitment is “one of the most difficult and time-consuming aspects of focus groups” and “the most common source of failure” (Morgan, Reference Morgan2019: 41, 54). Stewart and Shamdasani's (Reference Stewart and Shamdasani2014) recent book, for example, includes a brief discussion of using recruiting techniques such as random-digit dialling and membership lists from partnering organizations, but the authors do not elaborate much beyond this. In their chapter on virtual focus groups, they discuss recruiting from internet sources, but they focus on the benefits and drawbacks of virtual recruitment methods rather than the mechanics of them. Morgan's (Reference Morgan2019) book has chapters for “Asking Questions in Focus Groups” and “Moderating the Discussion” but not for recruitment (although important guidance on recruitment is woven into other chapters). Cyr's (Reference Cyr2019) content on recruitment is excellent, but brief compared to other sections of her text, and her use of Gamson's (Reference Gamson1992) focus group research as a case study necessarily limits the possibilities for discussing digital recruitment.
Existing literature also tends to point to traditional recruitment methods, such as outsourcing to professional firms, which is pricey (see, for example, Cyr, Reference Cyr2019: 45); partnering with relevant organizations (see, for example, Morgan, Reference Morgan2019; Stewart and Shamdasani, Reference Stewart and Shamdasani2014), which can limit the depth of the participant pool; and random-digit dialling (Stewart and Shamdasani, Reference Stewart and Shamdasani2014), which can be inefficient for recruiting small and/or hard-to-reach subpopulations. These are all useful techniques, but the repertoire of identification and recruitment strategies is expanding, and our guidance in this article responds to this evolving reality. Various experts recommend that identification and recruitment of participants should occur “where and how potential group members spend their time” and consider “what incentives are valued by the group” (Stewart et al., Reference Stewart, Shamdasani and Rook2007: 59; see also Cyr, Reference Cyr2019). The sheer number of people reached by a Facebook advertisement or Instagram post, for example, and the potential to target these to specified user demographics are huge advantages.
The mechanics of computer-assisted data analysis methods also tend to be underemphasized in the literature. For example, Cyr (Reference Cyr2019: 88) mentions equipment and software that can assist with generating focus group transcripts, such as Express Scribe or Dragon, and she focuses on principles of sound data analysis. She also has helpful content on the type of analyses focus group researchers might do, such as word counts, participant specificity, participant consistency, time spent on an issue, neglect of issues, and so on, as well as an appendix identifying some of the popular coding software (Reference Cyr2019: 119–21), but detailed instruction in computer-assisted methods is not one of the book's goals. General texts on content analysis may be helpful, especially those with a grounding in content analysis aiming to transition to computer-assisted techniques. However, particular dimensions of focus group data require specialized advice, such as capturing interaction between participants, that remain underexplored in the literature.
Morgan (Reference Morgan2019) identifies software that can help with transcription and data analysis but claims that automated content analysis is a “questionable fit” to interpretive approaches. While we do not dive deeply into epistemological or ontological questions, automated content analysis can be appropriate for any approach (positivist or interpretivist), but like any tool, it requires thoughtful deployment. Moreover, not all research questions necessitate automated analyses. There are tools that can help researchers develop insightful qualitative findings in focus group data. In the end, both positivist and interpretivist scholars need to work “across the aisle,” so to speak.Footnote 1 Quantitatively minded researchers ought to consider interpretivist approaches to strengthen their analyses and fortify their conclusions. Various scholars explore the benefits of an “ethnographic sensibility” (Boswell et al., Reference Boswell, Corbett, Dommett, Jennings, Flinders, Rhodes and Wood2019; Herzog and Zacka, Reference Herzog and Zacka2019), for example, arguing that it is useful for probing, questioning and refining our understanding of values. Rhodes and Tiernan (Reference Rhodes and Tiernan2015) provide an example of using focus groups in political ethnography, studying focus groups of former chiefs of staff to the Australian prime minister as a good way of “‘being there’ and side-stepping the problems of access and secrecy” that preclude direct observation.
While there is room for additional methodological advice on these various stages of focus group research, it is also important to note that many researchers using focus group methods have indeed embraced digital tools. One example is the use of online focus groups (for example, Bloor et al., Reference Bloor, Frankland, Thomas and Robson2001; Lobe, Reference Lobe, Barbour and Morgan2017; Moore et al., Reference Moore, McKee and McLoughlin2015; Morgan, Reference Morgan2019; Stewart and Shamdasani, Reference Stewart and Shamdasani2014; Vicsek, Reference Vicsek2016). The literature discusses the relative advantages and drawbacks of synchronous versus asynchronous focus groups, text-based versus video-based online focus groups, and more. Focus group researchers have capitalized on opportunities to expand their techniques as technology has evolved. Our goal in this article is to outline some best practices for the use of digital technologies in participant identification and recruitment, as well as data analysis, including consideration of the trade-offs involved in using these techniques.
Two Projects Provided Testing Conditions
This article's methodological insights are drawn from two separate projects. The first is a multimethod project on measuring sex and gender in survey research, combining survey data with focus group data. In Bittner et al. (Reference Bittner, Goodyear-Grant and Wallace2018), in-person focus groups were used to observe detailed conversations about sex and gender, experiences of gender in daily life and impressions of gender roles, with an eye to the development of new measures of gender for survey research. In Wallace et al. (“Perceptions of the Playing Field,” Reference Wallace, Goodyear-Grant and Bittner2019), we also explore questions in the focus groups that were specific to the experiences of women in politics, inquiring about how Canadians think gendered barriers affect women's political candidacies and careers.
A second project included a series of focus groups with the CMES (see, for example, Wallace et al., “Immigrants and Participation,” Reference Wallace, Goodyear-Grant and Bittner2019), where focus groups were used to analyze immigrants’ participation in local Canadian elections after uncovering large turnout gaps between foreign-born and native-born citizens. Wallace and colleagues found that immigrants were less likely to participate in local elections, and traditional indicators pertaining to electors’ motivations (such as political knowledge, efficacy and perceptions of voting as a civic duty) were unable to explain why. The CMES team conducted focus groups with immigrant voters to understand their lower engagement in local elections, inquiring about their understandings of local politics, the accessibility of local elections and information pertaining to them, and their feelings about the representation of their interests locally.
Harnessing Digital Technologies to Enhance Recruitment and Participation
Recruiting participants for focus groups can be challenging. Recruitment contact can take many forms, and none are “incorrect,” so to speak, as long as they allow proper engagement with the population of interest. For example, in Pollack's (Reference Pollack2003) study of incarcerated women, she recruited participants by hanging out in the prison smoking area, introducing herself to women and asking them to participate in the group discussions. For some groups, in-person recruitment is important, especially for gaining trust among some marginalized groups. For larger-scale focus groups or those that assess multiple demographic groups (such as Bittner et al., Reference Bittner, Goodyear-Grant and Wallace2018), this type of recruitment can be challenging and may not reach a broad enough audience.
A successful recruitment strategy requires careful consideration of the target population for the study and the best ways to connect and communicate accordingly. As we will discuss below, harnessing social media and online communications can improve our ability to recruit participants from marginalized or hard-to-reach populations. At the same time, digital divides mean that many living in rural areas, Indigenous peoples on reserves, and the poor may not be reached by virtual recruitment methods. As of August 2020, the average download speed was just under 6 Mb per second in rural areas in Canada, but it was more than 10 times faster in urban areas, at nearly 64 Mb per second.Footnote 2 If affected groups are part of a researcher's target population, a mixed recruitment strategy may be advisable. With all recruitment strategies, focus group researchers are well advised to be patient, because the methodological literature suffers from “over-emphasis on the degree of control researchers have over the relevant characteristics of individuals in their groups and often the exact composition of the groups will reflect circumstance rather than planning” (Bloor et al., Reference Bloor, Frankland, Thomas and Robson2001: 21). New tools for recruitment can give researchers greater control over identification and recruitment of participants.
Target groups and online advertisements
Deciding whom to study and how to best contact them is central to researchers’ recruitment strategy. For some studies, targeting a specific group that shares a common identification factor will be useful. This can include various socio-demographic markers (such as gender, race, sexuality, etc.) or behavioural-, interest- or experience-based characteristics (such as identification with a political party, career or position within an organization). In most cases, online advertising tools can help to bolster a recruitment strategy when the aim is to study broad or specific populations. Some of the best outlets include social media advertising, online classified listings and connecting with community organizations. A table summarizing the benefits and drawbacks of these various recruitment techniques is included in Appendix B.
Social media advertising
Social media platforms are increasingly used as recruitment tools for interview, focus group and survey participants (Snelson, Reference Snelson2016). They can serve as a stand-alone or supplementary mode of recruitment and are particularly useful for targeting specific socio-demographic markers or interests because of the ability to apply filters through social media program settings, thus allowing researchers to advertise to specific user identities and locations.
Facebook and Instagram are among the largest social media advertising giants. Note that in order to advertise on these platforms, you are required to start an account, which we recommend should be tied to a professional, project-based email account (more information is included in the section on ethics and best practices, as well as additional suggestions, below). Facebook ads allow you to set audience target parameters for location (including country, region, city or specific radius) and socio-demographic markers (such as age, gender, education and relationship status), as well as interests or behaviours, which can include an array of factors: hobbies or activities (such as entertainment interests), engagement with political associations or parties, and more. In the case of Wallace et al. (“Immigrants and Participation,” Reference Wallace, McGregor and Tolley2019), Facebook ads were particularly useful in connecting with people who identified as immigrants or expats living in Canada, weeding out contact with ineligible participants.
Once the audience is established, Facebook allows you to set parameters for the advertising campaign, including ad budget, ad placement and the length of time you want to recruit candidates. Pricing varies according to the specifications of the audience search, but control over budget allows you to carefully monitor and test the success of such advertisements.Footnote 3 Audience parameters can be altered over the course of a recruitment campaign to better ensure ads reach the desired population. Recruitment advertisements are most effective if primarily graphic-based, featuring a title (for example, “Participate in a Paid Research Study”), a photo or graphic (with no text), a one-sentence description of the promotion, and a “call to action” link (under taglines such as “Sign Up” or “More Information”) to a website or pre-screening application that includes more details on the study (more information on websites and pre-screening applications are discussed below).
Although far from perfect, social media advertising allows researchers to efficiently access a database of people from a variety of backgrounds and experiences, who can then choose to learn more or opt in to further discussion about the project. At the same time, Facebook and Instagram ads are not always effective at reaching specific subgroups, especially seniors and those who face challenges accessing the internet. This issue should be a central recruitment consideration, necessitating creative solutions to overcome any recruitment gaps. For example, in Bittner et al. (Reference Bittner, Goodyear-Grant and Wallace2018), recognizing that some participants may not have access to the internet, we also recruited in-person at public venues, such as recreation centres and libraries, and placed posters on community bulletin boards across the various cities under study. Social media and online advertisements were a critical forum for increasing engagement among youth and minorities, so this mixed approach helped us recruit widely. While there may be a growing perception that young adults have deserted Facebook, data suggest that individuals aged 18–34 are still the most frequent Facebook users. As of October 2020, 18-to-24-year-olds represent 15 per cent of users and 25-to-34-year-olds are 26 per cent of users.Footnote 4 Older generations are well represented among Facebook users too, with 12 per cent in the 55–64 category and 11 per cent over 65.
Online classifieds and community connections
In addition to placing ads on social media, researchers can also place ads in other online outlets and/or connect with associated groups and email listings that can reach specific populations. Online classified websites such as Kijiji, Craigslist and LesPac are often cost effective in reaching broad audiences. Although this approach can be more difficult than using social media advertising to target groups, these sites are visited frequently and may be useful for recruiting a wide applicant pool.Footnote 5
In addition to classifieds, we have also had success in networking with organizations connected to the target population. This produced a hybrid recruitment strategy that combined the advantages of online and traditional recruitment methods. For example, in Bittner et al. (Reference Bittner, Goodyear-Grant and Wallace2018), to best reach specific racial or ethnic minority groups in various cities across Canada, we frequently connected with local cultural or ethnic organizations to share information about our study with their membership email lists, social networking sites, and newsletters, which facilitated connection with marginalized groups.
Ethical considerations for online recruitment
Important ethical considerations arise in recruiting participants online, although researchers should not expect the adoption of new modes of recruitment and data analysis to cause delays in institutional review board (IRB) approvals or in securing the informed consent of participants, especially for the careful researcher. As Gelinas et al. (Reference Gelinas, Pierce, Winkler, Cohen, Lynch and Bierer2017) note, uncertainty still surrounds best practices for the ethics of social media use in research. They distinguish between passive online recruitment techniques, in which ads are placed and participants can freely choose to connect or volunteer, and active online recruitment techniques, in which researchers actively attempt to message or email people on social media who they believe are suitable for the study. To best maintain participants’ privacy, consent and dignity, researchers should avoid contacting potential participants directly. Instead, researchers should offer passive online advertisements that invite those interested to contact the research team or complete the pre-screener application (see below for more information).
Further, although online ads tend to be short in length (many have character limits for text), it is also critical that the researchers are transparent about the project and its objectives during recruitment. We suggest that online ads link to pre-screener applications and/or project websites that include information about the study, its core research questions, its risks and benefits, and the focus group research process. Further information pertaining to consent should be explained throughout the lead-up to, and introduction of, the focus groups. The limited space available in online advertisements is not grounds for withholding information about the study before participants sign up.
Pre-screener surveys and organizing focus groups
Once a sufficient pool of potential focus group participants has been recruited, it can be useful to learn more about these individuals, which will help to better organize the focus groups, given that group composition is a critical research design consideration. Studies should include groups that are homogenous on some or all the group identities of interest. For example, if a sex/gender perspective on a topic is important, some single-sex groups will be needed. Many dynamics can affect discussion, and since focus groups, at their core, consist of a fundamentally social means of data collection, group composition is a critical consideration in order to allow shared understandings to surface in discussions (see, for example, Allen, Reference Allen2006; Woodring et al., Reference Woodring, Foley, Rado, Brown and Hamner2006). Having too many groups that are mixed on characteristics of interest may be suboptimal because there are too many perspectives competing for attention; because of discomfort speaking about experiences or perspectives not shared by other participants; because members of socially privileged groups dominate conversation; or because members of socially privileged groups can discount, trivialize or silence other members (see Halcomb et al., Reference Halcomb, Gholizadeh, DiGiacomo, Phillips and Davidson2007), even without intending to.
Researchers can move applicant screening to a digital platform, streamlining traditional manual or phone-based surveys with prospective participants and permitting easier management of potential participants and group formation. A pre-screener is a useful tool for organizing focus group participants. It can take the form of a basic survey completed online by those interested in participating, in order to collect contact information and information relevant to group composition (for example, age, gender, race) as well as information pertaining to behaviours or other qualities of interest. For example, Wallace et al. (“Immigrants and Participation,” Reference Wallace, McGregor and Tolley2019) aimed to engage with immigrants who intended to abstain from voting in a local election in order to learn more about why they were disinclined to vote. Our pre-screener asked participants about their intention to vote in the upcoming election and then grouped individuals accordingly. In general, pre-screeners should avoid collecting personal information or identifiers in order to keep the survey relatively confidential and ensure the safety of participants’ data while researchers organize the groups. Pre-screeners should not request first or last names, birth dates, addresses or postal codes, and so on; instead, researchers can inquire about modes of communication and demographic markers (for example, email and/or phone number, year of birth, location by region). For a pre-screener example, see Appendix A.
The pre-screener can be posted online through a survey-based application, such as Qualtrics, SurveyMonkey, Google Forms or Nintex. Researchers can monitor completed forms to learn more about the applicant pool, adjusting the recruitment strategy as needed to better target under-represented groups. These web-based surveys are particularly helpful in complementing online recruitment strategies. For example, recruitment advertisements in social media can link directly to the pre-screener survey or the pre-screener link can be shared with organizations of interest, so prospective participants can directly provide their information if they choose to take part. Online pre-screeners can also be used with in-person recruitment by offering a paperless option for sharing information about the study. Instead of having research assistants carting around paper-based advertisements and surveys when we wanted to advertise in public spaces, we have had them take iPads or direct prospective participants to survey links on their smartphones. This approach reduces waste and keeps information about potential participants organized.
We also recommend asking for information on prospective participants’ availability on the pre-screener. We have recruited and retained far more participants when we included the dates, times and locations of the focus groups on the pre-screener application and asked respondents to select all they could attend. This information allows people to select groups that best fit their schedules. It also provides a preliminary indication for researchers about any potential issues with the spaces or timing of groups, especially for specific subgroups of potential participants (such as parents). If very few people indicate availability for a day and time, a researcher can pre-emptively cancel or reschedule that group to ensure time and resources are not wasted.
Researchers should also inquire about accessibility needs on the pre-screener (and subsequent participant surveys) to ensure they have the necessary information to facilitate everyone's equal participation in the group. Asking about such requirements well before the group meets, and on an anonymous survey, can enhance a participant's comfort in communicating accessibility needs, while also giving researchers time to meet those needs as far as possible. Accommodations may be needed for the physical setting of the focus group (for example, its location and accessibility and the set-up of the room), the types of activities or modes of participation undertaken in the focus group (for example, visuals, games or activities that participants are asked to explore) and various other potential considerations (for example, communications, interpretation services).
We found it useful to conduct the pre-screening of participants roughly 10 days before the first scheduled group. This generally gave participants enough time to know their schedules for the coming week and arrange any necessary transportation, childcare and other scheduling issues. Approximately three days after running the pre-screener, we examined preliminary results for dates and time slots that seemed to work best across our various target groups. For example, in the case of Bittner et al. (Reference Bittner, Goodyear-Grant and Wallace2018), we found within a few days of running the pre-screener for Calgary, Alberta, that a number of participants who were free for our second group were part of the LGBTQ+ community. We opted at that point to select only LGBTQ+ identifying participants for that time slot, allowing us to maximize in-group homogeneity per our group composition design. When it came to hosting an LGBTQ+ focus group, we messaged participants in advance of the group to notify them that they would be participating in a discussion with other members of the LGBTQ+ community and that we were interested in their perspective as self-identified members of this community. Participants could choose to continue in the group or register for another focus group that did not have a specific demographic target. In our experience, providing some information about the project, its intended goals and the value of speaking with certain demographic groups enhances the quality of discussion in the group.Footnote 6 In summary, using online technologies such as pre-screening applications can help us to learn more about participants and organize groups accordingly, and it also facilitates researchers’ engagement with participants before focus groups meet, in order to ensure that participants are informed about the research process and their role in it.
Web forums
In addition to online pre-screeners and advertisements, researchers can also use web-based forums to enhance the focus group experience when it comes to recruitment, engagement and research dissemination. At a basic level, this could include building a project website, blog or discussion forum—all useful for sharing information about the project with potential participants. Content could include descriptions of the research process and its objectives, communicated in various forms, such as text, video or audio content, as well as links to the pre-screener application. This type of online engagement can be especially useful for large-scale, multigroup projects and potentially those where lead researchers may not be engaging directly with the research participants. Offering a space where participants can hear from the researchers and learn more about their intentions or motivations surrounding the project could lead to greater trust and engagement in the focus groups. A project website can also provide a central location to post about the results of the study, including links to publications, talks, reports, op-eds, or other media. In particular, blog or vlog content that is intended to reach a public audience can be a helpful mode for dissemination beyond the scholarly community and may be of special interest to participants who contributed to the project.
After focus groups meet, online discussion forums can also be helpful in maintaining further engagement with participants, particularly if forums are used for participants to continue discussions about the content they explored in their groups. With consent from participants, these posts could provide additional textual data for the project from follow-up questions or discussions. For example, researchers can post a topical news article with a series of short questions that expand on focus group topics. This information can give participants an opportunity to elaborate on their perspectives. Participation should be optional and remunerated if researchers intend to include the data in their analyses. For these forums to work, expectations regarding respectful engagement should be clearly communicated to participants.
Encouraging attendance
The goal of recruitment is to ensure participants attend the focus group, so research design must also include associated techniques for encouraging participants to actually show up at the focus group. A widespread technique is to provide remuneration, which helps to compensate for participants’ time and reflects the value of the information and engagement they provide. As noted in the Wellesley Institute's investigation into compensation practices among Canadian researchers, “monetary compensation is a way that researchers recognize and respect the contributions of people with lived experience” (Cheff, Reference Cheff2018: 4). In our view, compensation should be at or above minimum wage as a fair recognition of the value of participants’ time, although some researchers suggest lower amounts. For example, Cyr (Reference Cyr2019) suggests $5 to $10 per participant or donating the group's “earned” money to a charity. The Tri-Council Policy Statement (Government of Canada, 2018), which governs research ethics in Canada, warns that incentives should not be so large as to interfere with voluntariness of the participation (see chapter 3). We have generally given participants $50 each for a two-hour focus group, in line with the median of $25/hour given by Canadian researchers for focus group and interview participants (Cheff, Reference Cheff2018; see also Bloor et al., Reference Bloor, Frankland, Thomas and Robson2001). In some urban markets, researchers may find greater success with higher remuneration: for example, $75 to $100 per two-hour session. Other forms of remuneration, such as childcare, transit and parking costs should also be provided to avoid excluding certain demographic groups.
Digital technologies offer some convenient and simple ways to encourage focus group attendance, including setting up automated SMS or email reminders to participants about the focus group's date, time and location. Personal phone calls to participants can be used too, which may be particularly effective in identifying last-minute no-shows. Additionally, over-recruiting to focus groups by about 10–20 per cent can be used in combination with follow-ups to encourage attendance. If a target group size is 10 participants, for example, a researcher would want to recruit 11 or 12, knowing that one or two of them will not show up, no matter what strategies are in place to avoid no-shows. No-shows and last-minute cancellations may also be an area where professional firms have an advantage, given that they tend to be paid on a “per show” basis. This approach incentivizes firms to secure solid commitments from participants and to replace no-shows as quickly as possible. The extra cost of hiring a firm to do focus groups may be worth it for some research projects, especially those that target hard-to-reach or small populations where participant absences can result in a lot of cancelled groups.
Additional suggestions for online recruitment and communications
• Set up a project-specific email address: With digital recruitment, researchers should streamline communications through a project email address. Social media and online classified ads can elicit extensive communications with participants, and a dedicated email account can help manage the large flow of communications. This can also help in cases where there are multiple research team members, ensuring that communications about the study are transparent and accessible to all.
• Run a test recruitment campaign: For researchers concerned about the targeted audience and reach of the project's online advertisements, running a small test campaign can be helpful. This could include, for example, posting a $10 budget for your Facebook and Instagram ads and assessing if, how and which types of prospective participants respond to the ad. Piloting in this way can identify changes necessary to improve the recruitment strategy.
• Seek feedback: In the pre-screener and/or follow-up questions in a feedback survey, ask how respondents found out about your study and their interpretation of the advertisement. This can be especially helpful in large focus group studies in which researchers may benefit from learning how to best allocate their budget and resources to reach specific populations in their subsequent groups.
• Acquire an email contact list: As noted above, information pertaining to the recruitment of participants should be deleted at the end of each study. However, participants can be asked if they wish to remain on an email contact list to learn about further opportunities for participation and/or information on the project's publications and reports. Some participants are keen to remain involved, often wanting to know about the conclusions of the research.
Digital Tech in Analyzing Focus Group Data
In addition to expanding recruitment, current digital technologies can be particularly valuable for organizing and analyzing focus group data. Focus groups can provide a remarkably rich source of data, allowing researchers to delve deeply into questions around political behaviour and public opinion, for example. With focus group data, four core dimensions of analysis tend to be considered:
• Responses: What answers did participants provide to the questions/topics? Is there a general consensus or do different themes emerge (perhaps across different groups of participants)?
• Rationales: How do people justify or explain their opinions and responses? What are the ways that they come to understand what they know?
• Communications: What type of language or terminology do people use in their responses?
• Interactions: How do participants interact with one another? Are there power dynamics or is there evidence of collaborative knowledge sharing among participants?
The literature on focus groups methodologies has long supported content analysis for examining focus group transcripts. Traditionally, researchers have highlighted the value of “scissor-and-sort” techniques (also known as “copy-and-paste” methods) for analyzing core themes in their datasets (see, for example, Stewart et al., Reference Stewart, Shamdasani and Rook2007; Stewart and Shamdasani, Reference Stewart and Shamdasani2014). This approach starts by identifying core topics and then classifies sections of focus group transcripts under the various themes. In the past, this literally entailed cutting relevant pieces or pages out of focus group transcripts and sorting them manually into their appropriate themes. Now this process generally involves copying and pasting texts in word processing programs under various classifications.
Although this technique is simple, it is cumbersome and raises questions. For example, while researchers may identify themes present in the transcripts, what about those that are absent or marginalized within group discussions? How do themes potentially differ across groups or participants? What about latent differences in interactions, language use or terminology across these large bodies of text? Many of these questions can be addressed with new and emerging technologies in content analysis, particularly software geared toward organizing, coding and analyzing collections of texts. The sections that follow assess some of the tools available to researchers conducting qualitative and automated content analyses, drawing on our experience with two projects.Footnote 7
Qualitative text analysis with focus group transcripts
The central objective of qualitative text analysis is to reduce large bodies of text to key concepts or results (Erlingsson and Brysiewicz, Reference Erlingsson and Brysiewicz2017). Qualitative data analysis software (QDAS) is hailed across disciplines for allowing researchers to efficiently organize and mine qualitative data, reducing many of its complexities and providing tools for visualizing core findings (Kaefer et al., Reference Kaefer, Roper and Sinha2015; Leech and Onwuegbuzie, Reference Leech and Onwuegbuzie2011). This is particularly the case when it comes to focus group data, given the large volume of text produced in transcripts.
Software such as QDA Miner, NVivo, Atlas.ti and MAXQDA are among the most popular applications. Building on traditional scissor-and-sort methods of analysis, as well as manual techniques in open, axial and selective coding, many features of these programs allow researchers to code transcripts across multiple dimensions. This can include coding transcripts by sections or questions to address key topics, as well as participant or group demographics. These applications can also identify different types of interactions between participants (such as disagreements among participants over various claims), to engage more deeply with the interactive dimensions of focus group conversations.
Although the terminology for various functions can differ across software, one of the core features of QDAS is the creation of data codes across multiple dimensions. Codes can generally be thought of as labels for various units of the focus group transcripts and can include sentences, speaking points or exchanges between participants. Codes can be applied to the following elements:
• Research questions or sections of the transcript: These codes can help to break down the transcripts into relevant sections for closer and comparative analyses. This generally allows researchers to home in on specific and relevant “chunks” of the text for closer examination.
• Key themes: These codes can be applied to core ideas or recurring themes in discussions. These can be developed through a deductive framework (preconceived set of topics established before data analysis) or an inductive framework (interpretation of key themes that emerge as a result of data analysis).
• Participant demographics: These codes are applied to participants in the transcripts and can be used to analyze variation in responses, language/diction and interactions. Relevant markers can include gender, race, age, sexuality, party identification, and so on.
• Interactions: These codes can capture different types of exchanges between participants, especially to learn more about positionality, authority and opinion formation. This can include identifying areas of agreement or disagreement between participants and how this comes to be resolved.
Codes can also be broken down into subcodes (such as specific responses to certain questions), and codes can overlap with one another. Indeed, one of the greatest values of QDAS is that researchers can explore the intersections of various codes (such as responses or key themes by gender), allowing us to dig deeper into the applications of the findings.
Beyond the codification and management of transcripts, QDAS offers a number of important tools for helping to identify core findings. At a base level, frequency and cross tab data for all codes can help researchers establish the prevalence of responses, key themes and keywords, which can be a useful function in helping to avoid very general language in the reporting of focus group data. As Leech and Onwuegbuzie (Reference Leech and Onwuegbuzie2011) observe, too often focus group researchers employ generic statements of findings that refer to “many,” “most” and “some” cases. Basic frequency and cross tab information can enhance specificity and precision for researchers and readers alike. For example, researchers can describe the prevalence of core topics as they occur across different focus groups (for example, “This theme was raised in 12 of the 20 focus groups on x . . . ”) and across different participant categories (for example, “40 per cent of woman-identified participants in the study mentioned this theme at least once . . . ”). In Wallace et al. (“Immigrants and Participation,” Reference Wallace, Goodyear-Grant and Bittner2019), the research team coded three explanations that foreign-born voters gave for lower turnout: lower attachment to cities/neighbourhoods, different priorities and limited time for learning about local politics, and lower knowledge of municipal government roles and responsibilities. By coding these responses and participant demographic variables in QDA Miner, we were able to parse the ways in which these explanations differed across subgroups.
Researchers can also use word count functions to analyze the prevalence of themes and can search for keywords to uncover more about the use of particular language. This tool can be especially useful in the study of symbols or metaphors, where researchers can search for and code specific mentions of these terms and more closely analyze their meaning in group discussions. The keyword-in-context function in QDAS programs also allows the researcher to survey the words adjacent to key terms or phrases. For example, when unpacking the focus group data in Wallace et al. (“Immigrants and Participation,” Reference Wallace, Goodyear-Grant and Bittner2019), we found that some participants talked about news media as a site of learning about local elections. We wanted to learn more about how this arose in conversation, so we searched for terms proximate to media in the keyword-in-context function and uncovered that a lot of participants identified the loss of local newspapers as an obstacle for learning about local elections.
QDAS can also be helpful for analyzing interactions among participants. Researchers interested in understanding if and how participants’ perspectives changed in response to their peers could manually code different types of perspective-taking, such as challenging, conceding or reframing various claims. They could then, in turn, explore if participant demographic variables or group composition have an effect on these types of perspectives.
QDAS software also allows researchers to incorporate elements of data collection beyond the focus group transcripts, enhancing documentation and thus transparency and accountability in research. These programs can be an excellent space to incorporate notes on focus groups, such as observations about nonverbal communications (for example, body language) during the session. These can be coded to complement or expand the research framework and provide more relevant contextual information in analyzing the focus group transcripts.
Automated content analyses of focus group data
The large quantity of text that can emerge from focus group transcripts also lends itself to automated content analyses. Automated content analysis differs from manual forms of qualitative analysis by empowering digital technologies to identify patterns in textual data. This is different from what we describe above, in that qualitative software generally allows the researcher to remain in the “driver's seat” when it comes to determining the parameters for coding, and although some aspects of analysis are mechanized for efficiency, the researcher often plays a more active role in the analysis. Automated analyses, on the other hand, utilize different functions of computational text analysis to uncover patterns in the text that are often not readily apparent to the researcher. While there is a range of programs offering different features, some of the more popular ones are Provalis Research Text Analytics (WordStat), Python, R, SAS and Lexicoder.
Although automated analyses are efficient, there can be some limitations and drawbacks when applied to focus group data. First, most automated analyses require a considerable volume of text to avoid errors in coding; in projects where researchers are convening fewer than three focus groups, or for those where focus groups are short in duration, it may be best to manually code transcripts and data. Similarly, if participants are required to participate in activities that require smaller group work (and, in effect, side conversations) or nonverbal communications, these approaches may not be appropriate for analysis. Some researchers may also find that, based on the topics, questions and flow of conversation within the focus groups, manual approaches to coding and analysis may be better suited to a project. We suggest, however, that where appropriate, some of these techniques can help to uncover new insights about the topics under study. As we expand below, some applicable types of automated analyses include dictionary and sentiment analyses, as well as topic modelling and word embeddings.
Dictionary and sentiment analyses
Dictionary methods are those that utilize lists of key terms and phrases to determine the frequency of a relevant theme, response or idea that emerges in the focus groups. In the case of focus groups, this approach can be used to help trace different narratives or prevalence of specific contributions to discussion. Dictionaries can be predetermined—borrowed from existing repositories—or custom-built by researchers to address specific themes identified in the focus groups. Numerous programs offer sentiment dictionaries, for example, which are built to measure the tone of a text by comparing the frequency of positive emotion words to negative emotion words. Sentiment analyses are popular in studies of political texts (see, for example, Young and Soroka, Reference Young and Soroka2012; Lawlor, Reference Lawlor2015; Wallace, Reference Wallace2018) and could also be useful in expanding our core takeaways from focus group conversations. In particular, analyzing sentiment or tone within and between different groups and types of participants could give researchers insight into the general feelings or orientations of different groups toward the issues at hand.
Some programs, such as Linguistic Inquiry and Word Count (LIWC), also provide researchers with a host of dictionaries that can be used to assess various psychological dimensions of the texts, such as anxiety, anger and sadness. These types of dictionaries could be particularly useful to researchers exploring political behaviour and public opinion—especially political polarization and resentment, to provide more specific examples—by offering insight into the emotions surrounding various questions, debates and interactions between participants. Indeed, where sentiment analyses in political science research typically focus on political communications, analyzing sentiment in the context of focus groups could help us to better understand how individuals communicate and rationalize about their political perspectives; this, in turn, could further help us decipher the ways in which these emotions may be differently experienced across participants according to gender, race, partisanship, and more.
Researchers can also create their own dictionaries based on categories of themes or responses that they wish to analyze. In our research, we have used this technique to gain greater insight into how voters think about the gendered barriers that women candidates face in getting elected to office, especially when it comes to the different types of scrutiny that they face in the campaign (Wallace et al., “Perceptions of the Playing Field,” Reference Wallace, Goodyear-Grant and Bittner2019). In our analysis, we developed a dictionary with four categories to assess if participants acknowledged whether women faced differing degrees of scrutiny than their male colleagues on their appearance, personality traits, family status and assumed abilities. This type of dictionary was specific to the topic at hand, but it revealed how participants perceived the playing field for women in politics and which participants were more inclined to discuss different dimensions of scrutiny over others.
Although dictionaries can be a powerful tool, it is important to keep in mind that they require considerable validation and need to be calibrated or contextualized for the topic at hand. This is especially the case when working with pre-existing dictionaries that may not accurately reflect the language and terminology expressed by participants in the group. In this regard, it can be helpful to employ computational tools that work behind the scenes to explore some of the organic or naturally occurring themes or topics in the texts, such as topic modelling and word embeddings.
Topic modelling and word embeddings
The central goal of topic modelling is to “determine structures in underlying document collections” (Eickhoff and Neuss, Reference Eickhoff and Neuss2017: 1328). The most common form of topic modelling, latent Dirichlet allocation (LDA), uses algorithms to draw out “clusters” of words that represent topics or themes based on their co-occurrence and frequency within a set of texts. Importantly, LDA can identify patterns in word use and frequency that are not necessarily readily apparent to the researchers and may constitute underlying themes within the texts. In the case of focus groups, this can be particularly valuable in avoiding interpretive biases. Contrary to custom-built dictionaries, this type of process occurs beyond the direct mediation of the researchers and may help to reveal subtle or previously untheorized topics in a text.
In Bittner et al. (Reference Bittner, Goodyear-Grant and Wallace2018), we applied a topic modelling approach to our analysis of focus group data on understanding how Canadians conceptualize gender and its effects on political life. Participants generally described gender in three ways: discussion of gender in relation to bodies and physical characteristics, role descriptions, and traits and dispositions. This was not a scheme that was readily apparent to the research team in reading the transcripts initially, but this type of clustering technique revealed that these were the various ways that participants came to define masculinity and femininity across the different groups.
In a similar vein, word embeddings also represent a more recent development in automated analyses that can be revealing of the context around word use and meanings. In its most basic sense, word embeddings are representations of words as vectors that demonstrate their proximity to other terms in the body of texts and the contexts in which they are used. This can be particularly useful for studying symbols or the use of key terms or phrases in focus groups, in order to learn more about how participants interpret these words. For example, building on our (Bittner et al., Reference Bittner, Goodyear-Grant and Wallace2018) analysis, we are now developing word embeddings around masculinity and femininity in our focus group data to understand how different groups may have interpreted the terms in distinct ways. Coding the most proximate words around masculinity and femininity in each of the transcripts—and further parsing this by participant markers such as gender, race and age—permits us to examine the contexts in which participants discussed masculinity and femininity and better understand how this connects to political orientations and attitudes.
In sum, these types of automated approaches to analyzing focus group data can expand the reach of the data. While there is a steep learning curve when it comes to using these technologies and they also require extensive validation checks,Footnote 8 they can effectively complement manual or finer-grained assessments of focus group data and can be used in conjunction with other types of textual data, such as discussion forums, social media data, and more.
Conclusion
For political behaviour scholars who typically conduct survey research, expanding your tool kit to incorporate focus groups into your analyses can be both challenging and intimidating. The learning curve is steep, and it can take some time for researchers to bolster their competence (and confidence) in the use of this method. We strongly believe that there is much benefit to the use of focus groups for scholars of political psychology and public opinion, as it has provided deeper and richer insights into the beliefs, attitudes and behaviours of citizens, allowing us to gain important knowledge into experiences and opinion formation, as well as facilitating the development of better survey instruments.
In this article, we draw a number of important conclusions that we hope will help the research community, including scholars who are considering the use of focus groups to answer their research questions and peer reviewers who may not be familiar with focus groups or who may be well versed in focus groups but not the various digital technologies that can be harnessed to facilitate them and analyze the data that emerge.
Aimed at both new and existing focus group users, this article has, in particular, outlined the use of digital technologies for the recruitment of focus group participants and for the analysis of focus group data. The refinement of methods and techniques is never finished and learning is never complete—an important truth across all fields and methodologies; and the one enduring piece of advice about both focus group recruitment and data analysis is to choose the best tool for the task. For some projects, technologically assisted recruitment and data analysis will be a remarkable boost to the use of focus groups in political science research.
Supplementary material
To view supplementary material for this article, please https://doi.org/10.1017/S0008423921000226
