1 Introduction
Studying group decision making is more challenging than studying individual decision making for multiple reasons. It requires more complex data analysis methods due to an increase in the number of levels of analysis and relevant variables. It also requires increased sample sizes for sufficient statistical power to test hypotheses, as groups rather than individuals are the unit of analysis. Groups consist of members that in themselves constitute sources of variance, and this is further complicated by variation in the social interactions of group members between groups. All this increases between-group variability, which increases the required sample size to detect the effects of experimental manipulations. Another challenge is the practical requirement of needing group members present at the same time and location, where single no-shows can lead to cancelled experimental sessions. In this paper, we suggest and demonstrate that one way to facilitate group decision research, without necessarily losing any validity, is to run such studies online.
1.1 Alternative media for doing research
Online research in individual decision making has been growing since the emergence of easy-to-use online-survey creation tools, such as SurveyMonkey, Qualtrics, and Google Forms, and large online subject pools, such as Mechanical Turk. For individual decision making, researchers have demonstrated that similar results are obtained with in-person and online samples (see Reference Baldassi, Weber, Johnson, Nair, Czaja and LiBaldassi, Weber, Johnson, Nair, Czaja, & Li, 2011; Reference Buchanan and SmithBuchanan & Smith, 1999; Reference Buhrmester, Kwang and GoslingBuhrmester, Kwang & Gosling, 2011; Reference Gosling, Vazire, Srivastava and JohnGosling, Vazire, Srivastava & John, 2004; Reference McGraw, Tew and WilliamsMcGraw, Tew & Williams, 2000; Reference Paolacci, Chandler and IpeirotisPaolacci, Chandler & Ipeirotis, 2010; Reference Robins, Trzesniewski, Tracy, Gosling and PotterRobins, Trzesniewski, Tracy, Gosling, & Potter, 2002).
Group research has not made this transition yet. Even though studies have been run online, studies that run exclusively online are relatively less prevalent. Studies that have made use of online participants have mostly examined the effects of differences between different types of communication, such as text-based communication, video conferencing, and face-to-face (FtF) interaction. Although this research has yielded interesting and important results (e.g., Reference Adams, Roch and AymanAdams, Roch, & Ayman, 2005; Reference Baltes, Dickson, Sherman, Bauer and LaGankeBaltes, Dickson, Sherman, Bauer, & LaGanke, 2002; Reference Hedlund, Ilgen and HollenbeckHedlund, Ilgen, & Hollenbeck, 1998; Reference Lowry, Roberts, Romano, Cheney and HightowerLowry, Roberts, Romano, Cheney, & Hightower, 2006), and much is now known about the differences between communication media and their effects on group decisions and the experience of group members, we argue that such potential effects of medium on communication are not always problematic for research that makes use of different media than FtF. We propose that, for many research purposes, online groups and FtF groups may produce essentially identical results on the variables of theoretical interest. What is more, in some cases non-face-to-face communication may be preferable to face-to-face communication. For example, for many research questions and research tasks, visual and other nonverbal cues are irrelevant and constitute undesirable noise that is difficult to measure and control for, since it requires considerable effort and skill to code. Since some online media do not allow for visual communication or allow visual communication to be blocked, such media may be preferable to FtF communication.
Moreover, the implicit assumption that FtF communication should be the standard for group research because of its external validity is rapidly becoming obsolete, as more and more real-world group decisions are made via computer-mediated communication. In today’s global society, groups frequently interact over long distances via teleconferences, voice over internet protocols (VOIP), or web conferences, which commonly include some combination of VOIP, document or screen sharing, and sometimes live video. These new online communication media that are part of what is popularly called Web 2.0 (“Web 2.0”, 2011) allow group members to work simultaneously on a shared project or decision (Reference FodorFodor, 1998; Reference Hoegg, Martignoni, Meckel and Stanoevska-SlabevaHoegg, Martignoni, Meckel, & Stanoevska-Slabeva, 2006; Reference Kim and BonkKim and Bonk, 2002).
Here we examine group decisions where group members communicate via conference call, while engaging in a shared online decision task. To demonstrate the viability of using web-conference technologies for group decision-making studies, we conducted a study that allows us to compare the results of group and individual decisions made by online and face-to-face groups. We argue that, contrary to the aforementioned literature that focuses on differences between face-to-face (FtF) and computer-mediated communication (CMC), decision outcomes are frequently very similar for the two types of interactions we investigate, namely web-conference groups (or online groups) and FtF groups. Our results show that CMC and FtF groups cannot be distinguished on the basis of their decision outcomes. Note that we do not claim that this necessarily holds for all forms of CMC; our online group method is but one of many possible forms of CMC, and our results do not necessarily hold for other forms of CMC.
We also investigate duration of the group discussion and the number of thought-units generated during the decision-making process. We do this because we expect that another advantage of the online method is greater efficiency in measuring the decision-process. With interaction between group members restricted to verbal communication, the group discussion is likely to contain more detail and elaboration that would have otherwise be communicated nonverbally in a face to face interaction. Since verbal communications are much easier to measure and analyze than elusive non-verbal behavior, such as smiles or raised eyebrows, we expect a greater density of measurable information for online groups.
As shown in a study by Reference Walther and BazarovaWalther and Bazarova (2008), communicators compensate for a lack of nonverbal communication by adapting their communication style and levels of effort to the medium. Others have found similar compensatory behavior (see e.g., Reference Walther, Loh and GrankaWalther, Loh, & Granka 2005; Reference Clark, Brennan, Resnick, Levine and BehrendClark & Brennan, 1991; Reference Gergle, Millen, Kraut and FussellGergle, Millen, Kraut & Fussell, 2004; Reference Hesse, Werner and AltmanHesse, Werner, & Altman, 1988; Reference HindsHinds, 1999; Reference Kenny, Kashy, Mannetti, Pierro and LiviKenny, Kashy, Mannetti, Pierro, & Livi, 2002; Reference KorzennyKorzenny, 1978) for a variety of online communication modes, such as email and chat.
It is notoriously difficult to argue for null effects and logically impossible to “prove” them. Thus, we go out of our way to show that the lack of differences we find between FtF and online groups is not simply due to a lack of power or an inadequate experimental setup. We do this by manipulating an independent variable, namely gain vs. loss framing of choice alternatives, an important moderator of risk taking, both for individuals (Reference Kahneman and TverskyKahneman & Tversky, 1979) and for groups (Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch, Weber, Appelt, Handgraaf & Krantz, 2009). We use this independent variable because it is one of the most well-known and robust findings in decision-making. Not only do we find the predicted effects of framing, but we find that they occur identically in FtF and online groups and that none of the effects are moderated by the communication medium.
We do not argue that FtF and CMC communication are identical. In fact, we expected and found several main effects of communication medium. However, these differences between media were not associated with differences in decision outcomes or in the effects of frame on decision outcome variables. We argue that, for certain types of decisions (described below), online groups’ decisions are indistinguishable from those of FtF groups and thus that studying online groups is a valid way to collect data for group decision studies. Since CMC is generally more efficient than FtF as a research medium, it can be an attractive alternative to FtF studies. We aim to demonstrate that both FtF and CMC are viable media, both for doing research (they can lead to essentially identical results, for instance in this study) and as a reflection of reality (more and more group decisions happen via CMC).
1.2 Computer mediated communication vs. FtF interaction
Previous research (e.g., Reference Adams, Roch and AymanAdams et al., 2005; Reference Arrow, Berdahl, Bouas, Craig, Cummings and LebieArrow, Berdahl, Bouas, Craig, Cummings & Lebie, 1996; Reference Baltes, Dickson, Sherman, Bauer and LaGankeBaltes et al., 2002; Reference Becker-Beck, Wintermantel and BorgBecker-Beck, Wintermantel, & Borg, 2005; Reference Hedlund, Ilgen and HollenbeckHedlund et al., 1998; Reference Lam and SchaubroeckLam & Schaubroeck, 2000; Reference Lowry, Roberts, Romano, Cheney and HightowerLowry et al., 2006) has found differences in decision making processes and outcomes between CMC and more conventional FtF groups. Many of the differences found in earlier studies may have been due to the novelty of CMC and the limitations of early CMC technologies. Today many people use CMC daily, and CMC may even have taken over as the most commonly used form of communication (Reference Baltes, Dickson, Sherman, Bauer and LaGankeBaltes et al., 2002). This trend is expected to continue into the future, making online group interaction an ever more externally valid and important research method. As CMC becomes more common and natural, we expect it to become more and more similar to FtF interaction, both in terms of decision outcomes and processes. Korzenny’s (1978) theory of electronic propinquity (i.e., the psychological feeling of nearness that communicators experience) argues that one experiences greater propinquity when there is greater bandwidth (or perceived amount of information per temporal unit; see also Reference Short, Williams and ChristieShort, Williams, and Christie’s [1976] social presence theory); when information is less complex; when greater mutual directionality is possible (Reference Daft and LengelDaft & Lengel’s [1986] media richness theory); or when individuals have greater communication skills, fewer rules, or a smaller number of perceived choices among communication channels (Reference Walther and BazarovaWalther & Bazarova, 2008). Our argument is that since both the bandwidth of communication media and people’s skills at communicating online constantly increase, differences between FtF groups and online groups are diminishing. Support comes from longitudinal studies that show that, as people get more familiar with CMC, differences between FtF and CMC communication disappear (Reference Hollingshead, McGrath and O’ConnorHollingshead, McGrath & O’Connor, 1993; Reference Van der Kleij, Schraagen, Werkhoven and De DreuVan Der Kleij, Schraagen, Werkhoven & De Dreu, 2009). Indeed, recent research has already shown striking similarities between online and FtF interactions. Reference Derks, Fischer and BosDerks, Fisher, and Bos (2008) in their recent review of the role of emotions in CMC argue that emotions are very much present in CMC, and that FtF and CMC groups are surprisingly similar with regard to the communication and occurrence of emotions. In the current study, we aim to demonstrate the similarity between online and FtF group decisions in an experimental setup and to show that these similarities hold even without the presence of an adaptation period.
Additionally, it has been shown that the effects of FtF versus CMC are moderated by the types of tasks under study, with decision-making tasks suffering least from differences in media (Reference Baltes, Dickson, Sherman, Bauer and LaGankeBaltes et al., 2002). In the literature on differences and similarities between online and FtF groups, an important distinction between task types is made by Reference McGrathMcGrath’s (1984) circumplex model, which distinguishes two dimensions of group tasks (see also Reference Baltes, Dickson, Sherman, Bauer and LaGankeBaltes et al. 2002; Reference Hollingshead, McGrath and O’ConnorHollingshead et al., 1993): the type of performance required (cognitive vs. behavioral) and the type of interdependence among group members (cooperative vs. conflicting). The circumplex model characterizes tasks in terms of the quadrants of the two-dimensional model: Quadrant 1 (cognitive/cooperative) consists of Generate-type tasks, such as creativity and planning; Q2 (behavioral/cooperative) consists of Choose-type tasks, such as Intellective tasks (choosing a correct answer) and Decision-making tasks (deciding issues with no correct answer); Q3 (cognitive/competitive) consists of Negotiate-type tasks, such as cognitive conflict and mixed-motive tasks; and Q4 (behavioral/competitive) consists of Execute-type tasks, such as performance and contest/battle tasks. Our study is focused on Quadrant 2 and specifically on group decision-making tasks that do not have a demonstrably correct answer (i.e., preference rather than inference or problem solving tasks).
Other studies examining such tasks have indicated an absence of differences between FtF and non-FtF groups when performing such tasks. Reference Hollingshead, McGrath and O’ConnorHollingshead et al. (1993), for instance, found no effect of FtF vs. non-FtF groups on decision-making tasks. Even Reference Baltes, Dickson, Sherman, Bauer and LaGankeBaltes et al. (2002), whose meta-analysis focused on comparing FtF interaction to asynchronous CMC (i.e., interactions such as chat and email where the sender and receiver are not simultaneously involved in communication) only found significant differences between FtF and non-FtF groups for intellective and mixed motive tasks, not for decision-making tasks. If such differences for decision-making are not present even when asynchronous media are compared to FtF interaction, it is likely that differences in decision outcomes between online and FtF groups will be minimal as well. We aim to show that even online groups who have not worked together before make very similar decision to FtF groups. Moreover, our study differs from these previous ones in that it investigates synchronous communication (i.e., non-text-based discussion).
1.3 Framing Effects and group decision making
To show the equivalence in decision-making for groups interacting via these two media, we replicate a well known and frequently replicated finding in decision-making, namely the effect of gain/loss framing on decisions. The powerful effect of outcome framing on the decisions made by individuals has been well established. Reference Kahneman and TverskyKahneman and Tversky (1979) hypothesized that framing outcomes as gains or losses influences choice by changing the valence of the certain option, i.e., whether it is seen as a sure good thing (gain) or a sure bad thing (loss). In their famous Asian disease problem, for instance, two alternative programs to combat a disease are proposed. Half of the participants are presented with the choice between program A: “Out of 600 people, 200 people’s lives will be saved”, and program B: “there is a one-third probability that 600 people will be saved, and a two-thirds probability that no people will be saved”. The other half of participants are presented with the choice between program C: “Out of 600 people, 400 people will die” and program D: “there is a one-third probability that nobody will die, and a two-third probability that 600 people will die”. People tend to prefer the certain option (program A, which is equivalent to program C) in the gain frame, and the risky option (program D, which is equivalent to program B) in the loss frame. The effect of framing has previously been examined in group decisions (Reference KühbergerKühberger, 1998; Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al., 2009; Reference Neale, Bazerman, Northcraft and AlpersonNeale, Bazerman, Northcraft, & Alperson, 1986; Reference Pease, Bieser and TubbsPease, Bieser, & Tubbs, 1993). We investigated the differential effects of framing on groups versus individuals in FtF and online settings.
1.4 Overview of present study and hypotheses
In the present study, we compared data from the Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al. (2009) study using FtF groups with data from an equivalent experiment run using online groups, who interacted via a web conference format that combined a conference call with document sharing. Both studies looked at the impact of framing on group decisions and processes. We expected our online groups to replicate Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al.’s (2009) findings for FtF groups. Specifically, we expected to find similar effects of risky choice framing (gain vs. loss) for FtF and online groups. In particular, we expected that individuals and groups in the loss frame would be more likely to choose the risky option than those in the gain frame (Hypotheses 1a and 1b). It should be noted that, although our focus is on group decisions, we also looked at individual decisions, since our argument that online and FtF groups make similar decisions would be severely weakened if we found differences at an individual level between decisions made in the lab or online.
We also compared other features of the online and FtF group interactions, such as numbers of verbalizations and time to reach a decision. As mentioned, we expected that the online method is more efficient with regard to the decision process: Since participants must substitute verbalizations for nonverbal cues due to the lack of visual communication (Reference Derks, Fischer and BosDerks et al., 2008), we hypothesized that there would be more verbalizations or thought-units (discussed in the Method section), in the online groups (Hypothesis 2a). We also expected that the online groups would be more focused on the task at hand (i.e., the decision) rather than on the social context of the group setting. We thus expected the online groups to need less time to come to a decision (Hypothesis 2b). Most importantly, we hypothesized that medium would not have a moderating influence on the effects of frame on any of our dependent variables (Hypothesis 3), since, as argued before, online groups compensate for the lack of richness of the medium by adapting their communication style and levels of effort to the medium.
It should be noted that in the study by Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al. (2009), prior consideration (whether or not participants were asked to make individual decisions before entering the group decision process) was also manipulated. No effects of prior consideration on decision outcomes were found, and since this was not a variable of interest for our current analysis, we did not generate hypotheses regarding prior consideration. However, we did include this factor into the design of the online part of our study, both to be able to make a full comparison with the Milch et al. data, and to optimize statistical power by using a full factorial design in our statistical analyses. We thus always added prior consideration as an independent variable in our model, though it never yielded any (interaction) effects with other independent variables.
2 Method
2.1 Participants and design
Participants in the online groups were recruited through email from the Columbia University Center for Decision Sciences’ online Virtual Lab subject pool. The criterion for participation was simultaneous access to a telephone (landline, mobile, or VOIP) and the Internet. Participants (N = 99, 76.8% female, mean age = 34.9 years, age range = 18 to 65 years) were run in 33 groups of three.
For analyses, we combined the online sample with the FtF sample from Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al. (2009), for a total of 201 participants in 67 three-person groups. For the group level analyses, we had to remove three groups: two groups had extreme decision completion times (> 3 SD above the overall mean), and one group’s data was not correctly recorded. We tested the effects of communication medium and frame in a 2 (frame: loss vs. gain) × 2 (prior consideration: predecided vs. naïve) × 2 (communication medium: online vs. FtF) factorial design. Only participants in the predecided conditions (i.e., half the groups) made the decision individually, the design for the individual decisions was a 2 (frame: loss vs. gain) × 2 (communication medium: online vs. FtF) factorial design (n = 105).
2.2 Procedure
2.2.1 Online groups
The email invitation included a link to an online Sign-up Page where participants could sign up for a timeslot to participate. When participants signed up, they were randomly assigned to either the predecided or naïve condition and the gain or loss frame and sent a confirmation email with instructions. Participants in the predecided group condition were sent a password and the link to an individual decision scenario (predecision). After completing the decision, these participants were given a conference call number to dial at the specified time. There was never more than half an hour between the pre-decision and the group decision. Participants in the naïve group condition were sent the conference call number via e-mail, without a prior individual survey. All participants received an email reminding them of their scheduled time and conference call number and providing a link to the online group decision webpage.
At the specified time, all participants called the conference call number and simultaneously accessed the online group decision webpage. Groups were composed of three predecided participants or three naïve participants. Within a group, all participants saw the same frame (gain or loss) of the decision, and predecided participants saw the same frame for both their individual and group decisions.
The experimenter followed a standardized protocol for each group. The experimenter welcomed the participants and explained the study procedure. Participants briefly introduced themselves. To break the ice, participants were asked to state from which city they were calling and what the weather was like there. Participants then read and discussed the decision scenario (described below), came to a group consensus, and made their decision. Discussions were recorded online. Lastly, participants were instructed to exit the conference call and complete additional questions individually. All participants were compensated $10 via PayPal.
2.2.2 FtF groups
The procedure for FtF groups in the Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al., 2009 study was similar, except that participants participated in person in the lab using pencil and paper. Additionally, FtF groups were drawn from pre-existing campus and work groups rather than being formed ad-hoc. Participants in the predecided group condition were first given the decision scenario (predecision) and asked to make the decision individually. They were then asked to make the decision again as a group by discussing the decision and coming to a group consensus. Discussions were recorded. Participants in the naïve group condition were only asked to make the decision as a group by coming to a group consensus.
2.3 Materials and measures
2.3.1 Decision scenario
The decision scenario used for both online and FtF groups was a version of Reference Tversky and KahnemanTversky and Kahneman’s (1981) Asian disease problem, which was modified to make the content more personally relevant to the participants. The scenario involved a threat of an outbreak of West Nile virus in the participant’s city. Participants were told that an estimated 600 inhabitants of the city would become infected and experience severe symptoms and that there was no known vaccine. Participants were then required to make a binary choice, selecting one of two potential research programs for the government to pursue in order to develop a vaccine. The riskless option (Program A) had deterministic outcomes, while the risky option (Program B) had probabilistic outcomes. Outcomes were framed differently between groups. In the loss frame, participants were told that if Program A was chosen, 400 residents would become infected with West Nile virus and experience severe symptoms. If Program B was chosen, there was a 1/3 chance that no one would become infected and a 2/3 chance that all residents would become infected. In the gain frame, Program A was associated with 200 residents being protected against West Nile virus, and Program B was associated with a 1/3 chance that everyone would be protected and a 2/3 chance that no one would be protected against the infection. For this decision task, groups were randomly assigned to the gain or loss frame.
2.3.2 Thought units
Transcripts of the group discussions were coded by two independent, blind coders for number of thought-units (Reference GottmanGottman, 1979), which is defined as “a sequence of words conveying a single thought” (Reference Weldon, Jehn and PradhanWeldon, Jehn, and Pradhan, 1991). The two coders were given coding instructions and examples and then coded 10 of the transcripts. Coders discussed discrepancies, recoded, and then interrater reliability, calculated using the intraclass correlation, was reevaluated. Once agreement was established at .75 or higher, a single coder finished coding the remaining transcripts. We also measured the time it took groups to come to a decision.
3 Results
3.1 Individual decisions
To test for effects of communication medium on individual decisions, a 2 (frame: loss vs. gain) x 2 (communication medium: online vs. FtF) logistic regression was conducted for the predecided individual decisions, i.e., the decisions the individuals in the predecided groups made before the group decision. As expected, the analysis revealed a main effect of frame, Wald χ2(1, N = 105) = 5.125, p = .02, but no main effect of communication medium, Wald χ2(1, N = 105) = .020, ns, nor an interaction effect, Wald χ 2(1, N = 105) = .087, ns. For the online pregroup condition, the framing manipulation affected individuals’ choices on the West Nile virus task, (χ2 (1, N = 51) = 7.07, p = .008): Individuals in the loss frame predominantly chose the risky option (81.5%), whereas individuals in the gain frame were almost evenly divided between the two options (45.8% chose the risky option; see Table 1). The same pattern was found within the FtF setting: 80.0% chose the risky option in the loss frame, whereas individuals in the gain frame were exactly evenly divided between the two options (50.0 % chose the risky option; χ2(1, N = 54) = 5.40, p = .020, see Table 1). This is consistent with Hypothesis 1a as well as the literature on framing effects for individuals (e.g., Reference KühbergerKühberger, 1998; Reference Tversky and KahnemanTversky & Kahneman, 1981). Chi-square tests yielded effect sizes of Cramer’s ϕ = .269 for the main effect of frame, Cramer’s ϕ = .086 for communication medium.
3.2 Group decisions
To test for effects of communication medium on group decisions, we ran a logistic regression with the three main effects of frame, medium and prior consideration as well as all interactions as independent variables. Hypothesis 1b was confirmed: We found a main effect of frame, Wald χ2(1, N = 64) = 6.49, p = .01, such that groups in the loss frame more frequently picked the risky option (84.8%) than groups in the gain frame (55.9%). There was no main effect of communication medium, Wald χ2(1, N = 64) = .24, n.s., and no effect of prior consideration, χ2(1, N = 64) = .41, n.s. We also found no significant interactions: all Wald χ2(1, N = 64) ≤ 1.00, n.s. (for all frequencies, see Table 1). Thus, as hypothesized, we did not find any significant effect of communication medium nor any interaction of medium with framing or with prior consideration on group decisions (Hypothesis 3). The fact that the effect of frame was significant in the combined sample (and not in the FtF sample alone; Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al., 2009) illustrates the importance of using sufficiently large samples for research on group decisions. It also shows that our sample was large enough to pick up significant differences of our manipulations, and thus that the lack of (interaction) effects of medium was not due to lack of power. Chi-square tests yielded effect sizes of Cramer’s ϕ = .296 for the main effect of frame, Cramer’s ϕ = .046 for communication medium, and Cramer’s ϕ = .021 for prior consideration.
3.3 Discussion time and thought-units
In order to determine whether there were differences in the discussion time for the group decisions by communication medium, we conducted a 2 (frame: loss vs. gain) × 2 (prior consideration: predecided vs. naïve) × 2 (communication medium: online vs. FtF) ANOVA on discussion time. The analysis yielded a trend toward shorter discussions in the online groups (M = 3.45 minutes, SD = 2.05) than in the FtF groups (M = 4.70 minutes, SD = 2.89; F (1, 56) = 3.68, p = .06).
We then conducted a 2 (frame: loss vs. gain) × 2 (prior consideration: predecided vs. naïve) × 2 (communication medium: online vs. FtF) ANOVA on number of thought-units. This analysis revealed a main effect of communication medium on number of thought-units, F (1, 56) = 13.56, p = .001. Discussions by the online groups contained more thought-units (M = 153.55, SD = 76.17) than those of the FtF groups (M = 93.33, SD = 56.42). We found no other significant effects.
We then created a new variable, dividing the numbers of thought-units per group by their discussion time. This yields a variable that represents the number of thought-units coded per minute of group discussion. When we analysed thought-units per minute to the same ANOVA, this yielded a significant main effect for medium (F(1, 56) = 56.50, p = .000), and no other significant effects. Discussions by the online groups contained more thought-units per minute (M = 50.46, SD = 30.05Footnote 1) than those of the FtF groups (M = 21.90, SD = 7.26). This analysis confirmed both our hypotheses that there would be a higher density of thought units in the online sample (Hypotheses 2a and 2b), and our hypothesis that medium does not interact with any of the other independent variables (Hypothesis 3).
4 Discussion
The main goal of our study was to compare group processes and decisions for online groups with FtF groups. Our study (1) provides insight into the effects of framing on group decision-making and (2) gives us the opportunity to compare the increasingly common medium of online collaboration with a more traditional face-to-face format. Our study suggests that web-conferences can be used as substitutes for FtF communication in group decision-making research. We found no significant differences in results based on communication medium (FtF vs. online). The only effects of medium that we did find, on discussion time and number of thought-units, actually suggest that online groups may be the preferred approach for research, since discussions were shorter, but number of thought-units was higher, indicating a more efficient use of time.
Our results supported Hypotheses 1a and b, which predicted similar effects of framing on decisions for both the online and the FtF samples. For our online groups, we found significant effects of framing on individual and group decisions, replicating the results obtained by Reference Milch, Weber, Appelt, Handgraaf and KrantzMilch et al. (2009) with FtF groups. The majority of individuals and groups in the loss frame chose the risky option, whereas the individuals and groups in the gain frame were approximately equally as likely to choose the risky option as the riskless option.
Some differences between media on process variables were hypothesized and found, in particular a larger number of thought-units and a shorter discussion time for the online groups (Hypothesis 2a and b). These did, however, not influence the group decisions. This supports our reasoning that people compensate for a lack of nonverbal communication by verbalizing their messages. Finally, there were no interactions between communication medium and any of our other manipulations (Hypothesis 3). In combination, our results show that our design was powerful and sensitive enough to pick up effects of experimental manipulations, but still did not show any moderating influence of medium on any of our other independent variables.
Whereas early literature (Reference Arrow, Berdahl, Bouas, Craig, Cummings and LebieArrow et al., 1996; Reference Lebie, Rhoades and McGrathLebie, Rhoades, & McGrath, 1996) tended to emphasize the impoverished nature of non-face-to-face communication, recent investigations have shown that more modern forms of technology-mediated communication can match face-to-face communications on such dimensions as amount of information transferred between people (Reference Derks, Fischer and BosDerks et al., 2008). This most likely stems from both the increasing richness of current forms of non-FtF communication and participants’ increased exposure to such media in their daily lives. As hypothesized, we found that groups in the online sample produced a significantly larger number of thought-units than groups in the FtF sample. This result suggests that one advantage of the online method is its increased ability to capture content of the group decision process, because the absence of nonverbal cues (which are very difficult to code for FtF groups) increases the need to vocally communicate information to other group members. Although one might expect some subtleties of communication to be lost when interacting via the Internet, people seem to adjust to and compensate for these losses by verbalizing thoughts that would appear as nonverbal cues in FtF. Instead of raising an eyebrow in an FtF interaction, for example, a participant in an online group must express doubt by a verbal statement, which is much less ambiguous.
Another advantage of running decision-making studies online, is that while there were more thought-units expressed by the online groups, the group discussions tended to be of shorter duration than the group discussions in the FtF sample. This higher density of verbal information transmitted by online group members may reflect a greater degree of task focus than in the FtF context. This increase in task focus may be linked to a decrease in non-task related interaction (e.g., social interaction). Despite these differences in the two communication media, results were very similar for framing effects in the group decisions across the two samples. These findings support the use of the online group decision method as a viable alternative to FtF communication. What is more, this result supports the notion that communicators compensate for the lack of nonverbal communication by adapting their communication style and levels of effort (Reference Walther and BazarovaWalther & Bazarova, 2008).
Of course it is important to keep in mind that we only investigated one type of online interaction, and one type of decision, but it seems likely, in light of other findings that show a lack of differences between FtF and CMC in decision making (Reference Walther and BazarovaWalther and Bazarova, 2008; Reference Derks, Fischer and BosDerks et al., 2008), that for many decision-making tasks, i.e., tasks in Quadrant 2 of Reference McGrathMcGrath’s (1984) model, our findings will hold. It should be noted that we do consider it likely that some types of decisions, even within this quadrant, may reveal differences between FtF and web-confercing, for instance when information density and task focus are very important determinants of decision-outcome. Also, it may be that other dependent variables, such as satisfaction or group cohesion may be differentially influenced by manipulations of medium. Similarly, there may be moderators that will interact with medium, if these differences in information density, task-focus or satisfaction are important determinants of these moderators’ effects.
One problem of our current analysis is the fact that the FtF group data and the online group data are strictly speaking gathered in two separate experiments: we did not run the FtF and online groups simultaneously. Moreover, participants were not recruited in an identical way: the online participants were recruited via email through Columbia University Center for Decision Sciences’ online Virtual Lab subject pool, whereas the FtF participants were recruited through flyers, email, and personal communication and came from clubs, organizations, and work teams from Columbia University or other locations in or near New York City. Participants in the online groups were strangers to each other, whereas many of the participants in the FtF groups were not, as they were recruited from campus organizations. This introduces a confound into the design, which is problematic for our results where they pertain to differences between media, since it is possible that such any observed differences are instead due to differences in familiarity. Our results fortunately suggest that this is not the case, since there is no reason to predict a higher density of thought units for the groups not familiar with each other, which is what we observed. The use of individuals not familiar with each other in the online sample, makes our tests of differences in choice outcomes conservative, i.e., lowers the likelihood of finding the similarities that confirm our hypotheses. More importantly, the problem of a potential confound only holds for our prediction about differences in process variables between groups (Hypotheses 2a/b), not for our other hypotheses. Also, it should be noted that we did make sure we used the same procedure and materials (to the extent that this was possible). Both experiments were run by the same team of researchers (with the addition of two junior researchers).
The online format we investigated in this study was partly born out of pragmatic and logistic considerations. The insights derived from group research are typically achieved at high costs, namely the frustrations and inefficiencies of recruiting and running groups for FtF laboratory interactions. Using online groups overcomes this problem by substituting FtF interaction for teleconferencing in combination with a web interface; this not only makes recruitment for participation and coordination of joint meeting times easier, but it also allows for a far more representative cross-section of participants than the typical on-campus undergraduate sample. In addition, companies exist that specialize in facilitating these kinds of online meetings by offering services that host, record, and transcribe these meetings, relieving researchers of the time, effort, and technical expertise required to execute these component tasks. The ability to conduct group decision making research online should yield a substantial gain in efficiency via an easier data collection process, as evidenced by the fact that it took us only a couple of weeks to run the online part of our study, whereas the FtF groups took over 6 months. Moreover, external validity increasingly mandates that we study groups in virtual interactions. Our results show that the two communication formats yield similar results, which means that previously established results for FtF groups are likely to hold also for online groups. We hope this paper adds to the toolbox available to researchers by opening new avenues for more efficient group decision-making research and lowering the hurdle for group decision research.
Open access
