Hypotheses

In this section, I developed hypotheses regarding tolerance for women's opinions and the expected impact of providing the normative message. As discussed, women are often undervalued because of their gender, even when they have abilities similar to men (Boring, Reference Boring2017; Hechtman et al., Reference Hechtman, Moore, Schulkey, Miklos, Calcagno and Aragon2018; Bosquet et al., Reference Bosquet, Combes and García-Peñalosa2019; Mengel et al., Reference Mengel, Sauermann and Zölitz2019; Huang et al., Reference Huang, Gates, Sinatra and Barabási2020; Ayalew et al., Reference Ayalew, Manian and Sheth2021; Ersoy and Pate, Reference Ersoy and Pate2023). This undervaluation is likely influenced by social norms that are unfavorable to women, which may lead to the devaluation of their abilities and opinions.

Although many studies have indicated that norm enforcement helps to sustain cooperation in society (Fehr and Gächter, Reference Fehr and Gächter2000; Gürerk et al., Reference Gürerk, Irlenbusch and Rockenbach2006), social norms can also exacerbate gender disparities (Gneezy et al., Reference Gneezy, Leonard and List2009; Field et al., Reference Field, Jayachandran and Pande2010; Alesina et al., Reference Alesina, Giuliano and Nunn2013; Bertrand et al., Reference Bertrand, Kamenica and Pan2015; Jayachandran, Reference Jayachandran2015, Reference Jayachandran2021). Specifically, when social norms dictate behaviors and ideologies that disadvantage women, such as prescribing how they should act, look, think and feel, they perpetuate gender inequalities (Cislaghi and Heise, Reference Cislaghi and Heise2020). Consequently, even individuals who personally hold egalitarian views may conform to unequal norms under societal pressures.

This dynamic is particularly pronounced in patriarchal cultures like Japan, where norms favoring men over women are deeply entrenched (Hamada, Reference Hamada2024). In such contexts, behaviors perceived as assertive or challenging to male opinions, especially when exhibited by women, are socially stigmatized (Seguino, Reference Seguino2000; Jayachandran, Reference Jayachandran2015; Lecoutere et al., Reference Lecoutere, d'Exelle and Van Campenhout2015). As a result, women's opinions and ideas may receive less acceptance, regardless of their quality. This argument leads to the first hypothesis:

In contrast, previous literature suggests that providing general normative messages can lead to various prosocial behaviors (Dimant et al., Reference Dimant, Van Kleef and Shalvi2020; Takahashi, Reference Takahashi2021a, Reference Takahashi2021b; Takahashi and Tanaka, Reference Takahashi and Tanaka2021; Bhattacharya and Dugar, Reference Bhattacharya and Dugar2022). Moreover, some studies demonstrate that normative messages specifically addressing gender equality can stimulate gender-egalitarian behavior (Boring and Philippe, Reference Boring and Philippe2021; Okuyama, Reference Okuyama2021).

One reason the provision of normative messages promotes gender-egalitarian behavior is by correcting misperceptions through these messages. In societies with unequal gender norms, individuals may support gender equality but misperceive societal norms due to prevailing stereotypes or biases. In such cases, transmitting normative messages endorsing gender-egalitarian behavior can correct these misperceptions and promote attitudes aligned with gender equality (Cislaghi and Heise, Reference Cislaghi and Heise2020). If gender inequality exists in the tolerance for opinions due to such misperceived norms, providing the normative message could mitigate this inequality. Accordingly, I propose the following hypothesis:

Experimental design and data collection

To test the above hypotheses, I conducted two online randomized experiments in Japan. The first experiment was conducted on August 3 and 4, 2021, targeting 1,600 individuals through the online survey platform ‘iResearch’.Footnote ¹ I conducted a second survey a month later (between September 3 and 7) to construct panel data. Although I invited 1,000 participants from the first survey, only 774 participated in the second survey (attrition rate was 22.6%). For each survey, participants received a participation allowance of 35 yen (approximately US$0.35), which is the standard price fixed by the survey company. The participants took an average of 6 min to complete the two tasks: (1) a demographic questionnaire survey and (2) evaluation of an anonymous statement.Footnote ²

Evaluation of anonymous statements

The main objective of this study was to identify whether people changed their attitudes toward statements depending on the gender of the statement poster. For this purpose, I asked the participants to evaluate their preferences for anonymous statements at the end of the survey.

Specifically, the participants were informed that 10 statements would be presented on the screen one at a time, and all statements were made by anonymous individuals. Table 1 shows the 10 statements used in the first and second surveys. Statements 1 through 4 in Table 1 are related to gender equity (hereafter, ‘gender-sensitive statements’). The first three statements focus on gender equity issues, which are of significant social concern in Japan. If male-preferential social norms or conservative beliefs are entrenched, people may strongly resist accepting these statements when made by women. In contrast, the fourth statement, which posits that women should stay home to raise their children, was included as a male-preferential opinion.

Table 1.

Ten statements presented during the first and second surveys

Note: The same statements were used consistently in the first and second surveys. The order of the statements presented to the participants was randomized.

Statements 5 through 8 pertain to environmental issues. Given the pervasive gender assumption that women are innately compassionate toward protecting the environment (Lau et al., Reference Lau, Kleiber, Lawless and Cohen2021), it is possible that pro-environmental statements made by women may be more readily accepted without strong opposition. The last two statements were included as neutral statements that are not considered to be particularly affected by gender.

To avoid ordering effects, the order of the statements presented to the participants was randomized. Then, the participants were asked to rate how much they agreed or disagreed with each statement based on a 7-point scale (hereafter, ‘agreement score’), ranging from ‘Strongly disagree’ to ‘Strongly agree’. For the analysis, I set a response of neither agree nor disagree as zero, while ‘Strongly disagree’ and ‘Strongly agree’ answers were scored −3 and 3, respectively.Footnote ³

Overview of the experimental design

Figure 1 shows an overview of the experimental design of this study. In the first survey, 1,600 participants were randomly assigned to one of the four groups. A total of 600 out of 1,600 participants received one or two interventions (groups 1–3 in Figure 1). It is important to note that for groups 1 and 3, which received the normative message treatment, the normative message was presented before participants were asked to evaluate their agreement with the 10 statements. The remaining 1,000 participants did not receive any intervention and served as the control group.

Figure 1.

Experimental design overview. Notes: The two interventions (i.e., the disclosure of poster's gender and the provision of normative message) are illustrated in dash box. Numbers in parentheses indicate the number of observations.

Since the control group participants in the first survey were not exposed to the interventions, they were exclusively invited to participate in the second survey to assess how the treatments influenced changes in agreement scores. A total of 774 individuals participated in the second survey, resulting in an attrition rate of 22.6%. This attrition rate is moderate compared with the average attrition rate of approximately 15% reported for field experiments (Ghanem et al., Reference Ghanem, Hirshleifer and Ortiz-Beccera2023). For the second survey, participants were again randomly assigned to one of four groups and were asked to rate their agreement with the same set of 10 statements presented in the initial survey.

The participants' demographic characteristics and the balance between the groups are reported in Supplementary Appendix B. Scheffe's multiple comparison test confirmed that there were no statistical differences in the average demographic characteristics between the four groups. Table 2 reports the average agreement scores of the groups for the first and second surveys. The total averages of agreement score were 0.85 and 0.86 in the first and second surveys, respectively. In both surveys, the average agreement score was relatively smaller for the groups with the gender disclosure treatment (i.e., both treatments and gender disclosure groups). While Table 2 presents the mean scores for the 10 statements at the individual level, the estimations use the agreement score for each statement as the dependent variable.

Table 2.

Average agreement scores at the individual level by the groups

Note. Standard deviations are in parentheses. The gender of the poster is disclosed only for the participants in the both treatments group and gender disclosure group.

Methodology

To identify how the agreement level was affected by the gender of the poster, this study employed both cross-sectional and panel data analyses. First, I started with a prefecture-level fixed effects regression model using observations from the first survey (cross-sectional analysis), as follows:

(1)$${\rm Scor}{\rm e}_{ij} = \alpha + \beta _1{\rm Femal}{\rm e}_{ij} + \beta _2{\rm Mal}{\rm e}_{ij} + \beta _3{\rm Nor}{\rm m}_i + \beta _4( {{\rm Femal}{\rm e}_{ij} \times {\rm Nor}{\rm m}_i} ) \\ + \beta _5( {{\rm Mal}{\rm e}_{ij} \times {\rm Nor}{\rm m}_i} ) + \gamma {\rm Stat}{\rm e}_j + \delta X_i + \rho _i + \varepsilon _{ij}, \;$$

where Score_ij is the agreement scale ranging from −3 to 3 for statement j for individual i. Female_ij and Male_ij are the dummy variables representing the gender disclosure treatment of individual i for statement j (hereafter, ‘female disclosure dummy’ and ‘male disclosure dummy,’ respectively). More precisely, Female_ij takes a value of 1 if the gender of the poster of statement j is disclosed as female, while Male_ij takes a value of 1 if the gender of statement j is disclosed as male to individual i. Norm_i is a dummy variable that takes a value of 1 if individual i receives the normative message. In Equation (1), I include two interaction terms between each gender disclosure dummy and the normative message dummy, shown as Female_ij × Norm_i and Male_ij × Norm_i. State_j denotes a set of dummy variables for each statement. X_i indicates a set of observable demographic characteristics of individual i (see Supplementary Appendix Table B1). ρ_i is the prefecture-specific fixed effect for individual i, which reduces the unobserved time-invariant differences between prefectures. Standard errors are clustered at the treatment level to account for autocorrelations in the error term ɛ_ij.

Next, I conducted an individual fixed effects model using panel data:

(2)$${\rm Agreemen}{\rm t}_{ijt} = \alpha + \beta _1{\rm Femal}{\rm e}_{ijt} + \beta _2{\rm Mal}{\rm e}_{ijt} + \beta _3{\rm Nor}{\rm m}_{it} + \beta _4( {{\rm Femal}{\rm e}_{ijt} \times {\rm Nor}{\rm m}_{it}} ) \\ + \beta _5( {{\rm Mal}{\rm e}_{ijt} \times {\rm Nor}{\rm m}_{it}} ) + \upsilon _i + \tau _t + u_{ijt}, \;$$

where t is the time of the survey round. υ_i and τ_t represent individual-specific fixed effects and time dummy. Note that Equation (2) excludes time-invariant variables.

In addition to Equations (1) and (2), I performed the regression by excluding the gender-sensitive statements (statements 1 through 4 in Table 1). The potential concern with including gender-sensitive statements is that the results, particularly for the gender disclosure treatment, could be affected by social desirability bias, male backlash or other confounding effects when participants encounter gender-sensitive statements. In addition, I excluded each statement from the observation and performed the estimation, which results are provided in Supplementary Appendix C.

Hypothesis 1 is tested by examining whether the agreement score is decreased when the gender of the poster was disclosed as female compared with when the poster's gender is disclosed as male. In Equations (1) and (2), the non-disclosure of the statement poster's gender is set as the baseline category. Therefore, β ₁ in Equations (1) and (2) represents the difference in agreement scores when the poster's gender is disclosed as female compared with when it is not disclosed. Since the exact same statements were presented to all participants in both surveys, if the agreement scores were determined solely by the content of the statements, the scores should be similar regardless of the disclosed gender (i.e., β ₁ = β ₂). However, if unfavorable social norms reduce tolerance for women's opinions, the coefficient for the female disclosure dummy is expected to be lower than that for the male disclosure dummy (β ₁ < β ₂).

Meanwhile, as proposed in Hypothesis 2, providing the normative message may mitigate the influence of social norms and improve tolerance for women's opinions. The general impact of normative message is captured in β ₃, while the study focuses on its specific impact on the agreement score for women's statements, indicated by the interaction term between the female disclosure dummy and the normative message dummy (β ₄). A positive β ₄ would suggest that the normative message helps correct the misperception of norms.

If the normative message treatment successfully offsets the underestimation of women's opinions, there should be no significant gender difference in the agreement scores. This is expressed as β ₁ + β ₄ = β ₂ + β ₅, meaning that the negative effect of disclosing a female poster's gender (β ₁) combined with the positive effect of the normative message (β ₄) should equal the effect of disclosing a male poster's gender (β ₂) combined with the normative message (β ₅). In other words, the normative message should neutralize the gender bias, resulting in no difference in agreement scores between genders.