THE TENSION BETWEEN REGRESSION ANALYSIS AND CONTEXT DEPENDENCY

At the core of regression models is estimating to what extent two variables (controlled for other factors) vary together and how likely it is that this correlation holds beyond the sample of studied units (e.g., voters). This can be in tension with context dependency for two reasons.

First of all, the goal of regression analysis is to draw conclusions that are generalizable beyond the cases studied. Random sampling—assuming no sampling or response bias—assures that results can be generalized to the cases the sample was drawn from, such as the U.S. or a European population.Footnote ² Our theories, however, are often formulated more generally with, for example, all industrialized Western democracies falling within a theory's scope. Generalizing results of a study beyond the population it samples from—for example, generalizing a U.S. sampled study to European countries—brings in the assumption that the core causal mechanisms are the same across contexts.

Second, the more cases that are included in the analysis, the higher the certainty (statistical significance) that the relationships found in the sample are also present in the larger population.Footnote ³ This makes pooling data attractive, as it increases “certainty.” In such studies it is common (and good) practice to control for different average levels on the dependent variable across countries. However, this practice does not account for different effects across countries—context dependency (Spierings Reference Spierings2012). Models based on pooled data might present the average effect across countries accurately, but they do not show whether the effect is actually the same in each country. Moreover, different sample sizes can lead to the results being dominated by the relationship in one oversampled or populous country.Footnote ⁴

FALSE SOLUTIONS

The tension between regression practices and context dependency is particularly relevant for gender studies, as we start from the notion that the social meaning and consequences of one's sex are different across countries and that the structural and cultural context (e.g., welfare state regimes and societal norms) shape this meaning.

The ways in which this issue is dealt with are diverse. Before turning to some real solutions, I discuss three archetypical ways of dealing with context dependency that hardly help to develop our understanding of its (gendered) dynamics, but of which derivatives and/or combination are regularly found, as illustrated with figures for the 2011–2015 volumes of this journal:

1. 1. The simplest way to deal with the possibility of relationships being context dependent is ignoring the issue. It is surprising how many studies frame their conclusions in general terms while not reflecting on the possibility that the results might be different in other countries (or context) or between the countries studied, and for what reasons. In the 31–40% of all reviewed studies this was the case; all were single-country studies.Footnote ⁵

2. 2. The second archetypical solution is claiming that the case(s) studied is (are) representative or, in the case of pooled samples, that results are reliable because the cases are highly similar or because they are rather diverse—both arguments are used. However, given the differences in political systems, cultural norms, welfare state regimes, and economic structures, there is generally good reason to expect that the strength or even direction of a relationship differs between countries. Albeit, this practice is not uncommon; examples were found in 15% to 20% of the single-country studies that did not ignore the issue overall (see above) and in five of the six reviewed pooled studies.

3. 3. The third response is the opposite of the second one: explicitly mentioning that the results are context specific and cannot be generalized. While this fits the argument that is made by some qualitative-oriented or country-specialist scholars (for instance, in the role of reviewer), these reservations are often rather empty. It does not provide a thought-through reasoning of why and how the results can be expected to be different in other spatial or temporal situations, which the authors as experts in their field should be able to provide. Even though claims about “absolute context specificity” might not be expected in quantitative studies, traces of this archetype were found in 4 of the 26 single-country studies reviewed.

These three responses to potential context dependency are cosmetic or even dogmatic and tell us little about how the context influences the gender dynamics in a particular domain.

THEORIZING AND MODELING CONTEXT DEPENDENCY

The “solutions” to the issue of potential context dependency above lack theoretical specification and empirical testing. Here, I will discuss three better ways of dealing with context dependency when applying regression analyses.

Positioned statistical case study. There can be very good reason to study data on only one country. One of those reasons is actually that one has more space to go into detailed descriptive backgrounds regarding the context and the position the country takes in the population of countries (see Seawright and Gerring Reference Seawright and Gerring2008). For instance, if a theory about political participation is said to hold for democracies, all democracies form the population. However, results of a study on a two-party democracy can be expected to differ from those of multiparty systems. Contextualizing the results of a single-country study and thinking through the implications of the contextual situation for the causal mechanisms specified in the theory provides a fairer and more reliable test of that theory. Or, if the mechanisms in the theory do not provide enough information for a priori expectations, the author can think through how the contextual circumstances have influenced the strength of the relationship found and generate new tangible expectations for other cases, even though these cannot be tested in that particular study. Three strong examples of positioned-statistical case studies among the reviewed articles were studies on Denmark (Bækgaard and Kjear Reference Bækgaard and Kjaer2012), Sweden (Folke, Freidenvall, and Rickne Reference Folke, Freidenvall and Rickne2015), and the UK (Bicquelet, Weale, and Bara Reference Bicquelet, Weale and Bara2012).

Context-disaggregated analyses. If multiple datasets are available, pooled analyses can always be rerun per contextual unit to test how robust the pooled relationships are. Evidently, in disaggregate analyses the number of cases is lower, leading to fewer statistically significant relationships, but coefficients per country can be compared to the coefficients from the pooled model on their substantial significance (see Miller and van der Meulen Rodgers Reference Miller and van der Meulen Rodgers2008).Footnote ⁶ Even if surveys on only a few contexts (space or time) are available, this provides a preliminary assessment of context dependency or might generate theoretical ideas on which contextual characteristics matter (e.g., Spierings and Zaslove Reference Spierings and Zaslove2015). This approach is less common; among the reviewed studies, Schwindt-Bayer (Reference Schwindt-Bayer2011) is the best example.

Multilevel-interaction models. If surveys on at least 25 contextual units (e.g., World Value Surveys; European Social Surveys) and data on these units are present, it is possible to systematically assess whether and why gendered relationships are different by context. The disaggregation approach discussed above cannot ascribe the differences to specific characteristics of the context, such as the public childcare budget, and it cannot rule out other moderating contextual characteristics. Multilevel regression models (MLM) do allow for this. MLM allows testing how variations in strength of microlevel relationships across higher-level units correlate with differences in contextual characteristics (e.g., whether variation in the employment-political participation linkage across years, countries, or states correlates with differences in welfare expenditures), without the risk of overestimating statistical significance. Alexander (Reference Alexander2012) and Luhiste (Reference Luhiste2015) are clear examples from the Politics & Gender articles reviewed.

MLM is clearly the most demanding one in terms of data and analysis as well as the most systematic one. At the same time, disaggregated models in combination with case knowledge can be particularly useful for hypothesis-generating approaches or exploring unexpected results.

GENDER AND MULTILEVEL MODELS

Because context dependency is intrinsically part of the concept of gender, the gender studies literature can most easily embrace and push our theoretical understanding of context dependency. This can be done in multiple ways if the approaches above, MLM particularly, are embraced.

We should go beyond simply claiming that patriarchy and gender regimes take different shapes in different times and places, and start specifying the dimensions of patriarchy and theorizing and testing how they shape the gendered relationship at the individual level. For instance, how do different patriarchal institutions influence the impact of family formation on women's social participation (e.g., Spierings Reference Spierings2014). In other words, we should go beyond claiming context dependency and start explaining context dependency. This implies ruffling of old paradigmatic feathers that withhold a constructive discussion about using regression models for understanding context dependency (see Spierings Reference Spierings2012).

Second, while politics and gender studies tend to focus on how clearly gendered contextual variables, such as the proportion of women in Parliament (Alexander Reference Alexander2012), shape individual-level relationships, MLM also stimulates us to formalize, systematize, and demonstrate claims about ostensible gender-neutral structures having gendered implications for individual-level relationships. For instance, it is generally acknowledged that the type of electoral system has gendered implication for representation, but how do differences between PR and majoritarian systems influence the impact of individual characteristics on gender differences in political participation? MLM challenges gender scholars to further specify in what way gender neutrality is a myth.

Lastly, gender studies have a strong track record in qualitative methods, leading to vast in-depth case knowledge on gender dynamics. The field is therefor particularly suited to show how mixed-methods studies and projects can combine regression analyses with case studies in order to develop systematic theories on context dependency. For instance, disaggregated regression models can show for which countries a relationship does not hold. In-depth case knowledge can then provide explanations that are translated to more general expectations, which subsequently can be tested in multilevel models.

CONCLUDING NOTE

In this contribution, I have reflected on two practices: (1) the gender criticism of not acknowledging spatiotemporal context dependency in quantitative studies and (2) the application of the gender-based theoretical notion of context dependency in regression models. It is my hope that combining gender studies’ in-depth situational knowledge with multilevel regression models will stimulate gender scholars to further develop their arguments on which aspect of the context actually influences which microlevel relationships and stimulate multilevel modelers in political and other social sciences to reflect on and test the internal and external generalizability of their results.