2 The Sources of and Potential for Dynamic Misspecification

Applied researchers often perceive serially correlated errors as noise rather than information (DeBoef and Keele Reference DeBoef and Keele2008). Yet, serially correlated errors clearly indicate a potentially severe model misspecification, which can result from various sources (Neumayer and Plümper Reference Neumayer and Plümper2017). Perhaps most obviously, serially correlated errors are caused by incompletely or incorrectly modeled persistency in the dependent variable, time-varying omitted variables or changes in the effect strengths of time-invariant variables, or misspecified lagged effects of explanatory variables. Conditionality makes modeling dynamics more complicated (Franzese Reference Franzese2003a,Reference Franzeseb; Franzese and Kam Reference Franzese and Kam2009). Few empirical analyses model all potential conditioning factors of the variables of interest. If, however, treatment effects are conditioned by unobserved time-varying factors—as for example the effect of higher education on income is conditioned by structural change of the economy and ruptures in economic policies—then treatment effects vary over time, and the strength of these effects also changes over time as un-modeled conditioning factors change. Finally, serially correlated errors may result from misspecification that at first sight have little to do with dynamics, for example from spatial dependence. Yet, spatial effects are certainly misunderstood if they are perceived as time-invariant, ignoring spatial dependence causes errors to be serially correlated (Franzese and Hays Reference Franzese and Hays2007). Virtually all of these complications depend on an arbitrary decision that no researcher can avoid: the periodization of continuous time that is a necessary condition if researchers wishing to study ‘periods’. If researchers choose relatively short periods, effects do no longer necessarily occur in the same period as the treatment. If periods cover a long stretch of time, the probability that estimates are biased by confounders rises quickly. In the social sciences, the lengths of a period is rarely chosen to optimize the analysis. Instead, social scientists often have to accept data that is collected on a daily, monthly or—most often—annual basis.

At least in an optimal world these model misspecifications should be avoided: dynamics should be directly modeled to obtain unbiased estimates. This proves to be difficult. Since dynamic misspecifications are manifold and complex, econometric tests for ’dynamics’ at best reveal serially correlated errors, but they are usually unable to identify the underlying root causes of autocorrelation. Often, these tests are also weak and do not reveal the true dynamic structure of the data-generating process, which may lead to overfitting of the data (Keele, Linn, and Webb Reference Keele, Linn and Webb2016). Thus, empirical researchers are probably best advised to simplify their empirical model and to treat problems such as serially correlated errors with straightforward econometric patches such as lagged dependent variables, period dummies, and simple homogeneous lag structures.

Yet, using misspecification patches should not mislead researchers into believing that the dynamics of their model are correctly specified. Econometric fixes are not correct per se because they are usually not modeling the true dynamic process in the underlying data-generating process. For example, periods do not exert a direct effect on the dependent variable but period dummies capture variation over time which can help to “clean” residuals. Perhaps even worse, more than one econometric specification allows eliminating serial correlation, and there is no guarantee that different models lead to identical or at least sufficiently similar results. Empirical researchers should also not expect that so called ‘dynamic econometric models’, e.g. the Arellano–Bond (A–B) estimator (Arellano and Bond 1991), solve various problems of dynamics. Dynamic panel models only eliminate Nickell-bias. Period dummies control for common trends, common shocks, and common breaks, but they do not perfectly account for unit-specific, heterogeneous trends, shocks, and breaks. Including a lagged dependent variable to the right hand side of the estimation equation without including lags of the explanatory variables ( $x$ ) assumes that the dynamics of all independent variables are identical. These assumptions are convenient, but not always plausible.

Still, the vast majority of panel data analyses pushes serially correlated errors into uninformative econometric patches: lagged dependent variables and period fixed effects appear to be the most common solutions, but they are not the only ones. More often than not analysts seem to “adopt restrictive dynamic specifications on the basis of limited theoretical guidance and without empirical evidence that restrictions are valid, potentially biasing inferences and invalidating hypothesis tests” (DeBoef and Keele Reference DeBoef and Keele2008, 184).Footnote ¹⁰ A review of recent political science publications reveals that a large majority of panel data analyses rely on of the following four strategies: do nothing and ignore the potential for dynamics (Humphreys and Weinstein Reference Humphreys and Weinstein2006; Ross Reference Ross2008), assume that all dynamics are captured by period fixed effects (Egorov, Guriev, and Sonin Reference Egorov, Guriev and Sonin2009; Besley and Reynal-Querol Reference Besley and Reynal-Querol2011; Menaldo Reference Menaldo2012 among many others), try to capture dynamics by a lagged dependent variable (e.g. Acemoglu et al. Reference Acemoglu, Johnson, James, Robinson and Yared2008; Guisinger and Singer Reference Guisinger and Singer2010; Lupu and Pontusson Reference Lupu and Pontusson2011; Kogan, Lavertu, and Peskowitz Reference Kogan, Lavertu and Peskowitz2016), or finally follow Beck and Katz (Reference Beck and Katz1995) and model dynamics by a combination of period fixed effects and a lagged dependent variable (e.g. Lipsmeyer and Zhu Reference Lipsmeyer and Zhu2011; Getmansky and Zeitzoff Reference Getmansky and Zeitzoff2014). Significantly fewer authors rely on GLS estimators (Mukherjee, Smith, and Li Reference Mukherjee, Smith and Li2009; Lupu and Pontusson Reference Lupu and Pontusson2011), distributed lag models (e.g. Gerber et al. Reference Gerber, Gimpel, Green and Shaw2011) or error correction models (Lebo, McGlynn, and Koger Reference Lebo, McGlynn and Koger2007; Kayser Reference Kayser2009; Soroka, Stecula, and Wlezien Reference Soroka, Stecula and Wlezien2015).Footnote ¹¹ Overall, the vast majority of panel data analyses in political science assumes rather simple dynamics.Footnote ¹² This finding is consistent with DeBoef and Keele (Reference DeBoef and Keele2008, 185), who also conclude that the vast majority of authors do not test for the underlying dynamic structure. Thus, social scientists often model the dynamic aspects with very little theoretical guidance (Keele and Kelly Reference Keele and Kelly2006; DeBoef and Keele Reference DeBoef and Keele2008),Footnote ¹³ use ad hoc econometric solutions, which make rather rigid assumptions, do not try to model the true data-generating process, and do not report results of minimal tests for serial correlation.

One strategy that may reduce the size of the problem is to use less constrained econometric solutions. Distributed lag models, models with a unit-specific lagged dependent variable, panel co-integration models, models with heterogeneous lag structure (Plümper, Troeger, and Manow Reference Plümper, Troeger and Manow2005), more attention to periodization (Franzese Reference Franzese2003a), better specified spatial models (Franzese and Hays Reference Franzese and Hays2007; Neumayer and Plümper Reference Neumayer and Plümper2016) may all reduce the size of the problem. However, as the number of possible dynamic specifications increases, a higher order problem of model selection arises: since all these different models likely generate different estimates and often demand different inferences, the question becomes how empirical researchers select their preferred model. To eliminate or at least reduce the arbitrariness of model selection, DeBoef and Keele (Reference DeBoef and Keele2008, 187) suggest a testing-down approach, starting with a full autoregressive distributive lag model and stepwise removing parameters according to pre-determined criteria, often the significance of parameters. This procedure will result in a dynamic specification that maximizes the variance absorbed by the minimum number of parameters. As with all testing-down approaches, this approach suffers from the arbitrariness in the choice of a start model because we do not have an infinite number of degrees of freedom. DeBoef and Keele recommend starting with a general autoregressive distributed lag (ADL) model. They argue that this model has ‘no constraints’. Yet, the model still assumes a homogeneous lag structure and it will run quickly out of degrees of freedom if the number of controls is large because a finite number of distributed lag parameters has to be estimated for each regressor. Accordingly, these models only work if the number of periods is much larger than the number of variables—a criterion that is not necessarily met in panel data analyses. Since the specification includes a lagged dependent variable, the estimator is inconsistent when unit fixed effects are included, though the bias declines if the number of periods increases (Nickell Reference Nickell1981; Kiviet Reference Kiviet1995). Gerber et al. (Reference Gerber, Gimpel, Green and Shaw2011) thus prefer an alternative strategy. Rather than relying on a single ‘best’ dynamic specification, they report the result of various different dynamic specification and they demonstrate that their results are robust “for varying lag lengths and polynomial orders” (Gerber et al. Reference Gerber, Gimpel, Green and Shaw2011, 143). Relying on robustness tests has at least two advantages: first, it largely reduces the necessity to make arbitrary dynamic modeling assumptions, and, second, it helps identifying possible relevant model uncertainties (Neumayer and Plümper Reference Neumayer and Plümper2017).

For our purposes and in the remainder of this article, the problem is not so much which techniques minimizes the potential for dynamic misspecification. Instead, we assume that dynamic misspecifications exist and analyze the performance of the fixed-effects model in the presence of various dynamic misspecifications—some of which could be dealt with easily, others are more difficult though not impossible to eliminate if only researchers knew the true data-generating process. But of course the whole point of estimation is that researchers do not know the true data-generating process and that theory, econometric tests, and testing-down procedures cannot identify the optimal model beyond reasonable doubt. Having said this, we do not claim that social scientists inevitably misspecify dynamics, but we emphasize that in the presence of dynamic misspecifications, the fixed-effects model has problematic properties. Needless to say that modeling dynamics correctly is always preferable.

3 The Bias of the Fixed-Effects Estimator with Dynamic Misspecification

This section analyzes how dynamic misspecifications cause fixed-effects estimates to be biased and we demonstrate that the bias of the fixed-effects estimator can exceed the bias of the naïve pooled-OLS estimator under plausible assumptions. We are not the first to do so. Lee (Reference Lee2012) analytically demonstrates that the fixed-effects estimator is biased when the lag order is not correctly chosen and stresses that “existing bias corrections would not work properly because the correction formulas assume correct model specification. In fact, attempts to adjust for the bias using formulas that correct for AR(1) dynamics would be wrong and may even exacerbate the bias when the true lag order is larger than one” (Lee Reference Lee2012, 57).

Misspecified lag structures are clearly not the only dynamic misspecification that biases fixed-effects estimates. Rather, fixed-effects estimates are likely to be biased in the presence of any dynamic misspecification or omitted time-varying variables.Footnote ¹⁴ Ahn, Lee, and Schmidt (Reference Ahn, Lee and Schmidt2013) argue that the fixed-effects estimator is biased when omitted variables vary over time and develop a generalized method of moments procedure that accounts for multiple factorial time-varying fixed effects. This estimator, however, requires the existence of instruments which are correlated with the dynamic fixed effects but not with the errors—an assumption that is unlikely to be satisfied and that cannot be tested properly since errors remain unobserved. Finally, Park (Reference Park2012) at least implicitly confirms the existence of bias in fixed-effects models with structural breaks and develops a Bayesian estimator that seeks to identify these structural breaks. As one would expect, a model that corrects for ‘turning points’ fits the data better than the classical fixed-effects estimator. We build on these contributions and prove that the bias from dynamic misspecifications can be larger for the fixed-effects estimator than for pooled-OLS. As we have already mentioned, we make this comparison not so much because we intend to rehabilitate the pooled-OLS estimator. Rather, we use this comparison to demonstrate how poorly the fixed-effects estimator performs in the presence of dynamic misspecifications.

3.1 Bias of the fixed-effects estimator induced by correlated within variance

Fixed-effects estimation accounts for potential bias from unobserved time-invariant variables by eliminating all between variation from the estimation. Obviously, the effect of variance that is dropped from the estimation cannot be biased by correlated confounders. And the correlation between the remaining within variation and omitted time-invariant cross-sectional variation is zero. Therefore, if the effect of omitted variables is really exclusively time-invariant, the estimates which rely on an analysis of the remaining time-varying variance does not suffer from omitted variable bias. This, of course, immediately changes when the aggregate effect of omitted variables is not strictly time-invariant.

In this section, we derive the causes of the bias of the dynamic fixed-effects estimator using a time-varying omitted variable as an example. We demonstrate that the bias of the fixed-effects estimate of $\unicode[STIX]{x1D6FD}$ exceeds the bias of the pooled-OLS estimate of $\unicode[STIX]{x1D6FD}$ when the correlation between $x$ and omitted within variation is larger than the correlation between $x$ and omitted between variation.

Assume that

(1) $$\begin{eqnarray}y_{it}=\unicode[STIX]{x1D6FC}+\unicode[STIX]{x1D6FD}x_{it}+u_{i}+\unicode[STIX]{x1D700}_{it}\end{eqnarray}$$

is the true data-generating process with $x_{it}$ a time-varying observed variable, $u_{i}$ a vector of time-invariant unobserved variables, and $\unicode[STIX]{x1D700}_{it}$ an i.i.d. error component. Note that the data-generating process is static. Estimating this model by a ‘naïve’ OLS estimator leads to bias if $\text{var}(\bar{x}_{i},u_{i})\neq 0$ , or $\text{var}(x_{it},\unicode[STIX]{x1D700}_{it})\neq 0$ .

The fixed-effects estimator eliminates the between variation from Equation (1) so that

(2) $$\begin{eqnarray}y_{it}-\bar{y}_{i}=\unicode[STIX]{x1D6FD}\left(x_{it}-\bar{x}_{i}\right)+u_{i}-\bar{u}_{i}+\unicode[STIX]{x1D700}_{it}-\bar{\unicode[STIX]{x1D700}}_{i}\end{eqnarray}$$

which is equivalent to

(3) $$\begin{eqnarray}y_{it}-\bar{y}_{i}=\unicode[STIX]{x1D6FD}\left(x_{it}-\bar{x}_{i}\right)+\unicode[STIX]{x1D700}_{it}-\bar{\unicode[STIX]{x1D700}}_{i},\end{eqnarray}$$

because $u_{i}-\bar{u}_{i}=0$ .

Assume now the following data-generating process

(4) $$\begin{eqnarray}y_{it}=\unicode[STIX]{x1D6FC}_{1}x_{it}+\unicode[STIX]{x1D6FC}_{2}w_{it}+u_{i}+\unicode[STIX]{x1D700}_{it};\quad \text{with }\unicode[STIX]{x1D6FC}_{2}=1\end{eqnarray}$$

where $x_{it}$ and $w_{it}$ are time-varying right-hand-side variables and $u_{i}$ is a unit-specific effect.

The omitted variable $w_{it}$ is correlated with the included right-hand-side variable $x_{it}$ . $\unicode[STIX]{x1D6FE}_{1}$ and $\unicode[STIX]{x1D6FE}_{2}$ indicate the strength of the correlation between $w_{it}$ and the within variance of $x_{it}$ and $w_{it}$ and the between variation of $x_{it}$ , respectively:

(5) $$\begin{eqnarray}w_{it}=\unicode[STIX]{x1D6FE}_{1}\ddot{x}_{it}+\unicode[STIX]{x1D6FE}_{2}\bar{x}_{i}+\unicode[STIX]{x1D714}_{it}\end{eqnarray}$$

with $\bar{x}_{i}=(1/T)\sum _{t=1}^{T}x_{it},\,\ddot{x}_{it}=\left(x_{it}-\bar{x}_{i}\right)$ .

Finally, the unit-specific effect $u_{i}$ covaries with the between variance of $x_{it}$ to a degree of delta.

(6) $$\begin{eqnarray}u_{i}=\unicode[STIX]{x1D6FF}_{1}\bar{x}_{i}+\unicode[STIX]{x1D708}_{i}.\end{eqnarray}$$

We omit $w_{it}$ from the estimation and can easily derive the biases for the fixed effects and the pooled-OLS estimators ( $\hat{\unicode[STIX]{x1D6FC}}_{1,FE}$ and $\hat{\unicode[STIX]{x1D6FC}}_{1,OLS}$ ) under the assumptions in (4)–(6). We also can demonstrate that under certain conditions the bias of fixed-effects estimates exceeds that of pooled-OLS estimates. Needless to say that neither of these two estimators is unbiased in case of time-varying omitted variables.Footnote ¹⁵

Conditional on all of the $x_{it}$ , Equation (7) derives the bias for the pooled-OLS estimator:

(7) $$\begin{eqnarray}\text{Bias}\left(\hat{\unicode[STIX]{x1D6FC}}_{1,OLS}\right)=\frac{\displaystyle \unicode[STIX]{x1D6FE}_{1}\mathop{\sum }_{i=1}^{N}\mathop{\sum }_{t=1}^{T}\ddot{x}_{it}^{2}+T\unicode[STIX]{x1D6FE}_{2}\mathop{\sum }_{i=1}^{N}\bar{x}_{i}^{2}}{\displaystyle \mathop{\sum }_{i=1}^{N}\mathop{\sum }_{t=1}^{T}\ddot{x}_{it}^{2}+T\mathop{\sum }_{i=1}^{N}\bar{x}_{i}^{2}}+\frac{\displaystyle \unicode[STIX]{x1D6FF}_{1}T\mathop{\sum }_{i=1}^{N}\bar{x}_{i}^{2}}{\displaystyle \mathop{\sum }_{i=1}^{N}\mathop{\sum }_{t=1}^{T}\ddot{x}_{it}^{2}+T\mathop{\sum }_{i=1}^{N}\bar{x}_{i}^{2}}.\end{eqnarray}$$

Equation (7) indicates that the OLS bias depends both on the correlation between $x_{it}$ and $w_{it}$ as well as the correlation between $u_{i}$ and $x_{it}$ .

As usual the bias for the FE estimator is given by:

(8) $$\begin{eqnarray}\text{Bias}\left(\hat{\unicode[STIX]{x1D6FC}}_{1,FE}\right)=\unicode[STIX]{x1D6FE}_{1}.\end{eqnarray}$$

The bias of the fixed-effects estimator depends on the correlation between $x_{it}$ and $w_{it}$ , but not on the covariance between the unit-specific effects $u_{i}$ and $x_{it}$ , because the within-transformation on which FE estimation relies, effectively eliminates all, endogenous and exogenous, between variation from the estimation.

If we assume that $\unicode[STIX]{x1D6FF}_{1}=0$ (no correlation between $u_{i}\,and\,\bar{x}_{i}$ ) and $\unicode[STIX]{x1D6FE}_{2}=0$ (no correlation between $w_{it}\,and\,\bar{x}_{i}$ ), then,

(9) $$\begin{eqnarray}\text{Bias}\left(\hat{\unicode[STIX]{x1D6FC}}_{1,OLS}\right)=\unicode[STIX]{x1D6FE}_{1}\frac{\displaystyle \mathop{\sum }_{i=1}^{N}\mathop{\sum }_{t=1}^{T}\ddot{x}_{it}^{2}}{\displaystyle \mathop{\sum }_{i=1}^{N}\mathop{\sum }_{t=1}^{T}\ddot{x}_{it}^{2}+T\mathop{\sum }_{i=1}^{N}\bar{x}_{i}^{2}}<\unicode[STIX]{x1D6FE}_{1}=\text{Bias}\left(\hat{\unicode[STIX]{x1D6FC}}_{1,FE}\right).\end{eqnarray}$$

In this case, for any given $T<\infty$ , the bias of the FE estimator that results from the omission of $w_{it}$ is larger than that of OLS. This is so because the fraction term of the OLS bias in Equation (9) is always smaller than 1.

This case might seem rare in real data but can emerge when neither $x_{it}$ nor $w_{it}$ have a specific dynamic structure (autocorrelation or trends) but only the variation over time and not across units of these two variables is related. An exogenous shock could have this property. Alternatively, $w_{it}$ has no between variation and represents an omitted common trend. More often, however, applied researchers specify empirical models that suffer from both, omitted between variation correlated with the regressors and omitted within variation correlated with the regressors. In these cases, one cannot say whether the fixed effects or the pooled-OLS estimator gives less biased estimates. One can ex ante know that both estimators give biased results, but which one is more reliable (or less unreliable) depends on the relative strengths of the correlations with the omitted variance. Unfortunately, these correlations cannot be observed. Researchers may often know that a relevant variable has been omitted, but one cannot know with certainty that no relevant variable has been omitted. Still, one can evaluate whether omitted variables are potentially more problematic for the included within or between variation by estimating how much of the within and between variance of the dependent variable remains unexplained.

Now consider a situation where $x_{it}$ (and $w_{it}$ ) follows a deterministic trend so that its within variance grows with increasing number of time periods, and approaches infinity $as\,T\rightarrow \infty$ . In this case, even if $\unicode[STIX]{x1D6FF}_{1}\neq 0$ (non-zero correlation between $u_{i}\,and\,\bar{x}_{i}$ ) the second term of the OLS bias Equation (7) approaches zero because the within variation ( $\ddot{x}_{it}^{2}$ ) grows but only appears in the denominator while the between variance ( $\bar{x}_{i}^{2}$ ) does not change. The bias that is caused by the correlation between $w_{it}$ and $x_{it}$ increases with $T$ because of the trend and will outweigh the bias induced by omitted time-invariant variables if $T$ grows large enough.

3.2 Bias from dynamic misspecification

Correlated within variation and common trends of included and excluded explanatory variables are obvious sources of omitted variable bias occurring in fixed-effects estimates. Yet, there are many examples of dynamic misspecifications that can cause bias. Assume a data-generating process representing the simplest form of dynamic misspecification, an explanatory variable that does not exert a contemporaneous effect on the dependent variable but a one period lagged effect:

(10) $$\begin{eqnarray}y_{it}=\unicode[STIX]{x1D6FD}x_{it-1}+u_{i}+\unicode[STIX]{x1D700}_{it}.\end{eqnarray}$$

If we estimate Equation (10) ignoring the lagged effect of $x_{it}$ , the probability limit (plim) of the OLS estimator of $\unicode[STIX]{x1D6FD}$ in the regression $y_{it}=\unicode[STIX]{x1D6FD}x_{it}+\unicode[STIX]{x1D700}_{it}$ is given by:

(11) $$\begin{eqnarray}\frac{\text{Cov}\left(y_{it},x_{it}\right)}{\text{Var}\left(x_{it}\right)}=\unicode[STIX]{x1D6FD}\frac{\text{Cov}\left(x_{it-1},x_{it}\right)}{\text{Var}\left(x_{it}\right)}+\frac{\text{Cov}\left(u_{i},x_{it}\right)}{\text{Var}\left(x_{it}\right)}.\end{eqnarray}$$

The second term of Equation (11) is similar to the bias of estimating a model without fixed effects while the true DGP has correlated unit effects: the estimated $\unicode[STIX]{x1D6FD}$ wrongly captures the unit-specific effects (unless $\bar{x}_{i}=0$ ).

The probability limit of the fixed-effects estimator equals:

(12) $$\begin{eqnarray}\frac{\text{Cov}\left(y_{it},\ddot{x}_{it}\right)}{\text{Var}\left(x_{it}\right)}=\unicode[STIX]{x1D6FD}\frac{\text{Cov}\left(x_{it-1},\ddot{x}_{it}\right)}{\text{Var}\left(x_{it}\right)}+\frac{\text{Cov}\left(u_{i},\ddot{x}_{it}\right)}{\text{Var}\left(x_{it}\right)}.\end{eqnarray}$$

The second term now vanishes since $\ddot{x}_{it}$ has no unit-specific mean. We can rewrite Equation (12) so that

(13) $$\begin{eqnarray}\displaystyle \frac{\text{Cov}\left(y_{it},\ddot{x}_{it}\right)}{\text{Var}\left(x_{it}\right)}=\unicode[STIX]{x1D6FD}\frac{\text{Cov}\left(x_{it-1},x_{it}\right)}{\text{Var}\left(x_{it}\right)}-\unicode[STIX]{x1D6FD}\frac{\text{Cov}\left(x_{it-1},\bar{x}_{i}\right)}{\text{Var}\left(x_{it}\right)} & & \displaystyle\end{eqnarray}$$

(14) $$\begin{eqnarray}\displaystyle \Leftrightarrow \frac{\text{Cov}\left(y_{it},\ddot{x}_{it}\right)}{\text{Var}\left(x_{it}\right)}=\unicode[STIX]{x1D6FD}\frac{\text{Cov}\left(x_{it-1},x_{it}\right)}{\text{Var}\left(x_{it}\right)}-\unicode[STIX]{x1D6FD}\frac{\text{Var}\left(\bar{x}_{i}\right)}{\text{Var}\left(x_{it}\right)}. & & \displaystyle\end{eqnarray}$$

The second term of Equation (14) equals $\unicode[STIX]{x1D6FD}$ multiplied by the between variance of $x_{it}$ divided by its total variance. The result will fall between 0 and 1. It follows that the probability limit of the within estimator (FE) is smaller than $\unicode[STIX]{x1D6FD}$ . The estimate will thus be downward biased and this bias increases as the share of ignored between variation in $x_{it}$ increases.

The total bias of the OLS estimator depends on the autocorrelation of $x_{it}$ and the bias induced by the omission of the unit-specific effects. If the majority of autocorrelations in real world data generation processes is positive (which seems to be the case), the bias of a fixed-effects estimator exceeds the bias of pooled-OLS. It is of course possible to estimate whether $x_{it-1}\,and\,x_{it}$ are positively correlated and how strong this correlation is. However, it is much more complicated to identify the correct lag structure of explanatory variables (Adolph, Butler, and Wilson Reference Adolph, Butler and Wilson2005; Plümper, Troeger, and Manow Reference Plümper, Troeger and Manow2005). Time series tests such as information criteria (BIC, AIC etc.) have low power in complex models and usually predict diverging lag lengths depending on the number of lags and right-hand-side variables included. The problem of misspecified lag length is exacerbated if the lag length is not uniform but varies across units which can occur frequently in political science data, for example because institutional settings will usually influence responsiveness of actors (Plümper, Troeger, and Manow Reference Plümper, Troeger and Manow2005). We analyze the effect of unit-specific lag length in the Monte Carlo experiments below.

4 Design of the Monte Carlo ExperimentsFootnote ¹⁷

Bias in fixed-effects estimation can result, inter alia, from omitted time-varying variables, from omitted trends, a misspecified lag structure, and other—more complex—dynamic misspecifications. Since social scientists often rely on standard dynamic specifications rather than on explicitly modeling the dynamics, bias may be reduced, but is unlikely to disappear. As we have shown in the previous section, the existence of any form of unaccounted within variation correlated with the regressors biases fixed-effects estimates. The Monte Carlo analyses in this section aim at exploring the relevance of the problem. To benchmark the bias of the fixed-effects estimator, we use the pooled-OLS estimator which is known to have poor properties in the presence of omitted time-invariant variables and dynamic misspecifications. Naïvely, one could expect that, since pooled-OLS suffers from (at least) two problems while the fixed-effects estimator solves the problem of omitted time-invariant variables, the bias of the fixed-effects estimator is always strictly smaller than the bias of pooled-OLS. However, this perspective ignores the fact that the fixed-effects estimator solely uses the within variation and is therefore more vulnerable to dynamic misspecification than pooled-OLS that uses both, the within- and the between variation. As we demonstrate analytically, it is thus possible that fixed-effects estimates are more biased than the pooled-OLS estimates under identifiable conditions. To study the properties of the fixed-effects estimator with potential dynamic misspecification and to reveal the conditions under which the use of fixed effects produces larger bias than the naïve pooled-OLS estimator, we employ a set of Monte Carlo experiments.

Our data-generating process follows a straightforward set-up:

(15) $$\begin{eqnarray}y_{it}=x_{it}^{1}+(x_{it}^{2})+u_{i}+\unicode[STIX]{x1D700}_{it}\end{eqnarray}$$

with $x_{it}^{1}$ , $x_{it}^{2}$ , $\unicode[STIX]{x1D700}_{it}$ , and $u_{i}$ being drawn from a standard normal probability density function.

We use three rather straightforward types of model misspecifications as examples: an omitted time-varying variable, an omitted time trend when the variable of interest $x_{it}^{1}$ is trended, and the simple dynamic misspecification analyzed formally in the previous section—a one period lagged effect of $x_{it}^{1}$ . We distinguish three levels of correlation between our variable of interest $x_{it}^{1}$ and an omitted strictly time-invariant, constant effect variable $u_{i}$ . We set this correlation between $x_{it}^{1}$ and $u_{i}$ to 0.0 (absent), 0.2 (weak), and 0.5 (substantive). Higher correlation between $x_{it}^{1}$ and $u_{i}$ implies higher bias of the pooled-OLS estimator, while the correlation between $x_{it}^{1}$ and $u_{i}$ does not bias the fixed-effects estimator. The higher the correlation between $x_{it}^{1}$ and $u_{i}$ , the larger the bias advantage of the fixed-effects model before we consider a dynamic misspecification. Obviously, in the absence of dynamic misspecification the fixed-effects estimates are unbiased. Throughout all specifications we assume that between and within effects are equal. We acknowledge that this is a strong assumption and that pooled-OLS gives an average estimate of the two effects while the fixed-effects estimator provides a clean estimate for the within effect only. For a discussion of dealing with different within and between effects see Bell and Jones (Reference Bell and Jones2015). We refrain from adding a discussion of different effects across units and over time since it would distract from the focus on bias stemming from dynamic misspecification.

We are of course aware that social scientists could be able to correctly model these simple dynamic misspecifications. But this argument misses the point: we do not seek to identify dynamic misspecifications which are so difficult to model that social scientists probably fail to fully eliminate them. Instead, we are analyzing the consequences of dynamic misspecifications. The advantage of simple dynamic misspecifications, thus, is that it is easy to understand how they bias the fixed-effects estimator. Only in a second step will we generate complex data-generating processes for which simple solutions are not available. None of the data-generating processes we study here are likely to be as complex as true data-generating processes. Given that we include simple dynamics, we do not just use a simple fixed-effects specification, but rather compare fixed-effects estimation with dynamic specifications that applied researchers are likely to use as econometric solutions for potential dynamic misspecifications:Footnote ¹⁸ a lagged dependent variable (or Arellano–Bond dynamic panel modelFootnote ¹⁹ ), the Prais–Winsten transformation, or period fixed effects. This also allows us to demonstrate that these simple fixes, which are widely used in panel and pooled analyses, do not sufficiently eliminate simple dynamic misspecifications. In addition to these simple but commonly employed dynamic fixes, we use more general dynamic specifications as offered by ADL models and show that capturing the most salient dynamic elements of a DGP can reduce the bias considerably. This is consistent with Pickup (Reference Pickup2017).

For our first two experiments, $x_{it}^{2}$ is the omitted part of the data-generating process. We first directly manipulate the correlation between the within variation of $x_{it}^{1}$ and $x_{it}^{2}$ and the unit heterogeneity, e.g. the covariance of the between variance of $x_{it}^{1}$ and the unobserved unit-specific effects $u_{i}$ with $\text{corr}\left(\ddot{x}_{it}^{1},\ddot{x}_{it}^{2}\right)=\left\{\,0.2,\,0.5,0.8\right\}$ .

The second set of experiments aims at demonstrating the logic of our argument without ex ante assuming that $x_{it}^{1}$ and $x_{it}^{2}$ are correlated. We generate a dynamic misspecification by merely trending both variables so that the correlation results from the trends only. We discuss two different variants of this second Monte Carlo experiment.Footnote ²⁰ The first variant assumes common trends across all units: Both included and excluded right-hand-side (RHS) variables are continuous with a common trend of 0.1 increase per time period:

(16) $$\begin{eqnarray}x_{it}^{1,2}=N\sim \left(0,1\right)+0.1\ast t,\quad t=1,\ldots ,T.\end{eqnarray}$$

The second variant relaxes this assumption and allows for unit-specific trends,Footnote ²¹ which merely means that trends are conditioned by other factors—a plausible assumption for social scientists, since trends are unlikely to be homogeneous across units. Specifically, we randomly draw a third of the units that receives a positive trend of 0.1 per time period (see Equation (16)), a third of the units remains untrended ( $x_{it}^{1,2}=N\sim \left(0,1\right)$ ), and the last third of units has a negative trend of 0.1 per time period $x_{it}^{1,2}=N\sim \left(0,1\right)-0.1\ast t,t=1,\ldots ,T$ ).Footnote ²²

The third experiment is based upon a slightly different DGP to account for a misspecified lag structure of $x_{it}^{1}$ :

(17) $$\begin{eqnarray}y_{it}=x_{it-1}^{1}+u_{i}+\unicode[STIX]{x1D700}_{it}.\end{eqnarray}$$

We compare the bias generated by a static OLS estimator ( $y_{it}=x_{it}^{1}+\unicode[STIX]{x1D700}_{it}$ ) to that of a static FE estimator ( $\ddot{y} _{it}=\ddot{x}_{it}^{1}+\ddot{\unicode[STIX]{x1D700}}_{it}$ ) where the lagged effect of $x_{it}^{1}$ is not taken into account.

Finally, we also allow the lag length of $x_{it}^{1}$ to vary across units in the following way: for one randomly drawn third of the units $x_{it}^{1}$ exerts a one period lagged effect on $y_{it}$ as in Equation (17), for the second randomly drawn third of units we observe a two period lagged effect $y_{it}=x_{it-2}^{1}+u_{i}+\unicode[STIX]{x1D700}_{it}$ and for the last third we model a three period lagged effect $y_{it}=x_{it-3}^{1}+u_{i}+\unicode[STIX]{x1D700}_{it}$ .

We vary the number of periods [ $T=\{10,30,50\}$ ] but we hold the number of units constant at 20 throughout all experiments. Note that increasing the number of units increases the between variation and favors pooled-OLS over fixed effects (Plümper and Troeger Reference Plümper and Troeger2007, Reference Plümper and Troeger2011). In each permutation of the experiments we estimate 500 models with independently drawn errors.

Since econometricians have developed different solutions for models with potential dynamic misspecifications, we incorporate these variants of the fixed effects and the pooled-OLS estimators into the simulation. The most commonly used ‘solutions’ to dynamic misspecification are the inclusion of the lagged dependent variable (LDV: $y_{it}=\unicode[STIX]{x1D6FC}+\unicode[STIX]{x1D6FD}_{1}y_{it-1}+\unicode[STIX]{x1D6FD}_{2}x_{it}^{1}+\unicode[STIX]{x1D700}_{it}$ ),Footnote ²³ and period fixed effects ( $y_{it}=\unicode[STIX]{x1D6FC}_{t}+\unicode[STIX]{x1D6FD}_{2}x_{it}^{1}+\unicode[STIX]{x1D700}_{it}$ ) or a combination of the two ( $y_{it}=\unicode[STIX]{x1D6FC}_{t}+\unicode[STIX]{x1D6FD}_{1}y_{it-1}+\unicode[STIX]{x1D6FD}_{2}x_{it}^{1}+\unicode[STIX]{x1D700}_{it}$ ). Less often researchers employ a Prais–Winsten transformation (PW: $\left(y_{it}-\unicode[STIX]{x1D70C}y_{it-1}\right)=\unicode[STIX]{x1D6FC}+\unicode[STIX]{x1D6FD}_{2}\left(x_{it}^{1}-\unicode[STIX]{x1D70C}x_{it-1}^{1}\right)+\left(\unicode[STIX]{x1D700}_{it}-\unicode[STIX]{x1D70C}\unicode[STIX]{x1D700}_{it-1}\right)$ ),Footnote ²⁴ or an ADL ((1,1): $y_{it}=\unicode[STIX]{x1D6FC}+\unicode[STIX]{x1D6FD}_{1}y_{it-1}+\unicode[STIX]{x1D6FD}_{2}x_{it}^{1}+\unicode[STIX]{x1D6FD}_{3}x_{it-1}^{1}+\unicode[STIX]{x1D700}_{it}$ ) model.

Though it seems to increasingly be the case that scholars estimate fixed-effects models without justification and thus as default, econometric textbooks suggest a variant of the Hausman specification test (Hausman Reference Hausman1978) to decide whether to estimate a fixed effects or a random-effects/ OLS specification. The Hausman test (and its variants) have been shown to be consistent (for a short overview see Baltagi Reference Baltagi2001, 65–70), therefore, if fixed-effects estimates are significantly different from random effects or pooled-OLS estimates, the latter are biased because of unit heterogeneity.Footnote ²⁵ However, the asymptotic properties of the Hausman test do not necessarily translate into favorable finite sample characteristics especially when other misspecifications do exist and are not accounted for. We also present Monte Carlo results for the performance of this test. This is related to our main research interest, because we intend to demonstrate that pooled-OLS may be less biased than the fixed-effects model in situations in which the Hausman test favors a fixed-effects specification. These instances may occur frequently and under plausible conditions.