Articles and Posts from ISQ

Joseph Weinberg’s piece highlights an important substantive and methodological question: how to analyze, theoretically and methodologically, differences in national policy autonomy among countries and across policy areas in the era of globalization or regional integration.  EU membership constrains the policy autonomy of member states, which can change the relationship between the explanatory variables of interest and the outcome variable.  As Weinberg argues, “While a particular set of independent variables may explain outcomes in sovereign countries, those same variables would have little explanatory power where decisions are made by a supranational body” (5). We agree wholeheartedly that, if membership in a supranational institution constrains certain policy outcomes, then researchers should account for that in their theoretical and empirical models.  We disagree, however, on the solution. In particular, we show how multi-level models have important advantages in modelling this phenomenon, compared to the split-sample regressions in his piece.

The issues Weinberg raises are examples of moderation.  In Baron and Kinney’s words, “moderator variables specify when certain effects will hold… [They] partition a focal independent variable into subgroups that establish its domains of maximal effectiveness in regard to a given dependent variable” (1986: 1174).  EU membership is a moderating variable.  It changes the relationship between an explanatory variable, e.g. type of electoral institution, and an outcome variable, e.g. trade policy.  Since EU membership constrains national trade policy, we might expect to see a different (potentially muted) relationship between electoral institutions and trade policy among EU members, compared to that relationship among non-members. Moderators can change the strength, direction, and existence of a stable relationship between the explanatory and dependent variable. Weinberg referred to the third type of moderated effect (from existence to nonexistence or vice versa) as “replacement.”

Chaudoin, Milner, and Pang (2015) demonstrate how multi-level modelling (MLM) can facilitate making inference regarding theories explaining moderation.  Here, we use Weinberg’s replications of arguments from Dutt and Mitra (2010) to demonstrate how MLM has three advantages for modelling the relationships made by Weinberg.

1)      MLM affords flexibility in matching the empirical model to the theoretical argument.

2)      MLM can be more efficient than split-sample or interaction term modelling.

3)      MLM facilitates model comparison for the types of theoretical issues raised by Weinberg.

Briefly, Dutt and Mitra argue that having a rural party in power (coded with a binary variable, rural) and the degree of executive constraints (from Polity, xconst) affect a country’s degree of agricultural protectionism (measured by an outcome variable, RRA).  They expect, and find, positive effects for both variables.  Weinberg argues that, since agriculture policy is set at the supranational level by the EU, these domestic variables should have little effect on policy for EU members.

To demonstrate (1) and (2), we first compare the models from Weinberg’s Table 2 (here, Models 1-3) with a MLM that allows the effect of the main variables of interest, rural and xcons, to vary by a country’s EU membership (here, Model 4).

This model is similar to Dutt and Mitra’s model in that it includes country- and year-specific intercepts (αi and δt) and similar to Weinberg’s model in that the effect of rural and xcons can be moderated by an EU membership indicator variable (EU).  However, unlike the split sample regressions in Weinberg, this model affords the researcher greater flexibility.  For example, this model does not assume that EU membership moderates all of the explanatory variables.  The researcher can pick and choose, based on theoretical knowledge, what factor moderates what.  In our Model 4,we have allowed EU membership to moderate two variables that Weinberg argued were likely, theoretically, to be moderated.  But we have not been forced to make stronger assumptions about EU membership moderating all the variables.

Table 1 displays the results.  As in Weinberg’s split sample replications, EU membership does appear to moderate the effect of xcons and rural on the outcome variable.  Rural has a positive and significant effect for non-EU members and a negative, insignificant effect for EU members.  Unlike the split sample regressions, however, greater executive constraints have a positive effect for non-EU members and a negative effect for EU members, with both being significant.  The MLM also uses the data efficiently.  The standard errors on the control variables, like ideology, are often half as large as those in the split sample regressions.

 

To demonstrate (1) and (3), we compare two more complicated models.  Model 5 estimates a country-specific coefficient for the rural and xcons variables.  Model 6 is similar to Model 4, but it allows the coefficient on every variable, not just rural and xcons, to vary by EU membership.  Model 6 is thus most similar to the split sample regression advocated for by Weinberg in his Models 2-3.

Figures 1 and 2 show the country-specific coefficients for the estimated effect of xconst on the outcome variable, from Model 5, with Figure 1 showing coefficients for Non-EU countries and Figure 2 showing EU countries.  Figures 3 and 4 repeat this for the rural variable.

For the most part, the effect of the two variables for EU countries is small and generally insignificant.  However, there is a large degree of heterogeneity in the effects of the two variables for non-EU countries.  The effect of the rural variable is positive and negative for some non-EU countries.  Although the variation of the effects of the two variables is smaller within the EU country group, it is clear that the importance of domestic variables on agricultural protectionism varies among EU countries, especially when we look at the effect of the rural variable. This heterogeneity is potentially of great interest to researchers, and it is uncovered only with the MLM.

We can also compare these models using statistics such as their AIC.  Model 6, which most closely resembles the split sample regressions in Weinberg’s replication, has the worst AIC.  The best model, based on AIC comparisons, is Model 5.  This implies that the heterogeneity of policy autonomy among EU countries is salient---EU-level decision-making does not “replace” national decision making, though the former changes (moderates) national policy autonomy in different EU countries to varying degree.

To conclude, we agree with the majority of Weinberg’s arguments.  Our main point of departure is to argue that multilevel models are a powerful tool for modeling the empirical relationships implied by Weinberg’s theoretical arguments.  These models afford researchers a great deal of flexibility and efficiency for modeling national-level policies in a globalized world.

 

Works Referenced

Baron, Reuben M., and David A. Kenny. 1986. The Moderator-Mediator Variable Distinction in Social Psychological Research: Conceptual, Strategic, and Statistical Considerations. Journal of Personality and Social Psychology 51 (6): 1173–82.

Chaudoin, Stephen, Helen V. Milner, and Xun Pang. 2015. “International Systems and Domestic Politics: Linking Complex Interactions with Empirical Models in International Relations.” International Organization. 69(2): 275-309.

Dutt, Pushan and Devashish Mitra. 2010. “Impacts of Ideology, Inequality, Lobbying, and Public Finance” in Anderson, Kym (ed.) (2010), The Political Economy of Agricultural Price Distortions, Cambridge and New York: Cambridge University Press.

Weinberg, Joseph. Forthcoming. “European Union Member States in Cross National Analyses: The Dangers of Neglecting Supranational Policymaking.” International Studies Quarterly.

 


 

Table 1: Baseline, Split Sample, and Multilevel Model Replications of Dutt and Mitra

Model 1

Model 2

Model 3

Model 4

Full Sample

Non-EU

EU Only

Full Sample

Ideo.

0.24

0.26

-0.82

0.11

0.24

0.25

0.65

0.14

Ideo.*GDP

-0.03

-0.03

0.08

-0.01

0.03

0.03

**

0.06

0.01

L.GDP

0.38

**

0.40

0.02

0.35

0.14

0.14

0.22

0.05

Exec. Constraints

0.00

0.00

0.09

0.00

0.00

0.11

Pres. System

0.10

0.06

.

0.05

0.07

0.07

.

0.03

Rural Party

0.45

***

0.44

***

0.05

0.11

0.12

0.09

Exec. Constraints

-0.13

**

(EU)

0.05

Exec. Constraints

0.04

***

(Non-EU)

0.01

 

Rural Party

-0.07

(EU)

0.19

Rural Party

0.50

***

(Non-EU)

0.03

 

Intercept

-1.96

***

(EU)

0.38

Intercept

-3.13

***

(Non-EU)

0.06

 


 

Figure 1: Country-specific Coefficients on xconst, Non-EU Countries, from Model 5

Figure 2: Country-specific Coefficients on xconst, EU Countries, from Model 5

 

Figure 3: Country-specific Coefficients on rural, Non-EU Countries, from Model 5

Figure 4: Country-specific Coefficients on rural EU Countries, from Model 5

 

 

ISQ On Twitter

The International Studies Association

Representing 100 countries, ISA has over 6,500 members worldwide and is the most respected and widely known scholarly association in this field. Endeavoring to create communities of scholars dedicated to international studies, ISA is divided into 7 geographic subdivisions of ISA (Regions), 29 thematic groups (Sections) and 4 Caucuses which provide opportunities to exchange ideas and research with local colleagues and within specific subject areas.
Help   |   Thanks   |   Privacy Statement   |   Terms Of Use