The following comment refers to:
Liu (2007) analyzes an interesting individual election contest in which the racial identities of voters and registrants are known. He uses this information to calculate true turnout rates by race (table 3 and page 12). He then compares these true turnout rates to turnout rates estimated by various methods of ecological regression.
In this comparison, Liu (page 12) concludes that the double regression estimates of black and white voter turnout are close to the known turnout rates. Consequently, he concludes that “(t)hese findings …seem inconsistent with the claim that there is no use in the double-regression approach … (Zax 2002, 2005).”
In fact, Liu’s result is completely consistent with the analysis in Zax (2005). Zax (2005, 65) proves that the double regression estimators of turnout by race are, perhaps surprisingly, statistically unbiased. In other words, they tend to be correct. The correspondence between true turnout rates and those estimated by double regression in the contest analyzed by Liu is simply an example of the proof in Zax (2005).
However, estimates of racial turnout rates are usually of secondary interest in applications of ecological regression techniques. The principle interest is generally attached to estimates of racial voting preferences. The conclusions in Zax (2005) regarding the inadequacy of double regression are not based on the demonstration that double regression turnout estimates are valid. They are based, instead, on the proof (2005, 65-7) that its estimates of voter preferences are invalid. These estimates are biased, inconsistent and have variances which are unknown and, for practical purposes, unknowable.
Liu (2007) doesn’t dispute this proof. Moreover, Liu’s (2007) data don’t report voting choices by race. For this reason, he can’t calculate true racial voting preferences and can’t compare them to double regression estimates of these voting preferences. Therefore, he doesn’t know if the double regression estimates of racial voting preferences in his example are “close” to racial true preferences.
In other words, Liu (2007) doesn’t refute Zax’s (2005) critique of double regression theoretically and can’t test it empirically. Consequently, Liu’s work doesn’t contradict any of the conclusions in Zax (2002, 2005). In particular, it offers no rebuttal to the proof that
double regression estimates of racial voting preferences are invalid. Unfortunately, Liu (2007) contains several other examples of unsound methodology. He doesn’t present the theoretical properties of any of his estimators, and has only a single empirical example of each. Consequently, he can’t calculate either their biases or their mean squared errors. What he refers to as “bias” (table 3) is, most likely, “prediction error” in the case of his example. The interpretation of Liu’s “mean squared error” (table 5) is unknown. Neither can be reliable indicators of the relative performance of the techniques from which they are derived.
Fundamentally, though, the comparisons that Liu (2007) attempts to draw are inappropriate from their inception. Liu (2007, pgs. 15-19) demonstrates convincingly that aggregation bias is present in his single election contest. Of the techniques employed by Liu (2007), only that of King (1997), as demonstrated by Liu (2007, 15-19), offers corrections for this problem. There is no claim or evidence anywhere in the literature that any of the other techniques are accurate when aggregation bias is present. Therefore, any “similarity” between the estimates of double regression or of these other techniques and known behavior in the context of this particular contest can only be understood as coincidental.
Ashenfelter, Orley, Phillip B. Devine and David J. Zimmerman (2003) Statistics and Econometrics: Methods and Applications, John Wiley & Sons, Inc., New York, NY.
Dougherty, Christopher (2002) Introduction to Econometrics, Second Edition, Oxford University Press, Oxford, U.K.
Gujarati, Damodar N. (2003) Basic Econometrics, Fourth Edition, McGraw-Hill Irwin, Boston, MA.
Hill, R. Carter, William E. Griffiths and George G. Judge (2001) Undergraduate Econometrics, Second Edition, John Wiley & Sons, Inc., New York, NY.
King, Gary (1997) A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior From Aggregate Data, Princeton University Press, Princeton.
Liu, Baodong (2007) “EI extended model and the fear of ecological fallacy”, Sociological Methods & Research, Vol. 36, No. 1, August, 3-25.
Murray, Michael P. (2006) Econometrics: A Modern Introduction, Pearson Addison Wesley, Boston, MA.
Ramanathan, Ramu (2002) Introductory Econometrics with Applications, Harcourt College Publishers, Fort Worth.
Stock, James H. and Mark W. Watson (2007) Introduction to Econometrics, Second Edition, Pearson Addison-Wesley, Boston, MA.
Studenmund, A. H. (2006) Using Econometrics: A Practical Guide, Fifth Edition, Pearson Addison-Wesley, Boston, MA.
Wooldridge, Jeffrey M. (2003) Introductory Econometrics: A Modern Approach, Thomson South-Western, Mason, OH.
Zax, Jeffrey S. (2002) “Comment on ‘Estimating the extent of racially polarized voting in multicandidate contests’ by Bernard Grofman and Michael Migalski”, Sociological Methods & Research, Vol. 31, No. 1, August, 73-84.
Zax, Jeffrey S. (2005) “The statistical properties and empirical performance of double regression”, Political Analysis, Vol. 13, No. 1, January, 57-76.
Jeffrey S. Zax
Department of Economics University of Colorado at Boulder
Campus Box 256, Boulder, CO 80309-0256
Originally written and peer reviewed in 2008.