## Search

## Blogroll

## Categories

- book review (15)
- full-text article (19)
- news (7)
- peer-reviewed comment (1)
- post (142)
- TOC (4)

## Tags

analytic narratives APC models association attrition bias categorical data causal inference Chris Winship cohorts comparative analysis compensation complex sampling data analysis data archiving data sharing double regression ecological regression Equal Employment Opportunity Commission finite mixture models front-door criterion Fuzzy-set fuzzy set analysis Fuzzy Sets gender inequality genetics goodness-of-fit historical sociology homophily identification strategies increasing returns inequality latent variable models life course log-linear model longitudinal data matching measurement mechanisms meta-analysis methodology methodology (data analysis) missing data mixture model model selection Monte Carlo simulations multilevel multilevel models news odds ratios optimal matching peer review professional workers qualitative comparative analysis quantitative methods randomized response rational choice theory regression replication segregation sensitive topics sequence analysis SMR social influence social networks statistics stratification structural equation modeling structural equation models survey participation survey research transparency variance components vignettes visual sociology web survey Web surveys## Recent Comments

## Top Posts

## Top Rated

## Archive

- May 2011 (10)
- January 2011 (10)
- November 2010 (8)
- August 2010 (5)
- June 2010 (4)
- May 2010 (6)
- April 2010 (5)
- February 2010 (119)

## Twitter

- New from Orgtheory: "Wal-Mart and Beyond: Can Social Science be Itself in Court? (Response to Professor Winship)": wp.me/pOKB-42k 6 years ago
- “Inferring from the General to the Specific: What Happened? Why?”, a forthcoming special issue of SMR http://wp.me/pNiTC-hv 6 years ago
- SMR 40.2 TOC http://wp.me/pNiTC-gZ 6 years ago
- Homophily and Contagion Are Generically Confounded in Observational Social Network Studies http://wp.me/pNiTC-h1 6 years ago
- Sensitivity Analysis for Contagion Effects in Social Networks http://wp.me/pNiTC-h3 6 years ago
- An Extended Model Comparison Framework for Covariance and Mean Structure Models, Accommodating Multiple Groups and La… http://wp.me/pNiTC-h7 6 years ago
- Measurement Equivalence of Ordinal Items: A Comparison of Factor Analytic, Item Response Theory, and Latent Class App… http://wp.me/pNiTC-ha 6 years ago
- Multiple Auxiliary Variables in Nonresponse Adjustment http://wp.me/pNiTC-hc 6 years ago
- When Change Matters: An Analysis of Survey Interaction in Dependent Interviewing on the British Household Panel Study http://wp.me/pNiTC-he 6 years ago
- Group-based Trajectory Modeling Extended to Account for Nonrandom Participant Attrition http://wp.me/pNiTC-hg 6 years ago
- Assessing the Robustness of Crisp-set and Fuzzy-set QCA Results http://wp.me/pNiTC-hj 6 years ago
- Walmart and the ASA (a guest post by Chris Winship): wp.me/pOKB-41K 6 years ago
- Peer Review: Trial By Twitter http://tinyurl.com/4pkucw5 What do social science researchers think about this? 6 years ago
- New for 2011: http://wp.me/pNiTC-fG 6 years ago
- Sensitive Questions in Online Surveys: Experimental Results for the Randomized Response Technique (RRT) and the Unmat… http://wp.me/pNiTC-gC 6 years ago
- Multiple Sources of Nonobservation Error in Telephone Surveys: Coverage and Nonresponse: http://wp.me/pNiTC-gx 6 years ago
- Estimating Propensity Adjustments for Volunteer Web Surveys: http://wp.me/pNiTC-gt 6 years ago
- The Effects of Asking Filter Questions in Interleafed Versus Grouped Format : http://wp.me/pNiTC-gr 6 years ago
- Two Algorithms for Relaxed Structural Balance Partitioning: Linking Theory, Models, and Data to Understand Social Net… http://wp.me/pNiTC-go 6 years ago
- Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys: http://wp.me/pNiTC-gk 6 years ago

## SMR Blog

## Share this blog

%d bloggers like this:

## Comment on “EI extended model and the fear of ecological fallacy”, by Baodong Liu

The following comment refers to: Sociological Methods & Research, Vol. 36, No. 1, 3-25 (2007), EI Extended Model and the Fear of Ecological Fallacy, by Baodong Liu

Liu (2007) analyzes an interesting individual election contest in which the racial identities of voters and registrants are known. He uses this information to calculate true turnout rates by race (table 3 and page 12). He then compares these true turnout rates to turnout rates estimated by various methods of ecological regression.

In this comparison, Liu (page 12) concludes that the double regression estimates of black and white voter turnout are close to the known turnout rates. Consequently, he concludes that “(t)hese findings …seem inconsistent with the claim that there is no use in the double-regression approach … (Zax 2002, 2005).”

In fact, Liu’s result is completely consistent with the analysis in Zax (2005). Zax (2005, 65) proves that the double regression estimators of turnout by race are, perhaps surprisingly, statistically unbiased. In other words, they tend to be correct. The correspondence between true turnout rates and those estimated by double regression in the contest analyzed by Liu is simply an example of the proof in Zax (2005).

However, estimates of racial turnout rates are usually of secondary interest in applications of ecological regression techniques. The principle interest is generally attached to estimates of racial voting preferences. The conclusions in Zax (2005) regarding the inadequacy of double regression are not based on the demonstration that double regression turnout estimates are valid. They are based, instead, on the proof (2005, 65-7) that its estimates of voter preferences are invalid. These estimates are biased, inconsistent and have variances which are unknown and, for practical purposes, unknowable.

Liu (2007) doesn’t dispute this proof. Moreover, Liu’s (2007) data don’t report voting choices by race. For this reason, he can’t calculate true racial voting preferences and can’t compare them to double regression estimates of these voting preferences. Therefore, he doesn’t know if the double regression estimates of racial voting preferences in his example are “close” to racial true preferences.

In other words, Liu (2007) doesn’t refute Zax’s (2005) critique of double regression theoretically and can’t test it empirically. Consequently, Liu’s work doesn’t contradict any of the conclusions in Zax (2002, 2005). In particular, it offers no rebuttal to the proof that

double regression estimates of racial voting preferences are invalid. Unfortunately, Liu (2007) contains several other examples of unsound methodology. He doesn’t present the theoretical properties of any of his estimators, and has only a single empirical example of each. Consequently, he can’t calculate either their biases or their mean squared errors. What he refers to as “bias” (table 3) is, most likely, “prediction error” in the case of his example. The interpretation of Liu’s “mean squared error” (table 5) is unknown. Neither can be reliable indicators of the relative performance of the techniques from which they are derived.

Fundamentally, though, the comparisons that Liu (2007) attempts to draw are inappropriate from their inception. Liu (2007, pgs. 15-19) demonstrates convincingly that aggregation bias is present in his single election contest. Of the techniques employed by Liu (2007), only that of King (1997), as demonstrated by Liu (2007, 15-19), offers corrections for this problem. There is no claim or evidence anywhere in the literature that any of the other techniques are accurate when aggregation bias is present. Therefore, any “similarity” between the estimates of double regression or of these other techniques and known behavior in the context of this particular contest can only be understood as coincidental.

ReferencesAshenfelter, Orley, Phillip B. Devine and David J. Zimmerman (2003) Statistics and Econometrics: Methods and Applications, John Wiley & Sons, Inc., New York, NY.

Dougherty, Christopher (2002) Introduction to Econometrics, Second Edition, Oxford University Press, Oxford, U.K.

Gujarati, Damodar N. (2003) Basic Econometrics, Fourth Edition, McGraw-Hill Irwin, Boston, MA.

Hill, R. Carter, William E. Griffiths and George G. Judge (2001) Undergraduate Econometrics, Second Edition, John Wiley & Sons, Inc., New York, NY.

King, Gary (1997) A Solution to the Ecological Inference Problem: Reconstructing Individual Behavior From Aggregate Data, Princeton University Press, Princeton.

Liu, Baodong (2007) “EI extended model and the fear of ecological fallacy”, Sociological Methods & Research, Vol. 36, No. 1, August, 3-25.

Murray, Michael P. (2006) Econometrics: A Modern Introduction, Pearson Addison Wesley, Boston, MA.

Ramanathan, Ramu (2002) Introductory Econometrics with Applications, Harcourt College Publishers, Fort Worth.

Stock, James H. and Mark W. Watson (2007) Introduction to Econometrics, Second Edition, Pearson Addison-Wesley, Boston, MA.

Studenmund, A. H. (2006) Using Econometrics: A Practical Guide, Fifth Edition, Pearson Addison-Wesley, Boston, MA.

Wooldridge, Jeffrey M. (2003) Introductory Econometrics: A Modern Approach, Thomson South-Western, Mason, OH.

Zax, Jeffrey S. (2002) “Comment on ‘Estimating the extent of racially polarized voting in multicandidate contests’ by Bernard Grofman and Michael Migalski”, Sociological Methods & Research, Vol. 31, No. 1, August, 73-84.

Zax, Jeffrey S. (2005) “The statistical properties and empirical performance of double regression”, Political Analysis, Vol. 13, No. 1, January, 57-76.

Jeffrey S. Zax

Department of Economics University of Colorado at Boulder

Campus Box 256, Boulder, CO 80309-0256

e-mail: zax@colorado.edu

Originally written and peer reviewed in 2008.

## Rate this:

## Share this:

## Like this:

RelatedThis entry was posted on Wednesday, April 21st, 2010 at 2:45 pm and is filed under full-text article, peer-reviewed comment. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.