Direct and Indirect Effects for Neighborhood-Based Clustered and Longitudinal Data

ResearchBlogging.org

Tyler J. VanderWeele, Sociological Methods & Research 2010 38: 515-544.

 

Definitions of direct and indirect effects are given for settings in which individuals are clustered in groups or neighborhoods and in which treatments are administered at the group level. A particular intervention may affect individual outcomes both through its effect on the individual and by changing the group or neighborhood itself. Identification conditions are given for controlled direct effects and for natural direct and indirect effects. The interpretation of these identification conditions are discussed within the context of neighborhood research and multilevel modeling. Interventions at a single point in time and time-varying interventions are both considered. The definition of direct and indirect effects requires certain stability or no-interference conditions; some discussion is given as to how these no-interference conditions can be relaxed.

Key Words: causal inference • direct and indirect effects • interference • longitudinal data • multilevel models • neighborhood effects • mediation • potential outcomes

Tyler J. VanderWeele

Harvard University, Boston, MA, USA, tvanderw@hsph.harvard.edu

Bookmark and Share

VanderWeele, T. (2010). Direct and Indirect Effects for Neighborhood-Based Clustered and Longitudinal Data Sociological Methods & Research, 38 (4), 515-544 DOI: 10.1177/0049124110366236

Advertisements

5 Responses to Direct and Indirect Effects for Neighborhood-Based Clustered and Longitudinal Data

  1. Pat Sharkey says:

    (How) can we figure out whether and why place-based investments work?

    Tyler VanderWeele’s article “Direct and Indirect Effects for Neighborhood-Based Clustered and Longitudinal Data” is excellent, and it is also timely. For the last several decades the most prominent federal initiatives designed to confront urban poverty have been targeted toward individuals or families—some examples being the earned income tax credit and the Moving to Opportunity experiment.* The Obama Administration has not altered course entirely, but it has proposed new initiatives that would target resources toward places, as exemplified in the Promise Neighborhoods and Choice Neighborhoods proposals. Promise Neighborhoods would build on the Harlem Children’s Zone model and provide concentrated resources for children’s academic development to a small number of neighborhoods across the country. Choice Neighborhoods would provide concentrated resources for community development to a similarly small number of neighborhoods across the country in which public housing projects have been demolished through the HOPE VI program.

    Neither initiative will include any randomization as part of the site selection process, leading to two key questions: 1) Will it be possible to know whether these programs work (e.g., will resources for Promise Neighborhoods increase the likelihood that a child will graduate from college)? 2) Will it be possible to know why these programs work (e.g., what portion of the potential effects on college graduation are due to attendance at a new, high quality charter school)?

    VanderWeele’s article lays out the methods and the assumptions that we would need to make in order to answer these questions. One of the strengths of the paper is the use of two examples of neighborhood-level interventions that make the intuition behind the statistics crystal clear; but to add some policy context I will depart from VanderWeele’s examples and instead use the example of the Promise Neighborhoods proposal and its impact on a child’s probability of graduating from college, with the mediator being attendance at a new charter school established with Promise Neighborhoods resources.

    Assumptions regarding SUTVA and “intact clusters” are problematic, and are discussed extensively in the article; but as always the most difficult assumptions pertain to confounding. One of the central insights in the VanderWeele paper that is often overlooked is the nature of confounding in a community-level intervention. As noted in the paper, if the location of an intervention is based on average community-level characteristics, such as the poverty rate in the community, then it is sufficient to control for community-level covariates in estimating causal effects of the intervention—it is not necessary to control for individual-level covariates in this scenario. This does not mean that confounding is any less of a problem, however. In the case of Promise Neighborhoods, one would have to know and control for the full set of covariates that were relevant in the site selection process in order to identify the impact of living in a Promise Neighborhood. Some of these covariates are obvious and measurable, like the poverty rate in the community, which is likely to be a factor in determining need for resources through the program. Others are less obvious and may be problematic to measure, like the competence of community leaders who put together the proposal to the federal government. Estimating the total impact of Promise Neighborhoods depends on the assumption that we can measure this type of covariate, which is questionable.

    And then there’s the second question of mediation. This is an increasingly important issue, as there is already an emerging debate about whether place-based interventions can work only if they change the quality of the schools children attend. Roland Fryer argues that attendance at a Promise Academy, a set of schools developed by Geoffrey Canada as part of the Harlem Children’s Zone, is the crucial mechanism for improving children’s test scores. Forthcoming research based on several observational and experimental studies of mobility programs argues the opposite—changes in neighborhood quality are enough to improve cognitive test scores even in the absence of changes in school quality.**

    In my hypothetical example of Promise Neighborhoods, if the mediator is attendance at a high quality charter school within the neighborhood, then we must measure and control for all of the covariates that predict whether an individual child is able to take advantage of the presence of this new school and attend. The potential for unobserved confounding is clearly enormous. One can imagine, however, the possibility that a lottery system similar to the one used in the Promise Academies might be implemented to select students for new schools within a designated Promise Neighborhood. This would satisfy the ignorability assumption for the mediator.

    To conclude, let me reiterate my belief that VanderWeele’s article will be extremely useful in extending other important research in this area, particularly the work of Raudenbush and Hong, and providing a framework for attempting to estimate the direct and indirect effects of setting-level interventions. But as increasing attention is given to the pathways by which such interventions may impact individuals, the article may be most important in emphasizing how difficult it is to decompose direct and indirect effects. This is bad news considering the impending rollout of two prominent federal initiatives designed to test the impact of community-level interventions. If we want to know whether these programs work, we have serious challenges with which to contend. If we want to know why these programs work, these challenges are even more formidable.

    * There are counter-examples to this generalization, such as the demolition of public housing projects via HOPE VI and the Clinton Administration’s Empowerment Zones/Enterprise Communities program. However, one can think of HOPE VI as a residential mobility program as much as a community-level intervention, and the EZ/EC initiative was rather weakly implemented and is generally thought to have minimal impacts on targeted areas.

    ** This evidence comes from a paper titled “Converging Evidence for Neighborhood Effects on Children’s Test Scores: An Experimental, Quasi-experimental, and Observational Comparison,” written by Julia Burdick-Will, Jens Ludwig, Stephen W. Raudenbush, Robert J. Sampson, Lisa Sanbonmatsu, and Patrick Sharkey. The paper was prepared for the Brookings Institution “Project on Social Inequality and Educational Disadvantage: New Evidence on How Families, Neighborhoods and Labor Markets Affect Educational Opportunities for American Children.”
    Link to the paper: http://cassr.as.nyu.edu/page/workingpapers

  2. …[Trackback]…

    Hello, you used to write excellent, but the last several posts have been kinda boring� I miss your great writings. Past several posts are just a bit out of track! come on!…

  3. I had been hunting for ideas for my personal website and uncovered your own blog post,
    “Direct and Indirect Effects for Neighborhood-Based Clustered and Longitudinal Data SOCIOLOGICAL METHODS
    & RESEARCH”, would you mind in case I personally utilize a number of your suggestions?
    Appreciate it -Susie

  4. Swen says:

    The best example of ginger’s anti-inflammatory potential is in the treatment of arthritis. Initially, personal chemistry between the three helped propel the music. Its effectiveness against headaches has been documented.

  5. Do not buy Men’s or Women’s luxury watches too cheap because the possibility
    of fraud is always present. – things that are still in very good
    condition at a lower price. Remove the watch from your wrist
    when winding so as not to place undue pressure on the stem.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: