|cc image: ZoL87 on Flickr|
So, imagine my surprise when I learned about a randomized social experiment with obesity as the outcome. Randomization balances the exposure [neighbourhood characteristic(s)] on known and unknown confounders, and rectifies the issue of temporality. Randomized controlled trials are the gold-standard in clinical epidemiology, but for ethical and economic reasons, are usually not feasible in social epidemiology (randomizing people to smoke, for instance, would never fly).
The study, published by Jens Ludwig and crew in the New England Journal of Medicine, was based on the Moving to Opportunity for Fair Housing Program, conducted by the US Department of Housing and Urban Development (HUD). The basic premise of this experiment was to determine how best to provide housing for those in need. Briefly, 4498 families with children living in public/project housing in high poverty neighbourhoods in Baltimore, Boston, Chicago, Los Angeles, or New York, were randomly allocated to one of three groups in the years 1994-1998 (one quarter of those eligible):
1. The MTO low poverty voucher group which received rental vouchers usable only in low-poverty areas (where < 10% of residents were poor), along with counseling and assistance in the search to find a private rental unit (n = 1788)
2. The traditional voucher group, which received rental vouchers where there were no restrictions on where the family could relocate, as well as support ordinarily given to families by local public housing authorities (n =1312)
3. The control group, which received no vouchers but remained eligible for public/project housing and other social programs, otherwise the status quo (n=1398)
For the most part, families were headed by African-American or Hispanic single mothers. From 2008-2010, health outcomes of female adults (usually the family head) were measured and included height, weight, and level of glycated hemoglobin.
Now, not all families moved or used the vouchers. The study used an intent-to-treat analysis which analyzes individuals based on groups to which they were assigned. This is the least biased and most conservative way to analyze a study like this. So even though a family may have been assigned to the MTO group but did not move to a low poverty neighbourhood, they would still be analyzed as part of the MTO group.
In the MTO group, 48% used the vouchers, in contrast to 63% in the traditional group. All groups were comparable at baseline in terms of 57 characteristics including age, race/ethnicity, marital status, employment, education, and federal assistance, for example. One year after randomization, the neighbourhood poverty rate was significantly lower in the MTO group, but this difference attenuated (still remained significant) at 5 and 10 y, as families in the control group moved to lower poverty areas on their own. Additionally, the proportion of women that said they felt safe/very safe in their neighbourhood, and the proportion that said neighbourhood adults would intervene in youth anti-social activity (defined as collective efficacy) were significantly higher in the MTO compared to the control group at 4-7 y and 10-15 y post-randomization. These same significant differences were seen for the traditional versus the control group, although there was no difference in collective efficacy at 10-15 y.
At 10-15 y of follow-up, after adjustment for baseline characteristics and allocation procedures, the prevalence in each category of extreme obesity was significantly lower in the MTO group (BMI ≥ 35 = 31.1%, and BMI ≥ 40 = 14.4%) compared to the control group (BMI ≥ 35 = 35.5%, and BMI ≥ 40 = 17.7%). There was no difference in obesity defined as BMI ≥ 30. The prevalence of elevated glycated hemoglobin was also lower in the MTO versus the control group (16.3% versus 20%). Differences were in the same direction but not significant between the traditional versus the control group.
This study was interesting to me mainly because of its design. Yes, significant differences were found, and interestingly, even with such low compliance. But there are some important things to keep in mind when interpreting the results of this study:
=> Significant differences were for severe obesity, not for overweight or obesity in general.
=> No baseline data was available for BMI or glycated hemoglobin so changes could not be assessed (the authors state that this should not affect internal validity, which I tend to agree with, especially if they found no significant differences in 57 baseline characteristics).
=> Allocation of participants and data collection procedures were extremely complicated; in many cases information was not collected from participants (even though they were eligible and appeared not to have refused), or they were randomly excluded. Perhaps because of word limits, reasons for treatment of participants during these processes were not clear to me.
=> Only one quarter of those eligible actually applied to be randomized.
=> I am wondering if exposure to environments after the initial move (e.g. subsequent moves) may have confounded associations. But I can’t really work out why this would be different across groups, given randomization, unless attrition was higher in one group versus another. Attrition is an issue in longitudinal study designs in general, but doesn’t appear to be an issue in this study (although, in light of what I said in the previous paragraph, I have trouble following calculation of response rates). I think the issue of multiple moves, and duration of time spent in each neighbourhood, warranted more of a discussion in the actual paper (some descriptive measures of neighbourhood characteristics were weighted by time spent in each neighbourhood, but I don’t think the main analysis accounted for this).
=> Is it the change in environment characteristics (and which ones are important), or just the move itself that is responsible for significant differences? Even though there were no significant differences between the MTO (had to move to a low poverty area) and the traditional group (who had no restrictions of where to move) in terms of the health outcomes, the authors say that differences approached significance for glycated hemoglobin, which they say, suggests that a change in the environment is important. I’m not sure if the results they are referring to can really support this assertion. It’s also evident that the traditional group moved to more affluent areas anyway so a comparison of the two groups in this regard may be moot.
=> This is a high-poverty, minority sample that examined adult women only. Although it may have higher relative internal validity for a social study, it lacks external validity or “generalizability” to other population subgroups.
- To expand a little on the high-poverty issue, I hypothesize that lower income individuals are more tied to their residential neighbourhoods (less mobile) than more affluent people (due largely to lack of access to a car). Thus, they accrue more exposure time than more affluent people. In this vein, I think residential environments are less important for more affluent individuals compared to those who are worse off. I also think that context in the US is likely not generalizable to the Canadian context (e.g. ghettoization based on racial segregation and poverty).
=> There is evidence that MTO families moved to areas lower in poverty but similar in racial distribution. These new areas still had more poverty than the country average.
=> Neighbourhoods themselves are not static entities, but were treated as such in this study. Some research has indicated that when change is considered, disadvantage is the same in the MTO versus the control group
=> Even though neighbourhood cohesion and safety were not outcomes, they are potential reasons for why significant differences were seen. However, the measures employed in the study were based on single items, which I find hard to accept that they accurately captured what they were supposed to measure.
=> Finally, this study was based on individuals as the unit of allocation and analysis, not neighborhoods. Thus, this was not a study of a neighbourhood-level intervention. Population interventions such as those for neighbourhoods are generally more cost-effective than those targeted to individuals. A discussion of the two in regards to the MTO study is provided by Sampson (2008).
All in all, the MTO is, and I’ll quote Sampson, “a major contribution to the long tradition of experimental social science.” There are certainly methodological issues with it, but I think that the NEJM study provides fairly strong evidence that small decreases in neighbourhood poverty can decrease prevalence of diabetes and extreme obesity in a highly disadvantaged population.