Unhealthful News 118 - How to take an unconfounded result and introduce confounding

Thanks to a tweet from @cjsnowdon I found this study in which the researcher showed a month-of-birth effect for anorexia risk (high if born in March-June, low if September-October).  The results were pretty strong and cannot be explained by confounding, and measurement bias seems extremely unlikely.  So as odd as it might seem, the effect seems to be real.

Highly technical point:  The test statistics they report are probably wrong (in the sense of being dishonest, though probably out of ignorance of honest methodology rather than lying) because they pretty clearly fished around to pick the most extreme time period.  Notice that Mar-Jun is four months but Sep-Oct is two, and you can be sure they did not specify their protocol to be "we will pick which of the four month periods, running Mar-Jun, Jul-Oct, or Nov-Feb, has the highest rate and which two month period, starting with Jan-Feb, has the lowest.  But I am sure they calculated their error statistics based on the assumption that they did exactly that.  So their results are biased upward a bit and their confidence intervals are too narrow.  I will make it a project to try to write up a version of this that fills in some details but is still widely understandable – I will post it when I manage.

Because the most sensible claim is basically "something about being born in a particular month" has an effect, there is literally no way for there to be confounding or reverse causation.  It cannot be that some unknown factor is causing both the ostensible cause and the observed effect (the most common source of confounding) because the "something about" conceptualization means that whatever that unknown factor is, it is actually captured in the broad phrasing of the causal factor.  Also, it is obviously impossible for someone's characteristics later in life to go back and affect their birth month.  So, in the spirit of yesterday's UN and my efforts to explain that there is no rigid hierarchy of study types, this correlation must be causation.  When people say that randomized trials (RCTs) are always better than any other study design, they are saying (though they often do even not understand enough to know this is what they are saying) that any systematic confounding is replaced by random confounding.  But the birth-month study is actually better than a RCT even for this one thing that favors RCTs:  There is no confounding, random or systematic.  Thus, the relationship must be causal.

Ah, but the tricky part is figuring out the causal pathway.  Unfortunately, the researchers did not seem to understand that this is a challenge and simply declared,

These results indicate that environmental risk factor(s) are operative during gestation or immediately after birth and their identification will be important for disease prevention strategies.

Really?  They apparently did not understand the "something about being born in a particular month" concept and managed to re-introduce confounding where none existed before by restricting the causal claim to "the physical act of coming to term in particular months and being a newborn in particular months" causes anorexia.  Unlike the claim that just trusts in what the data actually shows and must be right, this one makes huge assumptions and therefore is reasonably likely to be wrong.  If, for example, being conceived in a particular month is the actual cause of anorexia (maybe it is related to birth order, for example, or parents' SES), then under the authors' claim the result is actually due to confounding:  both the ostensible cause (being a newborn in a particular month) and the effect would be caused by an underlying common cause, and not otherwise related to each other.

The funniest part of all of this – at least to me – that in the BBC article about this, one of the authors declared,

However, our study only provides evidence of an association. Now we need more research to identify which factors are putting people at particular risk.

Allow me to just say, nonononononono NO NO NO!  Come on people, this is really not that difficult.  While it is never possible to prove than an association is causal, this study provides overwhelming evidence of causation (so long as you do not gratuitously introduce claims about why and stick with the "something about" that the data actually supports), not "only association".  There is almost no conceivable epidemiologic study that could provide better evidence of causation.  Now you need to figure out why there is causation.  The situation does not warrant some weaselly "more research blah blah which factors blah blah" that might be at home in some other study like the one I looked at yesterday.  I am somehow reminded of barely-literate star athletes awkwardly reciting the interview script after a game: "it was a team effort", "I could not have done it without….", and "the fans have been great".  You would like to think that someone working at a research university could do better than than reciting a pretend self-deprecating script while parading in front of the admiring national news rreporters. 

(And why did this highly technical study produce news coverage like scoring the winning goal? It is not as if the result is big enough to change anyone's behavior.  I guess that is a topic for another day.)

Of course, we do want to know why birth month is causing anorexia.  Thus we would not fault the researchers for positing a theory.  But that is not what they did.  They not only declared what particular aspect of birth month was causing the outcome, but they absurdly claimed "these results indicate".  But the results they report absolutely do not indicate that.  The results indicate what I wrote ("something about…"), but not the specific causal mechanism.  Other information would be needed to support their claim.  The BBC interview provided better support than their results.  The author noted that other very different psychological diseases have similar birth-month patterns, which tends to support the biological claim.  Nevertheless, I would wager on a social theory rather than a biological one.

My conception-date hypothetical may seem a little far-fetched but there are lots of alternatives.  The more promising ones to explore have to do with who is a bit older and younger when they first enter a school grade where the social pressures that cause anorexia start to manifest.  Keep in mind that anorexia undoubtedly has biological causal factors, but it is predominantly a social phenomenon.  Surely everyone has read the example, popularized by Malcolm Gladwell in Outliers, that professional hockey players who grew up in Canada are much more likely to have been born early in the year.  The explanation is that youth leagues form based on calendar-year birth cohorts, so the kids born in January are older, stronger, and more practiced than their contemporaries.  This would not matter by the time someone got to the NHL except for the fact that it matters a lot when someone is six years old, and the better players get more attention, and so become even better, and so play in better leagues as they move up the ladder, which makes them a bit better still, and so on.  Such a positive feedback loop seems less likely for anorexia, so the results would not be as dramatic.  But some story like that seems at least as plausible as a biological factor resulting from being a newborn in summer rather than winter manifests in a socially-constructed behavioral disorder. 

And surely the researchers must have read Outliers.  Even if you see that book, as some critics do, as just-so stories that rip off the clever ideas of other researchers without giving them due credit, it is still a must-read.  Kind of like The Spirit Level.  (That link is a plug for Snowdon's takedown of that book as a s/o for giving me this great example.)