Any of you who read the health news today probably predicted what I would write about. According to a widely-reported story, heavy beer drinkers are supposedly at high risk of gastric (stomach) cancer. This is all based on a conference presentation, so all we have to go on is what was in the news stories (presumably based on a press release touted by the researchers who have not actually provided us with a paper to look at). Presumably the study results make a bit more sense than what the press reported, doing so without critical analysis of course.
The result was based on a Western European survey of a cohort interviewed and presumably genetically tested in 1992 and 1998. The highlighted result was that those who drank 2-3 beers per day for "many years" had a relative risk of gastric cancer of about 1.7 (which they called a 75% increase because that sounds more impressive). If they had a particular genetic variant called rs1230025 they had a RR of 1.3. But if they had both exposures then they had a "more than 700% increase" (i.e., a RR over 8). Rounding out the inputs for the rather obvious calculation that no one bothered to carry out is that about 20% of the European population has rs1230025. So let's see: If the 20% of the drinkers have a RR of 8, that alone would increase the average RR for the drinkers as a whole to roughly 2.4. (Take that extra 700% for the 20% of the population and divide it across all the drinkers – 7 x .2 = 1.4 – and add that to the baseline risk to get 2.4.) This is substantially higher than the population average for the drinkers of 1.7. The only apparent explanation is that the beer is substantially protective for the 80% without rs1230025 cutting their risk by more than half.
So, if we believe the results that are claimed (hint: don't believe them) then everyone should get tested for that gene. Those who do not have rs1230025 should make sure to drink 2-3 beers per day. Those with the variant should make sure to stop at 1 – or keep going to 4. Perhaps that last option would not work out well, but it is hard to know what happens after 3 since the exact exposure definition was not reported. The stories are actually very detailed for health news stories (thus I can write this analysis) but still do not tell us enough.
So why just beer, you might ask? Well they did find a elevated risk for alcohol more generally, but apparently it was not nearly as interesting as the clever results they could distill for the beer data. This seems like a pretty clear case of data dredging, finding an interesting association in the data – as you pretty much always can if you look hard enough – and reporting it as if it means something. The authors proposed that a suspected carcinogenic chemical, NDMA, that is in beer in trace amounts is responsible, but it is always possible to fish around for such a hypothesis, so this does not actually tell us anything.
Why rs1230025, you might ask? Well apparently it is involved in alcohol metabolism, so is plausibly connected to the health effects of drinking. This is reassuring since we need some reason to consider a particular gene: If you fish through thousands of genetic variations you will certainly find some strong associations with any outcome you are studying, and this will be utterly uninformative without corroboration – very much like finding a cancer cluster, if you recall what I wrote about that: you might have found something real, but chances are it was just random spike. But wait, you might ask (though no reporter did), if the gene is of interest for the study because of alcohol metabolism, would not we expect to see the same effect regardless of drink? Why would it affect the beer-specific effect the authors are attributing to NDMA. An excellent question indeed.
Could it be that the entire headline-generating set of results was the result of fishing for interesting random error? (That is, is there enough random error in the study due to limited sample size that the authors could hunt through different exposure variations and probably find something interesting due to chance?)
It is certainly possible. We do not know quite how much random error there was, but it seems great enough. Some observers were undoubted tricked into believing this was functionally a very large study by the fact that it included over 500,000 subjects, but that is actually not a useful measure. The incidence rate of gastric cancer in the study countries is around 10 per 100,000 person years, which means about 50 cases per year in this population if they have the same age distribution as the overall population. If they skew even a bit younger then it will be even less since most cases are at the old end of the distribution. It is not clear how many years of follow-up they have for the average study participant, but it is probably less than 10. So we are talking about less than 500 cases. People who consistently drink 2-3 beer (specifically) per day for a long time seems unlikely to be as much as 5% of the population, so there would be about 20 cases with that exposure by coincidence alone, and thus would be about 30 in the data given the elevated risk. That is not a tiny number, obviously, but it leaves room for random error when fishing. What about "got gastric cancer"+"has that drinking pattern"+"has rs1230025"? That would be in the order of 4 cases by coincidence so 20-some to produce the RR of 8.
That is still a lot of extra cases compared to coincidence, and may well represent some real association, but it is not such a big number given the number of comparisons that could be considered in the fishing expedition. Since this particular comparison was chosen after looking at the data, choosing which of many combinations of drink type, exactly quantity, genetic characteristic, etc. to report based on it being an outlier, it would be almost certain that any association would be with a broader category rather than those exact people, and be a lot smaller. Recall that rs1230025 should affect all alcohol consumption, not beer specifically.
And, of course, the study author declared that this result means that people should avoid heavy consumption of alcohol, without doing any analysis of the actual costs and benefits (and ignoring that his results actually argue against moderate consumption of alcohol -- 2 beers per day -- which we have strong evidence is good for you). That does not necessarily make his study results wrong, but it calls his judgment and honesty into question. There are good reasons to not drink heavily, but this claim is not one of them, at least not yet. And obviously one risk is not sufficient to proscribe a behavioral choice. Of course, the anti-alcohol (and general anti-free to live one's own life) activists will leap on this to support their advice (or pushes for mandates) despite the lack of useful information. The only sensible piece of advice I read was from someone unrelated to the study who hinted that the genetic variation issue means that if you have a family history of gastric cancer, you should consider it likely that drinking is higher risk for you than for others.
I doubt anyone is going to advise people who are rs1230025-negative to make sure to drink beer, however, despite the study apparently showing a protective effect for them. Do you think maybe someone could figure out that if part of the advice you might give based on a study's results seems badly wrong, and you can figure this out even after downing several beers, that perhaps none of the study results should be translated into advice until we can figure out more. A little more understanding of the limits of research like this – by reporters and readers – might avoid the problems inherent in giving any advice based on this study. It is not too difficult to figure out that it is unlikely to look so good upon sober reflection the next morning.
The result was based on a Western European survey of a cohort interviewed and presumably genetically tested in 1992 and 1998. The highlighted result was that those who drank 2-3 beers per day for "many years" had a relative risk of gastric cancer of about 1.7 (which they called a 75% increase because that sounds more impressive). If they had a particular genetic variant called rs1230025 they had a RR of 1.3. But if they had both exposures then they had a "more than 700% increase" (i.e., a RR over 8). Rounding out the inputs for the rather obvious calculation that no one bothered to carry out is that about 20% of the European population has rs1230025. So let's see: If the 20% of the drinkers have a RR of 8, that alone would increase the average RR for the drinkers as a whole to roughly 2.4. (Take that extra 700% for the 20% of the population and divide it across all the drinkers – 7 x .2 = 1.4 – and add that to the baseline risk to get 2.4.) This is substantially higher than the population average for the drinkers of 1.7. The only apparent explanation is that the beer is substantially protective for the 80% without rs1230025 cutting their risk by more than half.
So, if we believe the results that are claimed (hint: don't believe them) then everyone should get tested for that gene. Those who do not have rs1230025 should make sure to drink 2-3 beers per day. Those with the variant should make sure to stop at 1 – or keep going to 4. Perhaps that last option would not work out well, but it is hard to know what happens after 3 since the exact exposure definition was not reported. The stories are actually very detailed for health news stories (thus I can write this analysis) but still do not tell us enough.
So why just beer, you might ask? Well they did find a elevated risk for alcohol more generally, but apparently it was not nearly as interesting as the clever results they could distill for the beer data. This seems like a pretty clear case of data dredging, finding an interesting association in the data – as you pretty much always can if you look hard enough – and reporting it as if it means something. The authors proposed that a suspected carcinogenic chemical, NDMA, that is in beer in trace amounts is responsible, but it is always possible to fish around for such a hypothesis, so this does not actually tell us anything.
Why rs1230025, you might ask? Well apparently it is involved in alcohol metabolism, so is plausibly connected to the health effects of drinking. This is reassuring since we need some reason to consider a particular gene: If you fish through thousands of genetic variations you will certainly find some strong associations with any outcome you are studying, and this will be utterly uninformative without corroboration – very much like finding a cancer cluster, if you recall what I wrote about that: you might have found something real, but chances are it was just random spike. But wait, you might ask (though no reporter did), if the gene is of interest for the study because of alcohol metabolism, would not we expect to see the same effect regardless of drink? Why would it affect the beer-specific effect the authors are attributing to NDMA. An excellent question indeed.
Could it be that the entire headline-generating set of results was the result of fishing for interesting random error? (That is, is there enough random error in the study due to limited sample size that the authors could hunt through different exposure variations and probably find something interesting due to chance?)
It is certainly possible. We do not know quite how much random error there was, but it seems great enough. Some observers were undoubted tricked into believing this was functionally a very large study by the fact that it included over 500,000 subjects, but that is actually not a useful measure. The incidence rate of gastric cancer in the study countries is around 10 per 100,000 person years, which means about 50 cases per year in this population if they have the same age distribution as the overall population. If they skew even a bit younger then it will be even less since most cases are at the old end of the distribution. It is not clear how many years of follow-up they have for the average study participant, but it is probably less than 10. So we are talking about less than 500 cases. People who consistently drink 2-3 beer (specifically) per day for a long time seems unlikely to be as much as 5% of the population, so there would be about 20 cases with that exposure by coincidence alone, and thus would be about 30 in the data given the elevated risk. That is not a tiny number, obviously, but it leaves room for random error when fishing. What about "got gastric cancer"+"has that drinking pattern"+"has rs1230025"? That would be in the order of 4 cases by coincidence so 20-some to produce the RR of 8.
That is still a lot of extra cases compared to coincidence, and may well represent some real association, but it is not such a big number given the number of comparisons that could be considered in the fishing expedition. Since this particular comparison was chosen after looking at the data, choosing which of many combinations of drink type, exactly quantity, genetic characteristic, etc. to report based on it being an outlier, it would be almost certain that any association would be with a broader category rather than those exact people, and be a lot smaller. Recall that rs1230025 should affect all alcohol consumption, not beer specifically.
And, of course, the study author declared that this result means that people should avoid heavy consumption of alcohol, without doing any analysis of the actual costs and benefits (and ignoring that his results actually argue against moderate consumption of alcohol -- 2 beers per day -- which we have strong evidence is good for you). That does not necessarily make his study results wrong, but it calls his judgment and honesty into question. There are good reasons to not drink heavily, but this claim is not one of them, at least not yet. And obviously one risk is not sufficient to proscribe a behavioral choice. Of course, the anti-alcohol (and general anti-free to live one's own life) activists will leap on this to support their advice (or pushes for mandates) despite the lack of useful information. The only sensible piece of advice I read was from someone unrelated to the study who hinted that the genetic variation issue means that if you have a family history of gastric cancer, you should consider it likely that drinking is higher risk for you than for others.
I doubt anyone is going to advise people who are rs1230025-negative to make sure to drink beer, however, despite the study apparently showing a protective effect for them. Do you think maybe someone could figure out that if part of the advice you might give based on a study's results seems badly wrong, and you can figure this out even after downing several beers, that perhaps none of the study results should be translated into advice until we can figure out more. A little more understanding of the limits of research like this – by reporters and readers – might avoid the problems inherent in giving any advice based on this study. It is not too difficult to figure out that it is unlikely to look so good upon sober reflection the next morning.
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.