When I read a report of the recent study that suggested that coffee consumption protects against stroke, it mentioned that the result controlled for several variables, including blood pressure. I immediately wondered if this was like the example of controlling for measures of blood lipids when studying alcohol and heart attacks, as I talked about in yesterday's post about confounding. It struck me that the beneficial effect might be an artifact of the detrimental effect of higher blood pressure (which is caused by caffeine use) being taken out, leaving an apparent benefit when the blood pressure effect was removed. There are a couple of ways this could play out – e.g., it could be that the negative effect of blood pressure was overestimated, leaving coffee drinkers looking healthier than they "should be" based on that, resulting in an apparent benefit. As I noted yesterday, you should not control for something on the pathway from the exposure of interest to the effect, and blood pressure is clearly on that pathway.
I should say that I was certainly not bothered by the possibility that lots of coffee is healthy. That is a delightful notion. But I had to wonder.
It turns out that the "blood pressure" measure was actually just a "yes or no" to some question about hypertension (presumably it was whether that had ever been diagnosed, though this was never clarified in the journal article). Also, the confounders did not seem to matter too much. To their credit, the authors reported what their effect estimate was for coffee adjusting only for age. (This is not quite adjusting for nothing but pretty close and since it did not seem to vary much among the exposure groups – those who consumed none or almost none, and three categories of increasing consumption – it probably does not matter.) That showed that their model that was adjusted for a bunch of factors got basically the same results. Therefore, they could not have been trying to mislead people as with the U.S. National Cancer Institute group I mentioned a few days ago. There were some things that made me a bit uneasy about the methods and claims, but obvious fraud was not on the list. (It seems that most researchers at the Karolinska Institute, where this study came from, are average or above-average; it may only be the ones researching snus that engage in unethical and then illegal practices.)
Mind you, I am not suggesting this study looked at confounders correctly, just that this seemed like it probably did not matter. After all, they still wrote, "we cannot rule out the possibility that our findings may be due to unmeasured or residual confounding" which, since it is always true, is kind of a silly thing to say. The authors still seem to have blindly thrown in whatever variables they happened to have, without regard to whether they should have been controlled as confounders or whether their relationship to the exposure and disease was such that they should not have been controlled for. As suggested in yesterday's post, properly controlling for covariates requires a lot more than this, even at a minimum. For example, I noticed that in this study population, body mass index and exercise averaged the same for each exposure group (as did most everything else measured), but calorie intake increased quite a lot as coffee consumption increased. Is coffee keeping heavy eaters from having health problems by burning off excess calories? Do coffee drinkers naturally have different metabolism? That is probably not an issue for the outcome in question, but the point is that if those variables should have been included in the model then the authors should have tried to figure out what was up with them. It is also possible that throwing in all the variables they had did not change the effect estimate, but controlling for only the variables that they should have controlled for might have changed it – we cannot tell because they did not report such models.
Of course, as with any study of this type, the important questions do not end there. There are also other explanations for the result like: the healthy worker effect (people with jobs tend to be healthier than those who lack them, often in ways that are unmeasured, and workers might drink more coffee – they do in a lot of populations, though I am not sure about older Swedish women); the unhealthy abstainer effect (some people who completely avoid common behaviors that might be unhealthy in some way do so because of a health problem that is not captured in the data); the Coke or tea that non-coffee drinkers consume causes stroke; and even the healthy survivor effect (the story would be that coffee actually triggers strokes at an age earlier than the study age in anyone who is prone to them, and so those coffee drinkers who survived without stroke to be in the study population were the ones who were not at risk). I am not suggesting that any of these is right, just that they are possible and could be tested.
Again, I think it would be great if they are indeed right and drinking six cups of coffee a day lowered your stroke risk by 20%. But I am a bit leery. Also, since other stimulants (decongestant medicines, ephedra-like drugs, nicotine) are widely believed (rightly or wrongly) to cause stroke, it is interesting that everyone is so quick to believe it and report the new result without skepticism. Hundreds of news sources covered it and I did not notice any of them expressing doubt, though obviously I did not review them all. Perhaps it is a plot by Big Coffee to mislead us. Or should that be Venti Coffee? After all, they spend millions on advertising their addictive product, add flavors to attract children who get hooked and then graduate to the hard stuff, and use sexually explicit cartoon images of ship-crashing temptresses – what more do you need to know?
I should say that I was certainly not bothered by the possibility that lots of coffee is healthy. That is a delightful notion. But I had to wonder.
It turns out that the "blood pressure" measure was actually just a "yes or no" to some question about hypertension (presumably it was whether that had ever been diagnosed, though this was never clarified in the journal article). Also, the confounders did not seem to matter too much. To their credit, the authors reported what their effect estimate was for coffee adjusting only for age. (This is not quite adjusting for nothing but pretty close and since it did not seem to vary much among the exposure groups – those who consumed none or almost none, and three categories of increasing consumption – it probably does not matter.) That showed that their model that was adjusted for a bunch of factors got basically the same results. Therefore, they could not have been trying to mislead people as with the U.S. National Cancer Institute group I mentioned a few days ago. There were some things that made me a bit uneasy about the methods and claims, but obvious fraud was not on the list. (It seems that most researchers at the Karolinska Institute, where this study came from, are average or above-average; it may only be the ones researching snus that engage in unethical and then illegal practices.)
Mind you, I am not suggesting this study looked at confounders correctly, just that this seemed like it probably did not matter. After all, they still wrote, "we cannot rule out the possibility that our findings may be due to unmeasured or residual confounding" which, since it is always true, is kind of a silly thing to say. The authors still seem to have blindly thrown in whatever variables they happened to have, without regard to whether they should have been controlled as confounders or whether their relationship to the exposure and disease was such that they should not have been controlled for. As suggested in yesterday's post, properly controlling for covariates requires a lot more than this, even at a minimum. For example, I noticed that in this study population, body mass index and exercise averaged the same for each exposure group (as did most everything else measured), but calorie intake increased quite a lot as coffee consumption increased. Is coffee keeping heavy eaters from having health problems by burning off excess calories? Do coffee drinkers naturally have different metabolism? That is probably not an issue for the outcome in question, but the point is that if those variables should have been included in the model then the authors should have tried to figure out what was up with them. It is also possible that throwing in all the variables they had did not change the effect estimate, but controlling for only the variables that they should have controlled for might have changed it – we cannot tell because they did not report such models.
Of course, as with any study of this type, the important questions do not end there. There are also other explanations for the result like: the healthy worker effect (people with jobs tend to be healthier than those who lack them, often in ways that are unmeasured, and workers might drink more coffee – they do in a lot of populations, though I am not sure about older Swedish women); the unhealthy abstainer effect (some people who completely avoid common behaviors that might be unhealthy in some way do so because of a health problem that is not captured in the data); the Coke or tea that non-coffee drinkers consume causes stroke; and even the healthy survivor effect (the story would be that coffee actually triggers strokes at an age earlier than the study age in anyone who is prone to them, and so those coffee drinkers who survived without stroke to be in the study population were the ones who were not at risk). I am not suggesting that any of these is right, just that they are possible and could be tested.
Again, I think it would be great if they are indeed right and drinking six cups of coffee a day lowered your stroke risk by 20%. But I am a bit leery. Also, since other stimulants (decongestant medicines, ephedra-like drugs, nicotine) are widely believed (rightly or wrongly) to cause stroke, it is interesting that everyone is so quick to believe it and report the new result without skepticism. Hundreds of news sources covered it and I did not notice any of them expressing doubt, though obviously I did not review them all. Perhaps it is a plot by Big Coffee to mislead us. Or should that be Venti Coffee? After all, they spend millions on advertising their addictive product, add flavors to attract children who get hooked and then graduate to the hard stuff, and use sexually explicit cartoon images of ship-crashing temptresses – what more do you need to know?
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.