Unhealthful News 189 - Absurd claims about the effects of smoking place restrictions, North Carolina edition (Part 1)

I was asked to comment on a report from the state of North Carolina which claims that their early 2010 implementation of a rule that prohibited smoking in a few public places where it was previously allowed (bars and restaurants) reduced emergency hospital admission for MI (heart attack) by 21%.  This story broke a few days ago, and Snowdon has already written about it, as has Michael Siegel.  Both of them offered the observation that this claim is just an artifact of complex modeling that can generate any result you might want.  Snowdon already pointed out that the downward time-trend in heart attacks actually flattened out after the ban (i.e., it was dropping over time, but dropped less after the ban than before).  That is pretty much all you need to know.  The time trend is by far the dominant statistic, and anything else has to be measured against it.  It is, of course, possible that the ban saved some would-be MIs, but since the time trend lessened, there is no possible way anyone can claim to see the result in the data.

(Siegel added the observation that for women there was actually an increase in MIs after the ban (it was less than the decrease for men, so the net was the continuing decline), though it is not actually clear that this means much -- after all, if men were the ones primarily "saved" from second-hand smoke, this is what we would expect to see.  I am not inclined to make much of this observation, since the report authors did not pull the obvious junk science trick of reporting just the result for men, trying to gloss over the result for women.  Just as it is always possible to find a subgroup that exaggerates an observed/claimed population effect, it is always possible to find one that runs counter.)

So, the main message is already out there, but I think I can add two things to it to bracket it:  (1) At a level that most anyone can understand, the NC claim is utterly implausible, regardless of what the statistical analysis says.  This is in keeping with my goal of showing how thoughtful people can often analyze science -- and call bullshit on it where appropriate -- without needing to understand all of the arcane details.  (2) I can also provide some additional insight into the statistical modeling, from the perspective of someone who can do statistics like that, and more important, has observed the behavior of other people who do it.  This is kind of a big topic, one that I wrote a paper about once, though never got around to publishing, so I will start with this post and then continue it.

What does it mean to claim that a particular intervention reduced a disease by 21%?  It sounds impressive.  Indeed it is.  It means that whatever it was that the intervention brought about -- in this case, the removal of second-hand smoke -- was causing one-fifth of the outcomes in question.  (Rounding to "one fifth" is a much more accurate way to describe the statistic -- reporting down to the last decimal place is good evidence that someone does not understand the limits of their statistics.)  So, second-hand smoke was causing one-fifth of all heart attacks?  Really?  That would make its impact roughly as great as that of smoking itself.  This is not even remotely plausible.  Right there is evidence that this result is wrong, and you do not need to know anything about how they did the calculation.

But, wait, it gets worse.  The claim is not that the totality of second-hand smoke exposure causes as many heart attacks as smoking, but that the fraction of exposure that is eliminated by the bar and restaurant ban was causing that much.  The more common and constant exposure in the home would not be eliminated; indeed it would probably increase as smokers gathered to drink somewhere they can smoke.  So the claim must actually be that second-hand smoke causes a lot more heart attacks than does smoking, up around half of all heart attacks, and this intervention eliminated roughly half of those.

But, wait, there's more.  The claim must be that exposure to second-hand smoke in restaurants and bars over a medium time period (roughly: measured in months) causes one-fifth of all heart attacks.  Epidemiologic studies, even those by anti-tobacco activists, have only been able to sometimes find an elevated risk in life-long nonsmoking spouses of smokers, or long-term workers in smoky environments.  But the smoking ban obviously did not eliminate lifetime exposure during its first year, the one year of data that was available.

Those who wish to defend this absurdity would undoubtedly reply with their "one puff" hypothesis, the claim that even a brief exposure to second-hand smoke can cause acute physiological effects that can trigger a heart attack in someone who is vulnerable.  But even setting aside whether that claim is plausible at all, it does not work in this scenario.  The claimed phenomenon is what is known, morbidly, as a "harvesting effect", triggering an event that is on the verge of happening a few weeks sooner.  Someone who was close enough to a heart attack in March that being in a smoking-permitted restaurant would have triggered it, but who avoids that event due to the new ban, is still on the verge and will likely encounter a similar trigger by April, or undoubtedly in June when hot North Carolina weather kills many of the vulnerable.  So, according to the story, some people are being saved from this trigger by a week or two or maybe ten.

If that were really the case, there would be a slight drop during the year after the ban, but it would be very slight since basically only a few weeks of heart attacks would be eliminated from the year:  The ones from the first week after the ban would just be shifted to later in the year; those that would have happened at those times would be pushed later, and so there would be close to a wash; and only those that would have happened at the end of the year would be pushed beyond the range of the data and thus represent a reduction.  It might be interesting to see if there was a drop in that first week, because that would be a good test of the "one puff" harvesting claim.  But that would be only a matter of scientific inerest, not a substantial effect on public health.

So, for there to be a major reduction due to this intervention, it needs to be the case that many heart attacks are not caused by accumulated lifetime exposure (which is not changed much) or a immediate-term trigger (which is only delayed), but that exposure accumulated over the last few weeks or months causes heart attacks that never would have happened or would have happened much later.  This story suffers from both the fact that there are no models or epidemiologic results to support it and because of the enormous portion of all heart attacks that would then have to be caused by second-hand smoke.  The claim would be that medium-term exposure in restaurants and bars causes one-fifth of all heart attacks, and so trigger-term and long-term exposure in those venues cause more still, and exposure in the home and other venues must cause at least that many again, which totals to a substantial majority of all heart attacks.  So if we can just eliminate the smoke, it looks like we can stop worrying about obesity and lack of exercise.  Even smoking itself is looking pretty good, as long as you ventilate the ambient smoke.

So, who would be stupid enough to believe this claim?

Since this is Unhealthful News, you can assume it includes the press.  More on that, and on the analysis, in Part 2.