Unhealthful News 191 - Absurd claims about the effects of smoking place restrictions, North Carolina edition (Part 3)

(Links to Part 1 and Part 2.  Sorry for the long delay in finishing this the series.  First I was at the tobacco harm reduction sessions that were part of the TabExpo conference which I blogged about at the tobacco harm reduction blog, then my computer broke and I wanted to wait until I could access what I had written it rather than rewriting it, and then holidays.)

In the previous two parts, I offered a back-of-the-envelope assessment of the possible impact of a restaurant/bar smoking ban on heart attacks.  I estimated that the effect, if it exists, would be in the order of 1% of the population’s total, and only that high if we take near-worst-case estimates of the effects of second-hand smoke and assume, without scientific support, that intermittent medium-term exposure accounts for much of the effect.  Let us now set aside the latter parts of that sentence and just assume that we are trying to detect a 1% decrease.  

Is it possible to detect a 1% change in something?  Yes, of course, but only under the right circumstances.  If we have a rigid object with a length X, and we are about to do something that we think will change the length to .99*X, then two measurements -- one before and one after -- would suffice to confirm this, assuming we have an accurate measuring device and were very confident in our ability to use it precisely.  But, of course, for the heart attack case we are not talking about something that was fixed and constant for all time before the event and again after at its new value.  The most important difference is that we are counting up random events that result in a series of realizations of an incidence rate that we are hypothesizing has dropped by 1%.

Is it possible to detect a 1% change in such a probability of a random draw?  Yes, of course, but only if you have enough observations and that some other conditions hold.  Imagine you have a coin that you have altered so that you hypothesize that when you flip it, it lands heads only 49.5% of the time, a 1% decrease from what it did before.  Could we throw the coin enough times to detect the change with a good confidence?  Yes, but the number of heads we would need to throw to have confidence in the observation would be greater than the number of heart attacks observed in the North Carolina results.  What does this mean for those results?  It means that even setting aside other complications, pretending that the pattern of heart attacks was as regular and reliable as flipping a single coin, we would still have enough random noise that it would be difficult to detect the signal.

In that scenario in which heart attacks are like coin flips, however, it would be extremely unlikely that we would err so much as to estimate an effect 20 times as great as the plausible true maximum.  So what happened?

The problem was that the "some other conditions hold" caveat I stated was not met -- not by a long shot -- and the analysts tried to deal with this by forcing the data into a complicated model.  Instead of just looking at before and after collections of a few tens of thousands of coin flips that vary only as a result of the one change, they were trying to deal with a series that showed lage changes over time that had nothing to do with the event they were trying to assess.  In particular, there was a downward trend in heart attacks over time.  So obviously if you just compare before the change with after, the latter will show a reduction.  This is exactly what some of the lowest-skilled dishonest analysts have done over the years, effectively claiming that the impact of the downward trend that existed before the smoking ban was due to the ban when it continued afterwards.

More sophisticated dishonest analysts used complicated models to try to trick naive readers (i.e., policy makers, news reporters, and 99.9% of everyone else) into believing that they had accounted for this obvious problem.  But before getting to them, what would an honest and sophisticated analyst do?  The answer in this North Carolina case is:  

Give up without even bothering to try.

As I already noted, merely being able to detect the hypothesized signal in the noise, assuming the ban is the only change between before and after, requires a bit more data than all that was used for this analysis.  Using up some of the statistical power to model the downward trend, even if it were a very simply shaped curve and you knew the shape, would leave even less power to detect the impact of the policy shock.  So an honest analyst who knew what he was doing would insist on getting a lot more data before doing the analysis.  And as it turns out, honest analysts who have gathered such sufficient data, for much larger populations with longer periods for estimating the time trend, have found no measurable effects on heart attacks from smoking bans.

So what did the present analysts do?  The only approach that would have any hope of working would be to assume that the downward trend was constant (in some sense, such as the percentage reduction from year to year was constant) except for the effect of the ban.  But that option did not work for the government employees who were tasked with "proving" that their department's pet regulation had a big effect. Sadly for them, the time trend clearly flattened out, so the gains after were less than those from before.  If the trend had accelerated they might well have claimed it was caused by the ban, but because it decelerated, they were not willing to do that simple analysis, which would blame the reduction in reduction on the ban.  

So they set out to use other data to try to explain the time trend.  This is not wrong in theory.  After all, the time trend is being caused by something -- it is not just a magical effect of calendar pages turning.  So if we had all the data in the world and knew what to do with it, we would not have to deal with the trend itself since we could predict the rate of heart attacks at any given time, however it was trending, with the other data.  But here we run into the problem again of not having nearly enough data, not only not enough observations (events) but not enough other variables to really explain the changes over time.  Very sophisticated analysts with lots of data might attempt to explain complicated phenomena like this.  

Such sophistication is more common in economics, but there are a few examples in public health, like the attempt to estimate the mortality effects of outdoor air pollution:  By collecting daily data on mortality, air pollution, weather, and many other variables from multiple cities for years, researchers attempted to estimate the effects of the air pollution.  Unfortunately, this was extremely difficult because hot weather is strongly correlated with high air pollution days, and the hot weather itself is at least ten times as deadly as the worst case estimate for the pollution, so basically the estimate for the effects of pollution is determined by exactly what is estimated to be the effect of weather -- make the estimate for that a bit low and it will look like pollution is doing a lot more harm than it really is.  The air pollution research is notorious for the researchers making a technical error in their first published estimate, and having to revise their estimate down by half.  (Added bonus lesson for the day:  Do not believe the oft-repeated claims about how many people are killed by outdoor air pollution.  They are little more than guesses.)

In the case of the smoking ban, it is the time trend that has effects in the order of ten times the hypothesized effect, but the implication is the same:  Unless you have a very good estimate of the effect of that other factor, or have enough data to figure it out, there is no way to estimate the effect of interest.  So, once again, no honest analyst who knew what he was doing would attempt this.  

A dishonest analyst, however, would find that he had all manner of options for getting the result he wanted by using different combinations of the variable he has and employing many different statistical forms.  The analysts could experiment with different options and report only one of them, as if it were the only one tried and as if it were somehow the right choice among the many different models.  This is the most nefarious type of what I labeled publication bias in situ, and is almost certainly what the smoking ban advocates have done in the various cases where they used complicated analyses to “show” that the effects of the ban are far greater than is plausible.

Finally, we might ask what an honest research might do if tempted to just give this a go, even realizing that the chances are that it would not be possible to get a stable estimate (i.e., one that does not change a lot as a result of whims of model details or the random sampling error in the data).  One thing that would be required would be to do some tests to see if the estimate was sensitive to reasonable changes in the model or data, and most importantly to report the results of those tests.  To their credit, the authors of the NC study actually did a bit of that.  You would never know it from reading the political documents surrounding the study, like the press release, but they did one of the more telling tests:  They took their model and calculated what it would estimate the effects of the ban were if it had been implemented at a different time.  That is, they kept the same data and used the model to estimate the apparent effect of a new regulation that started at a different time from when it really did.  The results for the two alternative times they report are a 27% decrease in heart attacks (recall that the touted “result” of the study was a 21% reduction) and a 12% increase.  That is, during months when their estimate of the effect of the new ban should have been zero (since it did not happen then), the estimates ranged from bigger than their estimated effect from the actual ban to a substantial effect in the other direction.  Put another way, their model is generating noise, and the 21% result is just an accident of when the ban was implemented and the details of their model; had the ban been implemented a month sooner or later, the same data would have "shown" a very different effect, though almost certainly one that was still far larger than was plausible, one way or the other.  They could have just as easily have gotten any other number within tens of percentage points.

And maybe they did.  That is, maybe they tried models that produced those results and buried them.  But I am thinking maybe not.  After all, the analysts would not have even reported the results of the little validation analysis if they were trying hard to lie.  If they were ASH-UK or Glantz, they would have buried those results or, more likely, never done the test.  If I had to guess, I might go with the story that the analysts tried to do an honest job and report that their model could not really find anything, but their political bosses insisted that they report something favorable without caveat.  The analysts left in the information that shows the political claim to be a lie because they could get away with that in the text.  The “public health” politicos are not smart enough to understand what that means, if they take the time to read it at all.  If that is really the story, however, it is not so good -- anyone who would allow the politicos to claim that their analysis showed something that it clearly did not, and who stayed in their job and accepted the behavior, shares much of the guilt even if they tried to sneak the truth in.

So, that's pretty much the story. We can estimate how bit the effect might theoretically be, and that tells us that it is either very small or zero. We can observe that there is not enough data to see an effect that small. But some random luck will generate an impressive overestimate of the true effect a lot of the time, and intentional misrepresentation of what the data shows can almost guarantee it, we hear about bit "effects". Everyone involved in the exercise to show that bans of smoking in a few places have miraculous effects is either dishonest, clueless about basic quantitative analysis, or both. There is simply no other explanation.

Unhealthful News 190 - Absurd claims about the effects of smoking place restrictions, North Carolina edition (Part 2)

In my previous post, I started commenting on the absurdity of a state of North Carolina claim that their recent ban on smoking in restaurants and bars caused a 21% reduction in heart attacks.  I presented part of a generic analysis, from an unpublished paper I wrote a few years ago, that basically says "if you are claiming that a such a ban caused such a large reduction in a disease rate then you are obviously wrong, regardless of what your statistical model says, and anyone who gives it a few minutes' thought should understand this".  In this post I make basically the same point, looking at it from a different angle, in case that might be clearer for some readers.

Start by considering the epidemiologic estimates for the increase in risk from second-hand smoke.  The evidence, when assessed by someone who is not intent on promoting smoking bans, puts the risk at so close to zero that it is impossible to say from the data that risk definitively exists.  (There are decent theoretical reasons to surmise that there is some risk, so it seems safe to assume that the risk is not zero.  But it is small.)  A few studies of people who have experienced the most extreme long-term exposure get numbers like a 20% increase in risk.  As is inevitable with random sampling and publication bias, there are a few that go tens of percentage points higher.

So let us consider the possibility that the risk is a bit higher that 20% -- that is, a nonsmoker who is exposed to second-hand smoke is over 20% likely to have a heart attack, at all or much sooner, compared to that person not being exposed.  (Re the "much sooner" point, see the observation from Part 1 that a very-short-term harvesting effect would wash out of the annual statistics.)  This number is unrealistically high and at most might be considered a worst-case estimate of the risk for those with the highest accumulated lifetime exposure.  But even if it were the average effect for those with passing exposure at smoking-allowed restaurants and bars, it would obviously be far higher than the effect of that exposure averaged across the whole population.  Only people who were exposed in the first place would have that risk, and only those who go from exposed to unexposed as a result of an intervention can benefit from it.

How many people go from being exposed to restaurant/bar smoke to unexposed as a result of the ban?  It is a bit fuzzy to define this since there will be a lot of people whose exposure is reduced, and a spectrum of how much it is reduced.  But we can start with the observation that roughly half of everyone had approximately zero such exposure before the ban, never or almost never going out to eat and drink, or avoiding smoking-allowed venues when they did.  (To really get this right, we would need to figure out the portion not of people but of total risk -- a 20% risk increase for an exposed 70-year-old would cause a lot more absolute risk than the same percentage would for the 25-year-olds who pack some bars -- but it seems likely this would strengthen the point I am making, since the older highest-risk people tend to go out to party less.)  Thus, even if you believed that exposure at the level of visiting restaurants and bars causes somewhat more than 20% increase in risk, which is an absurd belief in itself, there is no possible way the effect of the smoking ban could be more than about half of the claimed 21%.

Not only are there a lot of people who were not exposed in the first place, but many of those who are exposed are smokers (where do you think the smoke comes from?).  No one seriously claims that the minor increase in exposure from second-hand smoke dramatically increases the risk for a smoker, on top of the already substantial risk increase from smoking.  Perhaps it does somewhat, but it is going to be a lot smaller than the effect on a nonsmoker.  Many others who are no longer exposed in bars after the ban are still exposed at home -- perhaps more since their smoking spouses do more of their smoking at home or in the car before arriving at a venue.  Furthermore, most of the people who experience a substantial reduction in their total exposure -- all but the nonsmoking workers and hardcore nonsmoking barflies, rather tiny percentages of the population -- experience a reduction of an exposure that was far less than the extreme exposures that sometimes generate measurable effects in epidemiologic studies.

This is enough to show that the 21% estimate is utterly implausible.  Taking it further, what does this way of looking at it suggest would be a plausible maximum effect of a bar/restaurant smoking ban?

To start, even a 5% increase in risk from the bar/restaurant exposure would be a high estimate of the effect for everyone except the aforementioned workers and barflies.  We can figure that half of the population was not exposed in the first place, that easily a third of those exposed were smokers, that many of those exposed had very minor and occasional exposure, and that many others that were exposed had only a minor reduction in exposure since most of their exposure was elsewhere.  So it seems unlikely that even one-fifth of the population experienced a substantial reduction in exposure, getting the effect down below 1% of the total.  Even if we allow for a greater effect for the small highly-exposed minority, as well as some small effect for those with a very small reduction in their total exposure, it is difficult to come up with any at-all-plausible scenario that results in a reduction of more than about 2%.  (And keep in mind that this still depends on assuming the 5% increase in risk in the first place, something that is largely speculative.  Thus the real figure could be much lower than even this.)

Perhaps the full details of this analysis might call for more than common sense, but I have to assume that most people who thought about it would realize that claims of 10% reduction, let alone 20%, are completely incompatible with reality.  This brings us back to the question I asked in Part 1:

So, who would be stupid enough to believe this claim?

I suppose I phrased that too harshly to be a general statement:  The average casual reader of the news does not have time to think through most of the claims that they hear -- about the benefits of wars, the causes of unemployment, or health claims -- so their failure to question the claim should not be attributed to poor judgment on their part.  They just do not have time to judge.  But I will not back off on the harsh accusation when talking about news reporters and other opinion leaders who spend more than a few minutes on the topic.

Several North Carolina local news outlets reported the story without a hint of questioning the result.  Once again, it becomes apparent that the journalism curriculum for health reporters no longer includes the classes that teach governments lie habitually and that perhaps when someone (anyone) puts out a press release that claims "hey, everyone, look!  statistics show that the decision we made was a good one and did everything we said it would" it is perhaps not best to just assume they are correct and transcribe their claims.  Can you imagine if these guys were teachers?  "Class, now that you have finished your quiz, I am putting the correct answers up on the screen.  Please grade yourself and write your score on the grade sheet that I am passing around."

The good news might be that the national press was so bored of these claims (not critical, just bored) that it does not appear to have been picked up by any national news outlet.  But that did not stop Stanton Glantz and his junk science shop at UCSF from posting about it (h/t to Snowdon for reporting that post), and you can count on it showing up in future national news stories where these hacks are quoted.  We would not expect the thoughtful analysis like the above from these people; we can count on them to repeat (repeatedly) any absurd claim like the one from NC as if it were correct.  Indeed, we could count on them to conveniently ignore any result that was down in the realistic range.

(Q: How do you know if Stanton Glantz is spouting junk science in support of his personal political goals, damaging both science and public policy?  A: They have not announced his funeral yet.  Interestingly, it is not entirely clear whether he spouts junk because he has not acquired a modicum of understanding about the science in the field where he has worked for decades, or because he is a sociopath-level liar; I am not entirely sure which is the more charitable interpretation.)

So that is the easy side of this analysis, wherein reporters transcribe claims that are obviously wrong and extremist activists embrace them because they are wrong.  In Part 3 I will go into some details of the modeling that are beyond the abilities of reporters and junk-science activists, but that emphasize that those who reported the results are lying and/or are attempting analyses that are way beyond their abilities, and presumably know it.

Unhealthful News 189 - Absurd claims about the effects of smoking place restrictions, North Carolina edition (Part 1)

I was asked to comment on a report from the state of North Carolina which claims that their early 2010 implementation of a rule that prohibited smoking in a few public places where it was previously allowed (bars and restaurants) reduced emergency hospital admission for MI (heart attack) by 21%.  This story broke a few days ago, and Snowdon has already written about it, as has Michael Siegel.  Both of them offered the observation that this claim is just an artifact of complex modeling that can generate any result you might want.  Snowdon already pointed out that the downward time-trend in heart attacks actually flattened out after the ban (i.e., it was dropping over time, but dropped less after the ban than before).  That is pretty much all you need to know.  The time trend is by far the dominant statistic, and anything else has to be measured against it.  It is, of course, possible that the ban saved some would-be MIs, but since the time trend lessened, there is no possible way anyone can claim to see the result in the data.

(Siegel added the observation that for women there was actually an increase in MIs after the ban (it was less than the decrease for men, so the net was the continuing decline), though it is not actually clear that this means much -- after all, if men were the ones primarily "saved" from second-hand smoke, this is what we would expect to see.  I am not inclined to make much of this observation, since the report authors did not pull the obvious junk science trick of reporting just the result for men, trying to gloss over the result for women.  Just as it is always possible to find a subgroup that exaggerates an observed/claimed population effect, it is always possible to find one that runs counter.)

So, the main message is already out there, but I think I can add two things to it to bracket it:  (1) At a level that most anyone can understand, the NC claim is utterly implausible, regardless of what the statistical analysis says.  This is in keeping with my goal of showing how thoughtful people can often analyze science -- and call bullshit on it where appropriate -- without needing to understand all of the arcane details.  (2) I can also provide some additional insight into the statistical modeling, from the perspective of someone who can do statistics like that, and more important, has observed the behavior of other people who do it.  This is kind of a big topic, one that I wrote a paper about once, though never got around to publishing, so I will start with this post and then continue it.

What does it mean to claim that a particular intervention reduced a disease by 21%?  It sounds impressive.  Indeed it is.  It means that whatever it was that the intervention brought about -- in this case, the removal of second-hand smoke -- was causing one-fifth of the outcomes in question.  (Rounding to "one fifth" is a much more accurate way to describe the statistic -- reporting down to the last decimal place is good evidence that someone does not understand the limits of their statistics.)  So, second-hand smoke was causing one-fifth of all heart attacks?  Really?  That would make its impact roughly as great as that of smoking itself.  This is not even remotely plausible.  Right there is evidence that this result is wrong, and you do not need to know anything about how they did the calculation.

But, wait, it gets worse.  The claim is not that the totality of second-hand smoke exposure causes as many heart attacks as smoking, but that the fraction of exposure that is eliminated by the bar and restaurant ban was causing that much.  The more common and constant exposure in the home would not be eliminated; indeed it would probably increase as smokers gathered to drink somewhere they can smoke.  So the claim must actually be that second-hand smoke causes a lot more heart attacks than does smoking, up around half of all heart attacks, and this intervention eliminated roughly half of those.

But, wait, there's more.  The claim must be that exposure to second-hand smoke in restaurants and bars over a medium time period (roughly: measured in months) causes one-fifth of all heart attacks.  Epidemiologic studies, even those by anti-tobacco activists, have only been able to sometimes find an elevated risk in life-long nonsmoking spouses of smokers, or long-term workers in smoky environments.  But the smoking ban obviously did not eliminate lifetime exposure during its first year, the one year of data that was available.

Those who wish to defend this absurdity would undoubtedly reply with their "one puff" hypothesis, the claim that even a brief exposure to second-hand smoke can cause acute physiological effects that can trigger a heart attack in someone who is vulnerable.  But even setting aside whether that claim is plausible at all, it does not work in this scenario.  The claimed phenomenon is what is known, morbidly, as a "harvesting effect", triggering an event that is on the verge of happening a few weeks sooner.  Someone who was close enough to a heart attack in March that being in a smoking-permitted restaurant would have triggered it, but who avoids that event due to the new ban, is still on the verge and will likely encounter a similar trigger by April, or undoubtedly in June when hot North Carolina weather kills many of the vulnerable.  So, according to the story, some people are being saved from this trigger by a week or two or maybe ten.

If that were really the case, there would be a slight drop during the year after the ban, but it would be very slight since basically only a few weeks of heart attacks would be eliminated from the year:  The ones from the first week after the ban would just be shifted to later in the year; those that would have happened at those times would be pushed later, and so there would be close to a wash; and only those that would have happened at the end of the year would be pushed beyond the range of the data and thus represent a reduction.  It might be interesting to see if there was a drop in that first week, because that would be a good test of the "one puff" harvesting claim.  But that would be only a matter of scientific inerest, not a substantial effect on public health.

So, for there to be a major reduction due to this intervention, it needs to be the case that many heart attacks are not caused by accumulated lifetime exposure (which is not changed much) or a immediate-term trigger (which is only delayed), but that exposure accumulated over the last few weeks or months causes heart attacks that never would have happened or would have happened much later.  This story suffers from both the fact that there are no models or epidemiologic results to support it and because of the enormous portion of all heart attacks that would then have to be caused by second-hand smoke.  The claim would be that medium-term exposure in restaurants and bars causes one-fifth of all heart attacks, and so trigger-term and long-term exposure in those venues cause more still, and exposure in the home and other venues must cause at least that many again, which totals to a substantial majority of all heart attacks.  So if we can just eliminate the smoke, it looks like we can stop worrying about obesity and lack of exercise.  Even smoking itself is looking pretty good, as long as you ventilate the ambient smoke.

So, who would be stupid enough to believe this claim?

Since this is Unhealthful News, you can assume it includes the press.  More on that, and on the analysis, in Part 2.

Unhealthful News 188 - Follow-up on cigarette graphic warnings injunction: NYT chooses to burn its credibility

I suppose it was just too good to be true.  In my previous post, I praised New York Times reporter Duff Wilson, and by implication NYT reporting, for writing a story that was objective and factual about a politicized health issue.  Specifically, he wrote about a US court injunction blocking FDA from requiring large gruesome graphic labels on cigarette packages, and did so without spinning it as a defeat for what any right-thinking person should want (he accurately presented the labels as representing the goal of a particular government faction, nothing more or less).  He emphasized a basis for the decision, that these labels, while often inaccurately described as "warning labels", had no factual content and did not communicate information.  Thus, they re not warnings), but are merely intended to emotionally manipulate.

Alas, the NYT could not allow objective and accurate reporting to stand when the topic involved one of its pet causes.  So yesterday they published an editorial condemning the judge's ruling.  They completely ignored the bases for the decision, that the labels violated free speech rights in pursuit of government advocacy of a particular behavior, did not fulfill some compelling responsibility of the government (like issuing genuine warnings), were not the minimal way to intervene in free speech in support of some competing goal (like genuine warnings would be), and perhaps most importantly that there was really no evidence the labels would change behavior.  The editorial concluded:

The Obama administration should appeal Judge Leon’s preliminary injunction, which would put off the labeling changes for months, if not years. A delay on the labels would lead to more needless deaths.

So a bunch of write-about-anything, mostly political-beat journalists who have risen to the level of editor claim to know that these labels would reduce needless deaths.  Hmmm, now how exactly do they know that?  Are they privy to evidence that was not presented in the court case and that has not appeared in the scientific literature?  Or are they simply relying on the deep insight that comes from having a strong belief in a subject one lacks technical expertise in?

The editors also asserted:

If his ruling stands, the government will not be able to warn against the hazards of smoking in a way that’s actually noticed.

Really.  So by being limited to forcing cigarette manufacturers to print only those warnings that are, well, warnings, instead of adding images worthy of a slasher movie, the government has lost its ability to advertise, publish, pontificate, dictate "news" stories to the NYT and other papers that will be dutifully transcribed, and otherwise condemn smoking and smokers.  Gee, this will probably mean that a whole generation will grow up without ever learning that smoking is bad for you.  It is a good thing we have the NYT editors to warn us against this dire future.

It is no wonder that health reporting is usually so bad:  The editors apparently demand that it be bad.

Unhealthful News 187 - Cigarette graphic "warnings" injunction, great characterization of the content, but too bad about the understanding of statistics

Yesterday, a US federal judge ruled that the gruesome graphic labels that the FDA wants to mandate that cigarette manufacturers print on their packages would likely not stand up to first amendment scrutiny, and issued an injunction agains the regulation until the full constitutional issue could be litigated.  (The text of the ruling was online this morning but the link does not seem to work anymore; you can link to my saved copy.)  Since the litigation will likely take years, the judge ruled, requiring the labels appear until they are ruled unconstitutional, if that was indeed the ruling, would do irreparable harm.  Thus the decision must be to not impose the regulation until and unless it was ruled constitutional.

The judge in question, Richard J. Leon of the United States District Court in Washington, was the same one who saved e-cigarettes from being banned in the US, making him the man of the year for tobacco harm reduction and maybe public health in general, though that honor, for which he will never get his deserved nomination for a Nobel Prize, was presumably not his motivation.  He showed similar wisdom in the new ruling and that was very well reported by Duff Wilson in the New York Times.

The core basis of Leon's ruling was that the labels were not simply the communication of facts.  That is, as I have pointed out a few times, they were not warning labels.  Anyone who refers to them as such apparently has no idea what the word "warning" means.  I have characterized them as "emotional violence", intentionally infliction of distress, and pointed out that they are designed to manipulate behavior in a particular direction rather than give people the facts they need to make an informed autonomous decision.  Leon did not use the word "violence", but clearly argued that the graphics do not warn, but merely manipulate emotion; indeed, he emphasizes the legal evidence presented to him that this is what they were designed to do.  He wrote, "they appear to be more about shocking and repelling than warning".

It is gratifying to see this recognition of the true nature of these graphics.  I wonder if we will see anyone stop incorrectly referring to them as "warnings" as a result.  Not the anti-smoking zealots, of course, or the World Health Organization, but maybe some of the people who try to write about this topic honestly will get the memo.

Another key basis for the ruling, a somewhat more technical, and thus not the focus of popular press stories (at least not when written by health reporters -- there might be some more thorough reports by law reporters out there), is that the large warnings, covering most of both sides of the package, clearly do not meet the standard for being the minimal infringement on free speech necessary to achieve the government's purpose.  That part of the ruing actually included the seemingly sarcastic parenthetical, "...purpose (whatever it might be)".  Indeed, the judge wrote about a previous ruling he was drawing upon and clearly agreed with:

the dimensions alone strongly suggest that the Rule was designed to achieve the very objective articulated by the Secretary ofHealth and Human Services: to "rebrand[] our cigarette packs," treating (as the FDA Commissioner announced last year) "every single pack of cigarettes in our country" as a "mini-billboard.,,26 Mot. for PI at 6 (citing a June 2001 press briefing with Sec. Sebelius, and an FDA Tobacco Strategy Announcement). A "mini-billboard," indeed, for its obvious anti-smoking agenda!

And, yes, that exclamation point was in the original -- apparently you are allowed two of those in a 29 page judicial opinion.  Interestingly, Wilson's NYT article quoted the last few words and the punctuation, out of context, as well as another passage that characterizes the government agenda, without suggesting that the the agenda must be legitimate or even quoting some QUANGO activist endorsing the agenda.  It was remarkable restraint and professionalism for a health reporter, since declaring fealty to the government agencies that feed them most of their stories is the standard practice.  Wilson did include the mandatory QUANGO quote from Matt Myers, but chose a technical observation about how there will be an appeal of this ruling, rather than printing the self-appointed holy-man ranting that Myers no doubt also offered.

All in all, that was some pretty healthy health news.  But I have to take a few points off for something -- being a professor is a profession and state of mind, after all, not a matter of who writes you a paycheck, and I should stay true to the titular theme of this series.  Both Leon and Wilson did blunder when they wrote about the FDA's research on the effects of the graphic labels not addressing whether they would have a "statistically significant" effect on consumers' awareness of the risks of smoking, and specifically the research not being designed to be able to answer that.  Health reporters have no more business writing about statistical significance than they do writing about legal nuances of the First Amendment of the Constitution, and this is a great example of why.  (Judges might also want to shy away from using jargon from highly technical topics except by quoting its use by experts.)  Since it was obviously impossible to study the effect of the labeling before it happened, it is not clear how a study could be designed to test the effect, let alone designed to achieve statistical significance in so doing.

As a more general point about their error, statistical significance is a property of a dataset -- or more precisely of a dataset and a particular hypothesis that is being tested -- not a property of the world.  It has to do with the chance of seeing a pattern in the data due to chance alone, and relates to how sure we should be about the results.  What people care about, and what the law should care about, is whether there is a substantial effect.  "Substantial" and "significant" (without the "statistical") are rough synonyms in natural language.  They have to be defined by context and basically mean "it matters".  By contrast statistical significance is precisely defined (although only a small minority of those who use the term could actually give you the correct definition) and does not necessarily matter.

So, credit for recognizing that the proposed graphic labels are not warnings, and that they are emotional manipulation, and for genuine objective reporting that (contrary to the usual tone of the NYT) made clear that a particular goal of some people in government is just some people's preference, rather than some God-Given Correct Way.  But points off for still, in spite of all that, not understanding some fundamental points about how science works.

[Update:  The NYT undermines its good reporting with clueless editorializing -- has both irony and humor.]

Unhealthful News 186 - Cancer: screening is generally a bad idea; what about vaccines? (Part 2)

In my previous post, I noted that there were several arguments made against recommending (or, for that matter, mandating) HPV vaccines for pre-sexual children, to protect against that sexually-transmitted cancer-causing virus.  The one of these arguments that is actually fairly compelling, that the cancers the vaccine could prevent could very plausibly become easy to treat during the >30 years before they will occur.  But since almost no one seems to understand this, it is interesting to try to understand what took so long to make the recommendation that boys get the vaccine.  I noted that there was one misguided and two deplorable apparent reasons for that.

The first of those is that, in contrast with cancer screening, vaccines provide no stories of miraculous life-saving success.  Well, actually that is true for screening too.  There are lots of stories, but they are usually wrong.  That is, most people who are uncritically quoted in the unhealthful news reports, in any story about a particular screening regimen looking like bad idea, sobbing "I am living proof that this screening saves lives!" are probably wrong.  Most such screening tests detect many cancers that never would have led to morbidity, and others that would have been detected and successfully treated without that mass screening.  In other words many, often most, of the "saved lives" were not.  But they make a good story for the statistically illiterate decision makers, and are enough to let them be talked into funding/recommending/mandating the screens by those who stand to profit from that policy.

There is no such misguided constituency for vaccines.  No one realizes that their life was saved from a disease they never got.  Indeed, this is undoubtedly why vaccines for infectious agents that have other nasty effects are often forgone or even hated.  News stories never report on the person who insists "I would have died without that vaccine", even though the speaker would be no less certain than someone making that claim about the screening test.  Perhaps some of us should form a patient advocacy group as survivors of polio who never got it thanks to vaccination.

I will leave that for another day and move on to the even worse reasons the HPV vaccine has been only reluctantly embraced.  HPV can be transmitted sexually even if condoms are always used.  This means that, absent the vaccine, HPV risk is a reason to avoid even "safe sex".  While it is obviously not a reason that affects behavior to any measurable degree, as evidenced by extensive journal peer reviewed...  just kidding, I meant: as is obvious to anyone who is not totally clueless.  But that incentive, or more accurately, that opportunity for anti-sex propaganda, is something that those who want to scare people into not having sex do not want to lose.  Some have said as much, while others clearly share that sentiment but pretend to have other motives.  They are quite willing to hurt people to save their souls or whatever.

If this sounds familiar to many of my readers, it should.  It is basically equivalent to the anti-harm-reduction tactics used by anti-tobacco extremists.  In both cases, prohibitionists actively oppose making the activity in question less risky because they want to maximize the incentives for abstinence.  But that is not the only deplorable connection.  It seems fairly likely that one reason the recommendation boys get the vaccine (rather than just girls, to protect against cervical cancer) was so long in coming was because of the refusal to recognize that for at least a decade the evidence suggested an increasing number of oral cavity cancers are caused by HPV.  Indeed, many of the articles about the new recommendation have emphasized protection against anal cancer and anal warts, probably trying to create controversy by making it "a gay thing" and mention only throat (esophageal) cancer in addition to that, mysteriously not mentioning the dreaded oral cancer.

Why the failure?  It is difficult to say for sure, but it is also difficult to not attribute it to the anti-tobacco extremists claiming oral cancer as "their" disease.  They use it for misleading people into believing that smokeless tobacco poses substantial disease risk and is responsible for a growing epidemic of oral cancer.  There has never been the slightest doubt that smoking causes far more oral cancer than smokeless tobacco, but the notion that "switching to smokeless just trades lung cancer for oral cancer" has been a major contributor to delaying tobacco harm reduction for years.  The extremists -- pretending to be concerned with public health, just like the "Christians" who would intentionally avoid curing the lepers if leprosy was sexually transmitted -- got this myth so well established that it is quite difficult to communicate the causes of oral cancer.

There is a certain elegant symmetry in it.  The lefty pseudo-"public health" sermonizers who despise the right-wing anti-harm-reduction sermonizers who oppose safer sex or needle exchanges have ended up working in tandem with them on this issue.  Maybe it will be a wake up call to them and they will... just kidding again.

Finally, the symmetry extends to that one good argument I noted.  Just as an 12-year-old who gets the HPV vaccine will be protected from a disease that will not occur for decades, and might be quite easy to deal with by then, a young person who picks up a dangerous behavior now might be saved by advancing medical science.  The analogy is far from perfect:  HPV-caused cancer is a specific cancer that, if fully cured or prevented at the last minute, might do no damage.  By contrast, smoking damages lung and other tissue over time, and contributes to many diseases in complex ways, and that damage seems a lot less likely to be able to avoid or reverse.  On the other hand, if smokeless tobacco causes an oral cancer in someone who starts using it now, the evidence suggests that this will happen many decades from now, making it again like the HPV case.

So, screening is loved based on non-evidence; vaccine-based harm reduction for sexually transmitted disease is embraced by the health authorities but suspect among some political factions; uptake of that vaccine is limited, though no one seems to be motivated by the one good reason for not bothering with the vaccine; the oral cancer vaccine is not being recognized as such; and none of this will affect the behavior of the various extremists and activists.  The funny part, and the only reason this is not completely mortifying, is that a remarkable amount of the stupidity/naivety/dishonesty is canceling out other bits of the stupidity/naivety/dishonesty.

Unhealthful News 185 - Cancer: screening is generally a bad idea; what about vaccines? (Part 1)

Yesterday, and several earlier times in this series, I wrote about how cancer screening tests have been shown to be inefficient (i.e., too expensive for the benefit they provide, as compared to competing uses of resources) or even more harmful than beneficial (a bad idea, even if they are free).  This is not quite always true, and refers only to mass screening of people with no symptoms and who are otherwise not believed to be at higher than average risk.  Cervical cancer screening is cheap and easy, and detects incipient cases of the disease rather than just finding growing cancers a bit earlier.  Most other cancer screening looks pretty bad on close inspection.  Maybe three or four mammograms between ages 50 and 65 are worthwhile.  

Finding a vaccine for cancer and administering it to everyone is possibly a different story.  About two weeks ago, the US government finally got around to recommending the HPV vaccine for boys.  It has been recommended for girls for a while because it was recognized that just about all cases of cervical cancer are caused by sexually-transmitted papilloma virus strains that the vaccine can prevent.  But beyond cervical cancer, it has long been recognized that HPV apparently causes many other cases of epithelial cancers (i.e., the body parts that are in contact with the outside world), specifically the bits that experience sexual contact, the oral cavity, esophagus, and anus.  So some commentators have asked "what took so long?"

That is indeed a good question.  There are a few answers that occur to me, one is fairly compelling, one is very compelling but no one seems aware of it, one is interesting and understandable but misguided, and two are deplorable.

The fairly compelling answer is that the vaccine is extremely expensive and prevention is probably of fairly little value.  Recall that screening is usually too expensive to justify; this is primarily because it is expensive to screen millions of healthy people who do not have the disease to find the few who have it.  It is also expensive to vaccinate millions of people who will never get cancer from HPV to save only the few who will.  However, when the prevention happens, in contrast with screening, there are no costs of false positives, no further treatment cost, and no damage from the disease at all.

Can screening ever look so good?  Not for cancer, given the high cost of treatment, but for other diseases it can.  The example that comes to mind is syphilis screening, which is mandatory in the US for pregnant women, in the sense that if they do not get the recommended screen during pregnancy then it is a mandatory part of medically supervised delivery.  Of all the screening tests I have ever done cost-benefit calculations for, this is the only one that came out clearly positive (note: I have never run the numbers for cervical cancer screening).  The reason that it is done is that syphilis transmitted to a newborn, if undetected, can be devastating.  But the benefits go beyond that, since non-symptomatic syphilis is unlikely to be detected, and so when it is detected the mother and her partner(s) are treated, eliminating those cases and all those they would cause in the future.  This almost certainly lowers the equilibrium prevalence of the disease in the population by a lot, perhaps by half or more.  But what makes this such a bargain is the treatment is relatively harmless and dirt cheap (a simple antibiotic shot), the cost of a false positive is equally low, and the cure is near certain and complete.  Unlike most cancer screening, this is a very good deal.

But a problem with the current HPV vaccine is that even though it does not entail further treatment like screening does, it is still very expensive.  It costs payers hundreds of dollars, in contrast with the few tens of dollars it costs to get a flu shot.  Merck charges $300 for the drug itself, and there is clinical time for three injections, plus whatever markup the medics add.  (Incidentally, the official claim is that the cost for vaccinating just the boys as they come of age would cost $140 million/year.  But since there are 2 million boys in every US one-year age cohort, this seems low by about a factor of five.  The estimate must be based on the assumption that few will follow the recommendations.  Or someone is lying. I will leave the investigation of that discrepancy to the news reporters.  Hahaha - just kidding.  They will never pursue it, and probably not even think enough to realize there is a problem.  Unfortunately, I will probably not investigate it either -- if anyone does, please clue me in with a comment.)

What almost none of the unhealthful news reports bother to mention, however, is that this huge total expenditure is mostly not real resource costs.  That is, it does not consume actual stuff or labor, but rather just moves money from one entity to another.  Because drugs always are priced at greater than their manufacturing cost, because the price has to amortize development costs, and especially because the US does not negotiate drug prices like every other country and thus allows monopoly profit on top of that, that $300 price undoubtedly includes a huge profit for Merck.  While we might not want to give money to Merck, at least it is not an actual cost.  The same is true for the physicians' fees for administering the injections.

A more important and compelling argument against the vaccine than purchase price is that this vaccine prevents cancers that, except in very rare cases, will occur at least 35 years in the future, and often decades more than that.  Those of you who know a bit about these analyses might point out that any proper cost-benefit analysis discounts future costs and benefits, so this delay is already accounted for.  But that process of discounting assumes that events will occur, and merely adjusts for the fact that we weight the present more heavily than the future in our decision making.  But is that a reasonable view?  Should we assume that oral cancer or cervical cancer (this observation applies to the girls too) will still be expensive or deadly disease that far in the future?  I certainly hope not.  Four decades might not get us to the point that a single injection will reliably eliminate the disease, as with syphilis now, but it seems safe to assume that there will be much better treatment than currently exists.  

Remarkably, I have never seen anyone else make this observation, other than me and some of my students (who were thinking this through as an assigned exercise with the advantage, compared to most people writing about such topics, of having been taught what to think about).  The strongest argument against this vaccine is that we should really hope it will not matter much for those who are currently young enough to benefit from it.  Consider how much the value of an expensive syphilis vaccine for children would have been overestimated in, say, 1930 (it became easy and pretty cheap to treat once penicillin was developed and proven in the early 1940s).

That is the good argument against the vaccine.  But I have never seen anyone make it, so it cannot explain what took so long.  In Part 2, I will look at the not-so-good reasons that probably do explain it.

Unhealthful News 184 - The right answer about prostate screening, but as for the reasons...

Sign in London Underground car

During my hiatus, Chris Snowdon asked for my opinion about a commentary by the Australian public health community's own keystone kop, Simon Chapman, which argued that screening for prostate cancer is a bad idea.  Even though it was Chapman, the basic advice turns out to be good -- after all, a stopped clock is still right twice a day.  (For more on why I am being so negative about someone who got the right answer, the above links go to Snowdon's and my analyses of some of Chapman's other "contributions".)

To see why it was good advice, we need only look to something that came out a few weeks ago, in which a US panel led by Virginia Moyer, a former colleague of mine whose clock keeps perfect time, recommended that the common practice of PSA screening tests cease.  Here is Moyer briefly arguing for it in her own words.  An even briefer version:  Because most cases of prostate cancer are unlikely to grow fast enough to cause a problem before someone dies of something else, and perhaps the cases that will be deadly are unlikely to be prevented by screening, and because the treatment that results from screening kills some patients, it turns out there is no detectable mortality benefit from screening.  Meanwhile, the treatment often causes nasty side effects (impotence, incontinence).

Following this recommendation, there was the usual outcry against any recommended curtailment in wasteful medical spending.  It was the same as the reaction to scale back mammography that I wrote about before.  As always, the most remarkable comments are the breathless testimony from cancer survivors who are absolutely sure that the screening saved their lives, when obviously they have no way of knowing that, and indeed the statistics show that it is almost certainly not true. 

But what is a bit different about this case is how clear it is that the treatment, not the screening, is the real problem.  In contrast with a mammogram, a PSA test is not harmful and will never cause the disease it is meant to prevent.  Of course, the problem is that once someone screens positive, he is very likely to demand treatment even though it turns out to not be a good idea.  Still, the last chance to avoid the cost comes not from the decision to test, but from the decision that is made afterward.  Of course, if we agree that treatment after screening positive is a bad idea -- and obviously the same is true for treatment after screening negative -- then there is no value in screening other than to just know.

Those observations, in turn, led to a spate of occasionally interesting articles about whether it is better or worse to know.  A lot of the discussion came back to how difficult it is to just live with the knowledge rather than acting on it, even if acting is not beneficial.  The decision to treat is what actually causes the harm, but since that apparently bad decision is inevitable following a positive screen, the only way to avoid the mistake is to avoid the triggering knowledge.  It makes sense, but it is still interesting that people cannot resist acting.  

The above-pictured "In an emergency" sign from the London Underground is really amusing, with its major point of advice being "Do not take any risks".  What it is really trying to say (I assume) is to favor inaction over action until advised to do otherwise, which is quite often terrible advice and obviously entails some risks.  But someone apparently calculated that it is good advice, on average, for subway emergencies, and it turns out to be the right advice for treating the average detected case of prostate cancer.

To finish the story I began with, Snowdon's incredulity about Chapman's advice did not focus on the overall merits of the advice itself, but rather on how Chapman tried to support the claim:  He made a wandering argument in which he suggested that screening and treatment does have mortality benefits, but since prostate cancer kills mostly old men who have already had a good life and might die of something else shortly anyway, they should favor quality of life (avoiding the damage from the treatment) over longevity.  Snowdon pointed out the strangeness of this argument that men should not have the option of going for the supposed longevity gains, coming as it was from someone who is best known demanding the opposite choice.  Or, more pointedly, known for his willingness to pervert science, abuse the social contract, and do pretty much anything he can think of that might keep people from choosing to smoke or even choose to be in smoky environments.  He will ferociously fight to deprive people of the choice to take that risk, however much their lives might benefit from the choice, but then demands that men accept another mortality risk because he judges that their lives will benefit.  

The funny thing is that, according to Moyer et al., it is reasonable to deprive men of this choice because there seems to be no upside.  But Chapman claimed there is an upside to treatment, but that the choice to pursue it should still be denied.  A slow clock, rather than being right twice per day, might only manage once per month.

Unhealthful News 183 - Greetings from the middle (of history (of science))

Hello, everyone.  I promised you 180-some more UNs, and while I will obviously not fulfill my hope of doing that within 2011, I will continue toward the goal.  I will get started by including some general observations I have been wanting to write about, interspersed with the unpacking of recent news that is the central theme of the series.

This is hardly the first time this has occurred to me, but I have been struck over the last few months by how most people, notably including those with the experience to know better, seem to think that they are living at the end of the history of science.  People who understand how Newtonian physics, which seemed doubtless correct for centuries, had errors that were fixed by Einstein's relativity are remarkably unwilling to allow for any possibility that the current theory has some flaws, such as the possibility that a particle could move faster than the speed of light.  Of course, I am more interested in science and technology that is more immediately practical.

One place I observed the phenomenon of people not realizing they are living in the middle of history is with regard to "renewable energy".  I have been in some interesting battles about industrial wind turbines, which regular readers will know cause serious health problems for nearby residents and are so incredibly inefficient that they arguably offer no benefits at all.  I will come back to some details of what I have written and been dealing with there, but will start with my observation about what seems to be motivating some IWT proponents.

I have noticed that the well-meaning people (i.e., I am not talking about the industry and their hirelings) who argue in favor of building IWTs persistently fail to understand the technological reality that we are not at the end of history.  Their view seems to be the following:  The natural dynamics of the planet -- wind, waves, sunlight, temperature gradients -- contain plenty of energy that should be harvestable for electricity.  Moreover, we will probably (they would say inevitably) come to need those sources of electricity.  Typically they also point out that burning coal has huge environmental costs.  But the erroneous syllogism is the next bit, where they argue, "...and therefore, since the only technology that harnesses those dynamics that can be built on a large scale right now is IWTs, must be a good idea to build them."

I assume the gaping failure of logic here is obvious.  Decades or centuries before the Apollo Program, it was clear that it was possible to use develop technologies to send people to the Moon.  But that did not mean that getting aboard the best rocket that could be slapped together in 1950, wearing a diving suit, would have been a bright idea.  Yet when I or others point out that (a) IWTs are so incredibly inefficient that their net contribution to energy generation is quite possibly negative (i.e., installing new IWTs actually increases fossil fuel consumption), and even if the contribution is actually positive, the cost of that tiny benefit is enormous, (b) IWTs cannot affect baseline generation like coal or nuclear (which basically need to operate at full capacity all the time) because the wind is intermittent, and so only affect how much gas is burned (gas plants can be turned on and off -- it is not terribly efficient, but much better than it would be with coal), and (c) IWTs do terrible damage to local residents' health and the environment, I frequently get the response "but we cannot burn fossil fuel forever! and coal is evil!!!"  

Translating that charitably (i.e., resisting the very strong urge to scream "what part of 'approximately zero net energy contribution' and 'does not replace coal burning' is too complicated for you to understand?"), I can only conclude they they are falling victim to the end of history fallacy.  Since at some point in time we might be forced to get our electricity from "renewable" sources, and improving technology will make such generation efficient long before then, then doing so must already be a good idea.  After all, how could something possibly be a good idea in the future, but not a good idea now?  Aren't we the pinnacle of human civilization?  The evidence-based truth, that renewable technology (except for damning rivers for hydroelectric) is currently not ready, and thus immediate installation is a terrible idea, simply cannot penetrate that prejudice.  

A second version of the "end of the history of science" fallacy can be found in blind faith in the perfection of current health science methods and knowledge.  This problem explains much of the unhealthful research and news reporting I have covered in this series.  If only there were a bit of epistemic modesty and use of the phrase "the best we can do now" or "given the limits of current knowledge", there would not be nearly so many errors in health reporting.

Allopaths (mainstream Western medics) are particularly guilty of lacking modesty, epistemic and otherwise, and the tendency to mistake medics for scientific thinkers is particularly damaging to health science.  For a profession that so recently engaged in such practices as therapeutic blood letting (which physicians of the time were absolutely positive it was a good treatment) and was quite likely to kill those it was supposedly helping because physicians refused to wash their hands (and were absolutely positive that doing so would be madness), medics and their enablers show a remarkable arrogance in insisting that everything they currently believe is absolutely, positively, beyond-and-doubt correct.  What is worst, >99% of the time, the belief is not based on any scientific knowledge, but rather was just something someone was told.  And so they believe it with certainty.  If this this sounds a bit like another major social institution that has a history of being absolutely sure of highly destructive baseless claims on the basis of faith alone, you are not wrong.

I am not just talking about how many clinicians are absolutely positive that smokeless tobacco is highly risky and other areas where they are the victim of directed propaganda (though there is no excuse for that either).  I am talking about things like medics whose education includes the equivalent of one semester of epidemiology, one semester of immunology, and zero semesters of nutrition making absolute pronouncements about the evidence about food allergies, and often being very wrong.  I'll probably come back to that theme.

At a less dramatic level, the "end of history" mentality contributes to the reporting of every trivial, highly-technical research finding as if it were of huge practical importance by itself.  This is not the only reason for that, of course, and I have written extensively about some of the others.  But if consumers of the report (editors, reporters, consumers, policy makers) understood that they are sitting in the middle of history, they would not be so vulnerable to venal self-aggrandizement by researchers.

The metaphor that occurred to me is that this is like polishing roadway gravel.  Gravel is very useful in a workaday way, making it possible to move forward over otherwise muddy terrain.  But picking up a piece of gravel, polishing it to a high sheen, and putting it in a glass case does not make it a gemstone.  In a way, the unpolished bit of the road forward was arguably contributing more to the world than a gemstone.  But even if you do not take such an extreme workmanlike view of value, it should be clear that trying to polish and display a large portion of the gravel is a disservice to the value of both gemstones and gravel.

So, we should not polish the gravel, but should wait until we reach the end of the road where there will be a gemstone.  (Hint for interpreting that: like a rainbow, the road has no end, so that statement, like the one about the pot of gold, is true because it is vacuously satisfied.)