Unhealthful News 180 - Study of "No Smoking Day" may be a new low in bad epidemiology and health economics

Ok, that is probably not true, given how much other bad anti-tobacco "research" there is out there.  But this is a really good one.  It was so good I stole it from today's weekly readings in THR post so that I could expand on it here.

It was published in the quasi-journal Tobacco Control, of course.  I will provide the entire abstract here so you do not have to bother with the link.

How cost-effective is ‘No Smoking Day’?

D Kotz, J A Stapleton, L Owen, R West 

Participants: A total of 1309 adults who had smoked in the past year who responded to the surveys in the month following NSD (April 2007–2009) and a comparison group of 2672 adults who smoked in the past year who responded to the survey in the two adjacent months (March and May 2007–2009). 

Main outcome measures: The number of additional smokers who quit permanently in response to NSD was estimated from the survey results. The incremental cost-effectiveness ratio (ICER) was calculated by combining this estimate with established estimates of life years gained and the known costs of NSD. 

Results: The rate of quit attempts was 2.8 percentage points higher in the months following NSD (120/1309) compared with the adjacent months (170/2672; 95% CI 0.99% to 4.62%), leading to an estimated additional 0.07% of the 8.5 million smokers in England quitting permanently in response to NSD. The cost of NSD per smoker was £0.088. The discounted life years gained per smoker in the modal age group 35–44 years was 0.00107, resulting in an ICER of £82.24 (95% CI 49.7 to 231.6). ICER estimates for other age groups were similar. 

Conclusions: NSD emerges as an extremely cost-effective public health intervention.

Taking this from the top, we have to start by observing that they are claiming that about 10% of all smokers attempted to quit each month.  This indicates either some very faulty data or such an expansive definition of "quit attempt" (like "I woke up and decided I was going to quit, but I started again during my morning break") that it is meaningless.

Moving on, they assume that the entire observed difference is neither random nor the result of Easter/Lent, Passover, spring holidays, the misery of March, or anything else that might make April different from nearby months.  It seems like these might make a wee bit more difference than an arbitrary declared day that most people pay no attention to.  It is kind of interesting that they did not give us a month-by month breakdown, which we might have expected if the month with "No Smoking Day" were the global or even local minimum.

Beyond that, their interpretation of what NSD entails is quite silly.  They treat it as if it is some kind of medical intervention that is independent of other causes of quitting, but really it is (at most, even if it really works) a focusing event, causing people who are considering quitting soon to say "ok, I am going to do it that day".  So the effect, if there really is one, is to move quit attempts from May and June back to April.  Perhaps not of trivial import (remember that smoking for just a couple more months is as unhealthy as using smokeless tobacco for your entire life), but not the same as causing quitting that would not have otherwise happened.

And this says nothing of their their magical ability to detect permanent cessation from a cross-sectional survey.  Even if they have some standard prediction about permanence, quitting for different motivations, like say a focusing event, will inevitably have different permanency rates.

As if this were not bad enough, where they really jump the shark is the cost-effectiveness analysis.  Reporting the cost of a declared focusing day per smoker is LOL funny.  I wonder how much National Kale Week cost per meal at which kale was served; I will bet it was quite a bargain too. 

That "ICER" is the "incremental cost effectiveness ratio", which measures the cost-effectiveness of an intervention as compared to an alternative it could replace that is more cost effective (so a better deal) but less effective in total.  In other words it accomplishes an analysis like: "if we are going to take driver protections one step beyond seatbelts and add airbags, which are much more expensive but will save a few more people, we should make sure to not give airbags credit for the people that seatbelts alone would have saved anyway by comparing them to no restraints at all."  What makes this funny is that they pretend to be using a somewhat complicated good measure, one that is often not done creating erroneous results (e.g., airbags are measured against no restraints at all; pharmaceuticals are measured against placebos rather than existing effective treatments), to look at something that they got totally wrong.  In this case, the alternative that is crowded out by NSD (like airbags+seatbelts crowds out seatbelts-alone) is they same people quitting a bit later, which they completely ignore.  So what they claim is ICER is really just the most basic, and misleading, cost-effectiveness calculation that pretends nothing would motivate quitting were it not for NSD. 

So they go on to calculate the cost-effectiveness, not as £80/life-year-saved, but £82.24.  Even if their estimate of the effect of the intervention were as precise as the 2.8 percentage points they report (which is not even possible given that they are basing this on only a few hundred events) they could not get precision even to the first significant figure, let alone the fourth:  The guess – "estimate" would give it too much credit – about how many life years will be saved by someone quitting (again, even pretending that NSD caused it, and their estimate of permanent quits is based on anything, and that it would not have happened a month later even without NSD) requires assumptions about the next half-century of medical technology and other health effects.  It cannot be reasonably guessed-at within a factor of two, let alone to one part in 10,000 as they imply.  Someone develops a cure for cancer or emphysema, and the benefits plunge; some other breakthrough extends life by 100 years so long as you do not get cancer or emphysema and the benefits shoot up.

About the only thing that can really be said about their conclusion is that there is no doubt that NSD is more cost-effective than funding people who write articles for Tobacco Control.

Yes, this is what passes for science in anti-tobacco.  Is it any wonder that they can reconcile "hundreds of millions of people are not quitting" with "a tobacco free world by 2030" or whatever?  It would be humorous if it was not so incredibly damaging.  Oh let's be honest:  In spite of being incredibly damaging, it is frackin' hilarious.