Unhealthful News 4 – a pet scan of what constitutes useful negative evidence

Good for your health, bad for bunnies

This story is just a bonanza of unhealthful news points, and since the New York Times highlighted it on their op-ed page, the most read part of that paper, rather than putting it in the Tuesday health section it seems particularly worthy of attention.  I am breaking my analysis in to two parts for this one, saving the best point for tomorrow.

The column, by West Carolina University psychology professor Hal Herzog, has as its main thesis questioning the claims of pets providing health benefits of various sorts.  The point that there are no miracle cures from having pets may well be correct, but his conclusions go too far and introduce various bits of bad reasoning.

The author declares, presumably in an attempt to bolster his own field, that assessing claims about pets providing health benefits falls to “anthrozoology, the new science of human-animal relationships”.  But these questions are straight epidemiology of the most normal kind – studies of how a particular exposure affects health outcomes – and can only be studied with epidemiologic methods.  This is good news for pursuing the topic because, though much of the epidemiology that is done is junk science, population psychology research is even worse.  I will address this point tomorrow.

Today I will focus on how this article displays what might be called the Mythbusters fallacy.  For those not familiar with it, Mythbusters is one of the more entertaining and somewhat educational shows on television.  In each episode they do some ad hoc engineering projects, often but not exclusively things that go crash or bang, to test urban legends, historical claims, movie conceits, or the like.  Examples include “will a bullet that is fired straight up fall with enough force to penetrate deep into the ground (or someone’s head)?”, “can a blind (blindfolded) person drive a car based on instructions given to them by a passenger?”, “can soldiers marching across a bridge in step set up a resonance that shakes the bridge to collapse?”, and “can pouring water onto a very hot grease fire create a ten foot fireball?”

The epistemic problem with their experiments is that if they succeed in causing the result in question (as they did with the grease fire), they have indeed proven that it is possible.  There is no doubting an existence proof like that.  But if they fail to replicate the myth in question, it might be that they just did it wrong.  Yet they typically declare “this myth is busted” if they cannot make it happen.  That is a reasonable conclusion for such things as the gunshot into the air, where there is pretty much only one way to do it.  When the bullets made relatively modest holes in the ground they could reasonably conclude that they had shown that some of the more dramatic claims about falling bullets were wrong.  But in complicated cases (like when they created a machine that simulated marching soldiers and put it on a little bridge that they built, resulting in nothing interesting happening) there is no way they can conclude that it is not possible, but merely that it could not be done in the particular way they tried.

It is very easy to fail to find something, even if it exists.  Moreover, if it exists somewhere but not everywhere, it is very easy to do research that "shows" that it does not exist.  I have traveled a lot, but according to my research (experience), there are no ancient pyramids.  Of course, I have never been to Egypt or the Yucatan, so my evidence might be a bit faulty.  But if I drew that conclusion it would be no different from saying that a particular possibility has been ruled out based on failing to find it via one particular study.

Herzog wrote, “I have a stack of articles in my office supporting the hypothesis that pets are healthy for us.  Unfortunately, however, I also have another stack of articles, almost as high, showing that pets have either no long-term effects or have even adverse effects on physical and mental health.”  That might actually be a useful observation if we were talking about a single simple exposure and outcome, such as “does taking an aspirin every day reduce the risk of heart attack?”  But “physical and mental health” is a rather heterogeneous category, to put it mildly.  I would bet that having a pet is useful for curing depression stemming from ennui and loneliness, but is pretty useless against hepatitis or for setting bones.  Also, a generally healthy older person with no close family who gets a dog offers a particularly high potential for benefit, while a middle-aged woman with 17 cats may have some issues (which were probably not actually caused by the cats). 

Herzog goes on to note, as evidence that pets are maybe not so good for you, that research in Finland (which he attributes to "epidemiologists" though the affiliations of the authors and where they published does not support this characterization) found that people with pets had higher rates of sciatica, kidney disease, arthritis, migraines, panic attacks, high blood pressure and depression.  The depression might be related, and suggests that perhaps the conventional wisdom is wrong, though it could also mean that depressed people seek animals.  Most of the others seem very unlikely to be causally related, one way or the other.  Pets are pinching people’s sciatic nerves?  Presumably a lot of the “evidence” in the stack of negative articles was of the looking-for-pyramids-in-Belgium variety.  But this one seems to represent a failure to understand confounding.  (I would make some obnoxious comment about anthrozoologists perhaps needing epidemiologists after all, except so-called epidemiologists are typically no better at recognizing obvious confounding.  I wonder what it is like to be be able to make self-assured arrogant turf-war comments about one's field.)

There is no doubt that Herzog’s strawman conclusion, “pet lovers should probably keep taking their Lipitor and Prozac” is true.  But not because he has done anything to bust the claims about the benefits of pets.  More tomorrow.