The Perils of Amateur Epidemiology

The pitfalls of Over-extropationg (via XKCD)

I have just finished reading the excellent The Panic Virus by Seth Mnookin. It details the rise of the anti-vaccine movement, how their ideas have been scientifically discredited and further how a maelstrom of media, social and cultural pressures have meant that those ideas (despite the ‘debunking) have as much currency as they ever had in the public imagination. I have already written an article  on the Anti-Vaccine movement and if you found that interesting then you definitely ought to read this book. It is one of the most comprehensive journalistic works, on this issue. out there. (The Panic Virus has been added to the “Reading List” Page, where I have added other books of relevance to the content of this blog, if you want to check them out)

So with book recommendation out of the way, I’ll move onto a brief and particularly fascinating passage in the book. It is a simple yet highly illustrative case of how ‘common sense’ approaches to epidemiological matters can be misleading. Furthermore how that misunderstanding is then latched upon by the press in a bid to satisfy preexisting narratives which do not cater for most basic level of complexities. The rush to link ’cause’ to ‘effect’ is a very tempting thing to do and a large part of scientific endeavor is to account for any (confirmation/publication) biases that may influence any such accounting. We like to connect the dots, see patterns and while its a rule of thumb that serves us well on a day to day basis, it simply cannot establish cause and effect in terms risk exposures and disease outcomes. This is why I found this particular passage in book fascinating
In March 1992, Lorraine Pace was diagnosed  with breast Cancer. This came as a shock not, Pace said, because a cancer diagnosis is always upsetting, but because she thought her healthy lifestyle should have protected her from the disease. Even more alarming to Pace was her awareness of the rather large number of cancer patients in the area; Eventually she counted a total of twenty people in her neighborhood in West Islip (Long Island, New York) who’d also been diagnosed with cancer in the past several years alone. This might have been an informal sampling but it left pace convinced that an unidentified toxin was stalking her community. (The Panic Virus pg 138-139)
Sampling Error is the first thing we learn about in a statistics class. It is simply an acknowledgement of the fact that a small data set can be misleading indicator of the whole population (The Small Numbers fallacy). Of course Statistics is a field built on approximations of patterns in the whole population without having that information (in full), it is still a well established concept that small sample sizes are often very unreliable. This is especially key, while establishing ‘facts’ regarding the prevalence of disease rates in a community. Were the Cancer rates really higher in Lorraine Pace’s neighborhood (of West Islip)  than they were on a state or national level? We need to establish that beyond a  reasonable doubt (or a small sample size) before we entertain a further proposition of a causal link between the cancer and particular risk factor. Yet this is not what happened as Mnookin explains,
Frustrated with what she saw as lack of official concern, Pace took  matters into her own hands and founded the West Islip Breast Cancer Coalition. Day after day for months, the organization’s members would meet in Pace’s living room to add new data points to a giant, color-coded map; yellow dots for homes with malignant breast tumors, pink for benign tumors, and blue for no tumors at all. After analyzing its data , the group announced  that cancer rates in the area were 20% higher than the state average. The Media and and local politicians alike jumped on story much to the dismay of scientists, who knew that epidemiological studies that start with the desired outcomes in mind are almost by definition of worthless. (The Panic Virus)
So what could have been the mitigating factors which can explain/confound the existence of an apparent higher rate of cancer without having to resort to the ‘unidentified toxin’ hypothesis. (With some prompting from Mnookin) here are a few
1) The West Islip region area of Long Island has a higher proportion of wealthy people than normal, hence more people with better access to healthcare  which means that tumours in other communities may not be as readily noticed as in this community
2) The life-expectancy in West Islip is higher and cancer rates rise with increasing age. Hence that could account for a higher cancer rate
3) (According to the passage) West Islip women tended to defer their childbearing to later in life which is apparently points  to a higher risk of cancer
4) Sampling Error
Before we embark on establishing a causal link (toxins) for the apparently higher cancer rates, one has to account for the above (and undoubtedly other) confounding factors. So what happened
Ultimately, Pace’s crusade spawned a controversy that raged for nearly a decade at a cost to taxpayers of more than $30 million. When in 2002, an exhaustive study found the cancer rates in Long Island were barely distinguishable from those in the rest of the country; the news received a fraction of the attention the initial scare caused (The Panic Virus)
In this case, it was probably sampling error. But even if it wasn’t, there were quite a few confounding variables that had to be accounted for. This also highlights the pitfalls of the media coverage of health/science stories in general. The compelling narrative of a defiant, crusading cancer patient means that the ability to check for accuracy of the claims made, is compromised. This is one of the main reasons why things like this and vaccine scares get so much purchase in the media and popular culture as a whole.
This entry was posted in Cancer, Evidence-Based Medicine, Medicine, Science, Statistics, Vaccines and tagged , , , . Bookmark the permalink.

One Response to The Perils of Amateur Epidemiology

  1. imc says:

    Sampling errors? Check. Selection bias? Check. Wind turbine syndrome, anyone?

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s