Bad Medicine: Is Evidence What We Think It Is?

As Marion Nestle says, on the subject of what nutrition research typically assesses: “nutrients that are out of their food context, foods that are out of their dietary context, and diets that are out of the context of lifestyle.”

Studying nutrients the way we study drugs makes no sense, not only because the form of the nutrient employed in these studies is typically a pharmaceutical form, but because nutrients don’t work in isolation – they work and heal in the context of other nutrients, and of the lifestyle.

So, while we, firmly positioned in the holistic camp, believe that natural medicine heals, many of us still feel compelled to prove it. What are we using as the standard for that proof? What if we are looking behind the curtain to find that Wizard isn’t quite what we imagined?

A provocative and important piece in the Mayo Clinic Proceedings should change the way that medicine is practiced. It probably won’t. Here is the question:

“How many contemporary medical practices are not any better than or are worse than doing nothing or doing something else that is simpler or less expensive? This is an important question, given the negative repercussions for patients and the health care system of continuing to endorse futile, inefficient, expensive, or harmful interventions, tests, or management strategies.”

In this investigation, Prasad et al analyzed 2044 original articles published in the New England Journal of Medicine from 2001-2010 and classified the articles based on whether they tested a new or existing treatment and whether the results challenged or supported its efficacy. They found that 40.2% of the articles argued for “medical reversal” or – stop doing this because it’s actually not evidence-based. A total of 128 medical practices were brought into the harsh light of evidence over those 10 years which means that a major memo should have been disseminated and doctors around the country made aware of the need to change what they were doing – or, at least, make patients aware of the evidence calling the practice into question. To the contrary, Prasad et al discuss:

“Although there is a weak evidence base for some practice, it gains acceptance largely through vocal support from prominent advocates and faith that the mechanism of action is sound. Later, future trials undermine the therapy, but removing the contradicted practice often proves challenging.”

They reference a related review in the British Medical Journal that evaluated 3000 medical practices and found that more than one third are effective or likely to be, 15% are harmful, and 50% are unknown. I would add to this that the assessment of “no net benefits” in many of these short-term trials with limited controls for patient individuality are ill-equipped to estimate important functional risks to physiology that may take months or years to emerge and may never be linked to the original treatment exposure.

Prasad et al lose me however, when they start to apply a framework of using evidence to dismiss previous signals of risk. Establishing efficacy is one thing that these trials appear to be far less well-equipped to handle than we all have been led to believe, by modern medicine. Establishing risk is a far more complex endeavor and should rely, heavily, on the precautionary principal, which demands that, if there is any signal of harm, caution is exercised. Data such as this, and related meta-analyses are not designed to establish safety, and dismissal of previously identified risks is not the same as identifying a lack of efficacy.

We have learned this lesson through the push and pull with industry around “studies” intended to assure the public of safety and to erase any concerns about risk from the board. Only after decades of accumulated damage is the unavoidable population-based data able to overturn the pharmaceutical agenda. It’s happened again and again – cigarettes, DES, thalidomide, COX inhibitors, BPA, the relationship between neurological damage and vaccination.

This is not the first time that objective investigations into the shortcomings of our evidence base have been brought into the spotlight. The Cochrane Database was reviewed in 2004 to demonstrate that 47.8% of assessments concluded that there is not sufficient evidence to endorse the queried practice or intervention. In 2011, this trend had not changed with an insufficient evidence rate of 45%.

Close to my heart was a recent related paper that explored this phenomenon in the world of obstetrics. Practice bulletins are the premise for assessing “gold standard” applications of science in the service of patient care. Certification testing is based upon these bulletins, as are malpractice allegations. In this paper, 84 practice bulletins discussing 717 individual recommendations. Of the obstetrical recommendations in this assessment, only 25% are considered to reflect “good and consistent evidence." The rest divided into 39.7% for limited or inconsistent evidence and 34.8% for consensus and opinion. When I consider the practice of continuous fetal monitoring, monthly ultrasound, glucose tolerance test, episiotomy, and treatment of Group B Strep in the face of evidence contradicting its utility, this does not surprise me. If departure from physiologic home birth is argued on the basis of “safety," where is the data to support that? I would offer, as have others, that our maternal and infant mortality rates, and two large scale studies also argue for a re-examination of just what we think we are doing to women in this country.

The take-home to this review is to emphasize the fact that conventional medicine often employs consensus opinions arrived at through “expert” discussions. Even if evidence were driving the ship, the evidence itself is compromised by conflicts of interest that have been well exposed. The more you can take your health into your own hands and employ low risk, potentially high yield natural interventions after attending to diet, exercise, and stress management, the less you will get caught in the avalanche of cards when the house falls.