Remember the old children’s game of sitting in a circle and passing secrets from person to person and finding out, once the secret had been passed around the circle, what the original statement actually was? This is sometimes called the Whisper Game. News reports on research studies are sort of like this. It’s interesting to see what spin different reporters put on the research studies they cite.
Kaiser Health News published research results on a study led by Dr. H. Gilbert Welch from the Dartmouth Institute for Health Policy and Clinical Practice in Hanover, New Hampshire. While this research report has no doubt been noted elsewhere, Kaiser listed four news spots mentioning the study (Reuters, Modern Healthcare, Medpage Today, and Medscape) .
The headline: Unjustified Repeat Tests Common Among Medicare Patients
These four news stories all concentrated on emphasizing that a certain percentage of Medicare patients receive diagnostic tests (five tests studied: echocardiography, imaging stress tests, pulmonary function tests, chest computed tomography, cystoscopy, and upper endoscopy) more often than standard protocols suggest. Medscape also noted this study was done with the support of colleagues from the Medicare Payment Advisory Commission, Washington, DC. All the subjects were Medicare patients.
Medpage Today reported that there was an accompanying commentary by physicians which, it appears, was meant to skew those reporting on the research to take the “let’s save Medicare money” approach:
After decades of attention to unsustainable growth in health spending and its degradation of worker wages, employer economic vitality, state educational funding, and fiscal integrity, it is discouraging to contemplate the fresh evidence by Welch et al. of our failure to curb waste of health care resources.
This physicians making these comments had evidently missed Dr. Welch’s caveat from Reuters:
Welch said the only time repeat tests make good medical sense is when patients develop a new set of symptoms that doctors want to check out after the first test. But for physicians, financial incentives typically support more frequent testing, no matter what the purpose.
Or as reported in Medscape:
Welch and colleagues cautioned that they did not incorporate any diagnostic information in the analysis, but the findings raise “the question whether some physicians are routinely repeating diagnostic tests” rather than basing the decision on patient needs.
O.K. Even though the leader of the study admits diagnosis wasn’t part of the statistics used to analyze whether the repeat tests were needed or not, the very next sentence, a non sequitur, if you will, claims doctors order all these unnecessary tests just to make money.
Did you get this? Let’s question what physicians were doing by tabulating the ordering the repeat tests without actually collecting the data to determine if the repeat tests were needed.
Dr. Welsh stated the tests selected for study were ones used to diagnose patients, and he also made very clear that he did not collect data about whether the repeat tests were called for because of new patient conditions. But the conclusions drawn by reviewers were clearly influenced by the media’s common assumptions that physicians don’t know what they are doing and just order tests to increase their income. With a nice push, of course, from the carefully orchestrated “commentary” from physicians who may well have had a vested interest in expressing their opinions.
Unfortunately, readers aren’t going to be able to ferret out these misrepresentations of the data unless they read the find print.
Medicare patients are often elderly, often have multiple chronic diseases, and often develop additional problems or recurring problems because of the complexity of their overall array of health issues.
The key word here is diagnostic. Physicians use these tests to diagnose certain conditions. Many of these elderly patients receiving repeated tests may well have needed them for diagnostic reasons, but this information was not collected.
This study does not show that the repeated tests were unnecessary. It shows nothing more than that a certain percentage of the Medicare population does undergo repeated testing of the five kinds of tests for which data was collected. The data to determine the cause of the repeated testing was not collected. But clearly that doesn’t keep those who report on this study from drawing all kinds of unjustified conclusions from the data.
I’ve posted before about the problems of taking data out of context. One of the best descriptions of the problems with trying to measure something as complex as the quality or need for health care can be found in Ron Baker’s “The Seven Moral Hazards of Measurements.”
Ross M, in his response to Ron Baker’s post, actually mentions the Heisenberg Uncertainty Principle:
Heisenberg demonstrated through his Uncertainty Principle that the very act of measuring something affects it, and therefore introduces a margin of error which one struggles to understand, let alone control.
Ron Baker shared some of his sources for his post on the seven moral hazards in “Moral Hazards of Measurement—Follow-up.”
Evidence-based medicine and health care studies such as that of Dr. Welsh and his collaborators have a purpose, but we must be very careful that the conclusions drawn from the studies are actually supported by the studies. Anyone putting much stock in a research study on health care should be sure that what is whispered in ears is the statement that started the discussion.
