In my statistics class many moons ago, we were taught that as statistical measures, surveys were at the bottom of the heap of ways to establish meaningful evidence. It was implied that if anyone in the class planned to use surveys in their studies, he or she would be laughed out of the classroom.
So low and behold, all these many moons later, surveys are being promoted as something reliable enough to be used to help determine health care provider reimbursment
Who do the promoters of surveys as reasonable ways to establish patient satisfaction—and then basing physician reimbursement on it—think they are kidding?
Paul D. Cleary in “The Increasing Importance of Patient Surveys” suggests “…there now are valid and reliable instruments that ask patients objective questions about aspects of care that both clinicians and patients think represent quality.” That is, payers now supposedly have a reliable way to relate payments for providers to patient satisfaction with their care.
Let me be really blunt here. There are enough “what if’s” here to sink a battleship.
Cleary says surveys, now that the measurement techniques have improved, can guide improvement in healthcare delivery. Yes, opinions are appropriate for guiding improvements.
However, Cleary’s article was published in the British Medical Journal in 1999, over ten years ago. In terms of research, this is an aged article. Not quite Rip Van Winkle, but approaching it.
In this article, Cleary admits that surveys were flawed measures of healthcare quality before “rigorous methods” began to be used to gather these opinions. Rigorous methods are defined as no longer using “clinical conversations” to elicit patient views regarding treatment decisions and quality of care, but instead asking patients questions about their specific experiences with the care offered (rather than, for example, the color of the carpet). With these “improved measures,” Cleary states that “such questions are less subjective and less influenced by patient characteristics, are more interpretable, and thus may be acted on for quality improvement purposes.”
Wait a minute. A survey, then, is still a survey. That is, opinion is subjective. This change in topic from carpet color to patient care is somehow perceived as less subjective and less influenced by patient opinion. Huh?
As for “more interpretable,” more interpretable than what? Surveys based on poorer questions?
I have no objections to surveys being used to find out what kinds of things might make a patient experience more pleasant. Actually, most health care entities wouldn’t need to bother with surveys if they paid any attention to the complaints patients reported.
My problem is with the implication that surveys are somehow a reliable measure of anything statisticians consider meaningful. Surveys are totally dependent upon asking the right question at the right time, and in the right context, but no amount of rightness of the question is going to change the fact that we are dealing with people’s opinions here. What one person loves another may hate. There’s an underlying assumption here that everyone will recognize quality health care when they experience it. Anyone who spent any time visiting with physicians would know this simply is not true.
To the credit of Paul Cleary, he does clearly state in his article that from the outset, those in his study agreed “to use the data not to judge ‘winners and losers’ but to educate and inform hospitals and consumers and to focus and facilitate quality improvement efforts.” Paul Cleary made it very plain in his article that the “focus is not on identifying ‘bad apples’ but on stimulating and guiding quality improvement efforts.”
So how, then, do we come to the issue of how surveys, because studies have shown survey designers have learned to better hone their questions, have somehow been magically elevated to highly respectable statistical data—the kind of data one might have some justification for basing provider reimbursement.
Measuring? No. Sampling what kinds of care people like or dislike. Sure. It’s opinion. But measuring, as in basing practitioner reimbursement on the perceptions of patients who may or may not recognize quality health care when they see it—absolute rubbish!
