Tell Me Something New: Report Cards and the Referring Physician
Article Outline
“But even though quality cannot be defined, you know what quality is.”
–Robert Pirsig, Zen and the Art of Motorcycle Maintenance1
Report cards are used in health care to evaluate hospitals and doctors across a variety of medical conditions and procedures.2 There are 2 main reasons for publishing report cards: to stimulate quality improvement efforts among medical practitioners and to steer patients toward high-quality providers. Ideally, patients benefit as provider quality improves, and better-performing providers are rewarded with recognition and increased volume. While the logic behind report cards is straightforward, there is limited research on their effectiveness.3, 4
Many studies have focused on the New York State cardiac surgery report card, which compares risk-adjusted mortality and other measures for coronary artery bypass graft (CABG) patients across hospitals and surgeons statewide. The evidence from New York is mixed on whether report cards led to genuine quality improvement among providers.3 There is an association between the release of this report card and CABG mortality decreases, although it is unclear how much credit belongs to the report card itself versus confounding factors (such as patient selection, increased regulatory oversight, or others). There is even less evidence to suggest that report cards have improved referral patterns; report card publication has had, at most, a minimal impact on steering patients.
Why might report cards fail to drive patient volume? One possibility is that patients might not have the capacity to choose their provider because of clinical circumstances, health insurance plan restrictions, or other reasons. For example, a patient needing an emergency procedure may not have the time to consult a report card beforehand, let alone have the ability to switch hospitals. Another possibility is that patients are simply not aware of the existence of these report cards. A 1996 survey of CABG patients in Pennsylvania showed that only 12% knew about their state's paper-based CABG report cards.5 Although publishing report cards online has surely helped increase access, we suspect that patient awareness of specific report cards remains generally low. A third possibility is that report cards in their current form may not be understandable or convincing. Well-designed report cards require detailed clinical data and complicated statistical methods to produce, and often are not available until years after the data are collected. Furthermore, in many cases, multiple report cards cover the same area but give conflicting grades, serving only to confuse patients.6
A final, overlooked possibility is that referring physicians do not need report cards to inform them about the quality of hospitals and other doctors; they are skilled at recognizing quality on their own and make referrals accordingly. One study showed that CABG patients in New York were being steered to low-mortality surgeons and away from high-mortality ones even before the first New York report card with surgeon ratings was ever published.7 At least for cardiac surgery, it appears that referring physicians are able to observe the performance of and discriminate among the institutions and physicians to whom they send their patients. However, we know very little about how referral patterns develop or how physicians evaluate their referral options. Studying physicians' referral behavior will yield valuable clues to how they perceive provider quality.
If more research shows that referral patterns reliably capture relative performance information about hospitals and physicians, the information contained in referral decisions could be used to create a new kind of report card. By collecting and aggregating data on physician referral recommendations, we could compare hospitals and physicians by the rate they were referred to, ideally controlling for the appropriate denominator. As referrals may be based on considerations in addition to quality, this approach would be imperfect unless we could account for the other factors. Nevertheless, although any single referral decision may be influenced by financial, interpersonal, and other nonquality factors, pooling data on referral decisions for many patients and by many physicians would help isolate the common signals of quality. Moreover, because referrals occur frequently and all over the country, it would be straightforward to develop timely, easily-interpretable report cards even for specialties lacking valid and reliable outcomes-based performance metrics. One way to begin would be to use large administrative claims databases to trace referrals as patients move through the health care system. These data are collected by Medicare and other insurers, but lack the clinical detail to generate valid outcomes-based report cards. Future efforts could become more sophisticated, perhaps by surveying physicians about their referral preferences directly. Similar work is already underway. Castle Connolly, a commercial firm, bases its “top doctors” designation on surveys asking physicians to identify other doctors who are excellent. Recently, USA Today began featuring the results of an analysis that combines physician data from multiple sources to identify the “most influential doctors” (available at http://www.usatoday.com/news/health/qforma-most-influential-doctors.htm). Even as commercial efforts may contribute valuable information, having a disinterested third party, such as a government entity, produce these report cards would attenuate concerns about conflicts of interest.
Analyzing data on referral patterns to develop consumer decision aids is complementary to other approaches. Outcomes-based report cards offer objective performance assessment, which helps to motivate providers' quality improvement efforts and inform referrals, but they take longer to develop and implement. Approaches that rely on patients' reports about their provider experiences (eg, Zagat's recent collaboration with WellPoint) provide potentially useful and important information on aspects of clinical care not available from other sources. Even so, physicians have the opportunity to observe more information per provider for more providers than do patients, and they also have the medical training (and the ethical imperative) to evaluate the clinical quality of the care offered by others.
Publishing report cards has not led to substantial shifts in patient volume. This lack of response may be revealing to us the wealth of information on quality that physicians already possess. Understanding how physicians recognize quality in their peers and institutions, and leveraging the information contained in their collective referral decisions are, in our view, important next steps toward improving health care quality.
References
- . Zen and the Art of Motorcycle Maintenance. New York: Harper Perennial Modern Classics; 2008;
- . Public report cards—cardiac surgery and beyond. N Engl J Med. 2006;355:1847–1849
- . Do cardiac surgery report cards reduce mortality? (Assessing the evidence). Med Care Res Rev. 2006;63:403–426
- Systematic review: the evidence that publishing patient care performance data improves quality of care. Ann Intern Med. 2008;148:111–123
- . The use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA. 1998;279:1638–1642
- Choosing the best hospital: the limitations of public quality reporting. Health Aff (Millwood). 2008;27:1680–1687
- Quality report cards, selection of cardiac surgeons, and racial disparities: a study of the publication of the New York state cardiac surgery reports. Inquiry. 2004;41:435–446
Funding: None.
Conflict of Interest: The authors have no conflicts of interest to report.
Authorship: Both authors contributed to writing the manuscript.
PII: S0002-9343(09)00799-2
doi:10.1016/j.amjmed.2009.08.010
© 2010 Elsevier Inc. All rights reserved.
