Jha AK. Back to the Future: Volume as a Quality Metric. JAMA Forum Archive. Published online June 10, 2015. doi:10.1001/jamahealthforum.2015.0024
Recently, a group of leading academic institutions asked all hospitals to pledge to minimize the number of patients who undergo certain surgeries performed by surgeons and hospitals who seldom do those procedures. The “Take the Volume Pledge” campaign, initiated by 2 of the most respected experts on quality and safety in the nation, John Birkmeyer, MD, of Dartmouth-Hitchcock health system, and Peter Pronovost, MD, PhD, of Johns Hopkins Medicine, makes a lot of clinical sense. We know that when patients receive these surgeries at low-volume institutions or in the hands of low-volume surgeons, they tend to fare worse.
Ashish K. Jha, MD, MPH
What’s remarkable isn’t that these leaders asked hospitals to focus on volume as a way to improve patient outcomes, but that they are doing it in 2015. And herein lies a story of volume as a quality metric, long used because it was all we had, then falling out of favor, and now coming back again. This story also reminds us of the nation’s inadequate efforts to measure quality meaningfully and the price we pay in lives lost and patients harmed.
We have always known that volume matters. The notion is simple and intuitive: practice makes perfect; experience creates better physicians. Surely, a surgeon who performs a single esophagectomy a year will do the surgery less well and manage complications less effectively than a surgeon who does one every week or even every month. This is why surgical training is so long and why we value experts who have seen many cases similar to the one confronting them today.
And for more than 2 decades, work of leading scholars has shown clearly and convincingly that volume matters. Mortality at low-volume centers for certain procedures is as much as 5 times higher than at the highest-volume centers. Such evidence has prompted quality and safety organizations to encourage patients and providers to use high-volume centers.
Over the past decade, however, quality measurement efforts moved away from volume. Volume was obviously just a proxy. What we cared about was good outcomes, and if we could measure outcomes directly, why bother with volume? Although high-volume centers may be, on average, better than low-volume centers, surely there are some poor-performing, high-volume centers and well-performing, low-volume ones. So the thinking was that we could dispense with volumes if we could directly measure outcomes such as mortality and complications.
But the journey toward measuring outcomes has not panned out as we had hoped. Although some programs, such as the National Surgical Quality Improvement Program (NSQIP), have been measuring outcomes for many kinds of surgeries, most of those data are not widely available. And for the 3% of US hospitals with publicly available NSQIP data, the information is difficult to comprehend.
For all other hospitals, Medicare has been measuring and reporting readmissions, a utilization measure that may be related to quality, using statistical models that ensure nearly every hospital—especially the ones that do only a few cases a year—looks average. In the most recent reports on Medicare’s Hospital Compare, of 3495 hospitals that perform hip or knee surgery, only 32 (less than 1%) were labeled as being worse than expected on readmissions. If everyone is average, why bother improving? What’s worse is that for most of the complicated procedures (eg, colectomy, esophagectomy, or abdominal aortic aneurysm repair—those with high rates of mortality, complications, and a substantial risk of disability) almost no national data exist. One reason for this is that most hospitals have too few cases to create statistically meaningful assessments.
But the problem with our current measurement approach is not only having many institutions with too few cases, but also that we haven’t measured many of the things that matter. For example, what outcomes might be important for a pancreatectomy to treat a resectable pancreatic cancer? Surviving the surgery and the first 30 days thereafter is paramount, but patients care about so much more: not developing a postoperative complication, getting a clean resection of the cancer, surviving to 6 months or a year (or longer), and being able to return to work and perform daily activities, to name a few.
Our national measurement strategy has not focused adequately on these other critical outcomes. If you knew that a surgeon and her team only performed a few cases of pancreatectomies a year but had kept careful records demonstrating that over the past decade, they had achieved high rates of success on effective resections, long-term survival, and other key outcomes, you might very well choose to have her perform your surgery. But most of that information is not available because nobody is measuring it. Instead, we have statistical readmissions models in which everyone looks the same and a few generic quality surgical measures with little information about the specific surgery or surgeon. It’s not surprising that patients have chosen to ignore the information available.
This lack of good information has brought us back to volume, because volume is a reasonable proxy for many of the outcomes that do matter. Higher-volume surgeons and hospitals don’t just have lower short-term mortality, they also have fewer complications, fewer readmissions, and much better long-term functional status and quality of life.
The current return to volume as a proxy for quality also comes with a far more sophisticated understanding of why volume matters, and it’s not just that practice makes perfect. High-volume centers likely have teams that work more effectively together, systems to identify complications early, and the ability to effectively respond to complications. They also may be more likely to have critical support programs, such as wound care, nutrition, and occupational therapy, to maximize patients’ abilities to return to their activities of daily living.
A few years ago, my colleagues and I found that when hospitals with few esophagectomy cases had more nurses, positron emission tomography scanners, and 3 clinically unrelated sets of services (lung transplant, complex oncology, and bariatric surgery) available, their outcomes were nearly as good as high-volume hospitals. And more than a decade ago, a remarkable study found that the volume of lung resections performed in a hospital was a better predictor of mortality after pancreatoduodenectomy than the volume of the pancreatoduodenectomies. These and other studies suggest that the story is a bit more complicated than “practice makes perfect.”
So, in 2015, we’re back to volume as a surrogate for quality. Why have Birkmeyer, Pronovost, and others championed the volume pledge? They understand that it is possible to be low volume and high quality; I suspect they would even agree that if we could perfectly measure all the things that matter, we would not need volume as a surrogate. But, as practitioners and leaders, they have pushed the volume pledge because it will save lives today.
Implementing it will not be without challenges. For example, what should a high-volume hospital do with a newly minted surgeon who hasn’t yet met the volume threshold? And some will inevitably argue that the volume pledge may be self-serving because it creates monopolies for highly reimbursed, complex surgical services. The response to this objection is that the strength of the evidence for volume is clear and the pledge is being championed by 2 clinical leaders who are unmatched in their commitment to and effectiveness in improving patient outcomes.
So hospital leaders around the nation should absolutely take the volume pledge, but we still need to work to develop the measures that are meaningful to patients and clinicians and that are readily available in a way that is accessible and comprehensible. In 2015, we should expect nothing less.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Ashish K. Jha, MD, MPH Ashish K. Jha, MD, MPH, is K. T. Li Professor of International Health and Health Policy at the Harvard T. H. Chan School of Public Health in Boston, Massachusetts, Director of the Harvard Global Health Institute, Professor of Medicine at Harvard...