June 8, 2015

Are the standard measures of health care quality—structure, process and even outcomes—all that good?

A generation ago, many observers doubted that the tools of quality measurement could be applied at all in healthcare.  Now we have moved beyond the belief that “we know quality when we see it.”  We have acknowledged the limitations of structure (e.g., clinician training) and process (e.g., maintenance of normothermia in anesthetized patients) measures.  We have placed the emphasis on outcomes and have created considerable numbers of performance measures to assess clinical outcomes.  Yet, when we examine and compare performance scores across providers, the information is not always convincing.  Varying definitions of outcomes, and the underdeveloped state of risk adjustment methodologies are just some of the factors that make quality data unreliable in many cases.  Quality measurement in healthcare has come a considerable distance, but it still has a long way to go.

That may be the reason for a resurgence of interest in volume or frequency as a determinant of quality.  On May 20, 2015, HealthLeaders Media reported in an online article entitled Limits Urged on Surgeries by Low-Volume Providers that Dartmouth-Hitchcock Medical Center, the Johns Hopkins Hospital and Health System and the University of Michigan Health System had announced a “Take the Volume Pledge” campaign among their combined 20 hospitals to reduce complications linked to insufficient practice by setting minimum volume thresholds on 10 surgical procedures. The limits apply to hospitals and surgeons.  Theoretically, they could apply to anesthesiologists as well.

According to John Birkmeyer, MD, a surgeon, outcomes researcher, and executive vice president for enterprise support services at Dartmouth-Hitchcock, “The effort's first goal is to get hospital leaders to rethink whether they should allow surgeries to be done within their organizations relatively infrequently, and by individual surgeons who rarely tackle such operations.  One option may be to tie surgeon volume with staffing privileges.  ‘We'll worry at another time about outcome differences between medium volume centers and very high volume centers.’”

The minimum numbers of procedures to be performed by hospitals and by surgeons, respectively, appear in the following table:

Volume alone does not demonstrate “quality.”  A surgeon who has a good outcome in 10 out of 20 procedures is not necessarily a better clinician than one who has a good result in eight out of eight cases.  Patient and other factors may account for the apparently different success rates.  More complex cases are often referred to a subset of experienced specialists.  Because those patients are in poorer condition from the outset, results may appear worse.  Generally, however, surgical skill is correlated with experience and in turn with the rate of complications (e.g., “As compared with surgeons in the top quartile of summary skill ratings, surgeons in the bottom quartile had lower mean annual volumes of laparoscopic gastric bypass procedures (53 vs. 157, P=0.005) and of any bariatric procedure (106 vs. 241, P=0.02.”  Birkmeyer JD, et al. Surgical Skill and Complication Rates after Bariatric Surgery.  N Engl J Med 2013; 369:1434-1442.)

After having been out of favor as an indicator of quality for approximately the past 15 years, volume is making a comeback, according to an interesting article by Cheryl Clark, Welcome Back, Volume—the Original Quality Measure, published online in HealthLeaders Media on May 21, 2015.  In addition to the Dartmouth-Hopkins-Michigan “Take the Volume Pledge” campaign mentioned above, Clark notes that (1) U.S. News & World Report’s Best Hospitals for Common Care found that patients who receive five common procedures at low-volume hospitals have a much higher risk of death or complications, while patients at higher-volume facilities have a reduced risk and (2) CMS has begun posting on  the Hospital Compare website the number of Medicare beneficiaries who received care for any of 64 medical conditions or surgeries at various levels of complexity, suggesting that a fragile patient may require a hospital with experience treating a more complex case.

According to the article,

Ashish Jha, MD, director of the Harvard Global Health Institute, and a hospitalist at the VA Boston Healthcare System, agrees that volume is on the rebound.  "But we're swinging back with a far greater sophistication of why volume is important than just 'practice makes perfect,' which is what we thought 15 years ago.  It may be related more toward other components that come with larger volumes, such as more nurses per patient, better nutrition programs, or even having important equipment like a PET scanner."

Birkmeyer says Jha is "exactly right."

"It may have to do, too, with having a more coordinated team that surrounds the surgeon and is there after surgery," Birkmeyer says.  "It means the scrub nurses and the anesthesiologists and other teams work well.

"And that if you're seeing a condition often enough, you're able to make good judgments about who needs surgery in the first place, and obviously making sure you do those surgeries well.

"It may be the team is good at not just avoiding complications but in rescuing patients when things start to go sour.  With greater procedural volume, you become more adept in seeing when something isn't right, and taking the right steps toward pulling a patient out of the fire."

CMS recently announced the third annual release of the Medicare hospital utilization and payment data (both inpatient and outpatient) and the second annual release of the physician and other supplier utilization and payment data.  The 2013 dataset has information for over 950,000 healthcare providers who collectively received $90 billion in Medicare payments.  This new information, accessible at the individual and institutional as well as at the aggregate level, makes it increasingly easy to determine who has performed which procedures and at what cost—but it continues to lack the outcomes data that would make a number such as “60 hip replacements in one year (at average cost)” meaningful.

Volume may help the public assess very broadly the individual and comparative quality of providers—but it does not guide clinicians in performance improvement, which is an objective served by many now-familiar outcomes or performance measures.  Greater volumes of good outcomes indicate concentrations of high-performing providers during the time frame of measurement.  That may be all that they mean.  Defining best practices and measuring hospitals’ and doctors’ ability to maintain or even improve those practices is as important as ever.

With best wishes,

Tony Mira
President and CEO