In Reply: Dr Fierer raises an important point: 3 of the performance measures used by the Hospital Quality Alliance that we studied are not based on evidence from randomized controlled trials. It may, however, be reasonable to implement some performance measures based on observational data because some interventions are not feasible to study in a randomized controlled trial. Limiting measures to those with randomized controlled trial evidence would limit the clinical areas in which performance measurement is possible, increasing the difficulty of providing meaningful information about quality through performance measurement. Indeed, numerous established performance measures are based on results from observational studies and expert consensus.1 However, when the potential benefit from a measured intervention is uncertain or small, there is increased risk that the inaccuracy of performance measurement will outweigh the benefits. We agree that there must be vigilance in balancing the potential unintended consequences of performance measurement, such as the overuse of broad-spectrum antibiotics, with the potential benefits of measures such as early administration of antibiotic therapy for pneumonia, a benefit that has not been demonstrated in randomized controlled trials.
Werner RM, Bradlow ET. Medicare’s Hospital Compare Performance Measures and Mortality Rates—Reply. JAMA. 2007;297(13):1430-1432. doi:10.1001/jama.297.13.1431