Marchalik D, Shanafelt T. Addressing Burnout Among Health Care Professionals by Focusing on Process Rather Than Metrics. JAMA Health Forum. Published online September 30, 2020. doi:10.1001/jamahealthforum.2020.1161
The coronavirus disease 2019 (COVID-19) pandemic has had a profound effect on health care professionals around the world. An early report from China shed light on the pandemic’s negative impact on health care workers’ mental health—a trend that appears especially pronounced among frontline workers, in whom the odds of depression, anxiety, and distress increased by more than 50%.1,2 A similar pattern was observed during the severe acute respiratory syndrome (SARS), Middle East respiratory syndrome (MERS), and Ebola virus pandemics.3
The current crisis places an added strain on health care workers already struggling with rates of burnout and occupational distress that significantly exceed those of the general population.4 Linked to decreased productivity, absenteeism, and turnover, burnout also places increased stress on a health care delivery system already facing extraordinary financial and staffing issues. Therefore, current events have inspired a growing interest in establishing or expanding the organizational infrastructure that addresses system issues that erode the well-being of health care professionals. Although this momentum is leading to positive change, it presents several challenges. Chief among those is defining targets for improvement and success. This is particularly important given health care’s fixation on metrics. But metrics themselves can have a deleterious effect. Where there are metrics, there are rankings, and where there are rankings, a desire to rank first often follows, whatever the method.
This phenomenon is well recognized across a variety of industries. First described in 1975 by economist Charles Goodhart, Goodhart’s Law states that “when a measure becomes the target, it ceases to be a good measure.”5 The famous, and possibly apocryphal, example of Goodhart’s Law is the story of nail factories in the Soviet Union. To boost output, the factory performance was pegged to the number of nails produced, leading to the manufacturing of millions of tiny and useless nails. In response, the performance metric was changed to the nails’ weight. The factories, in turn, adjusted their approach and produced a small number of giant, and equally useless, nails.
Hospitals, too, have been susceptible to Goodhart’s Law. One well-known case is the 2014 Veterans Affairs (VA) scandal. To facilitate timely care of its constituents, the VA instituted a policy that all new patients would be seen within 14 days of the desired date. Wait time for new patient appointments subsequently became a performance metric with direct financial incentives extended to administrators. In 2014, an investigation triggered by a whistleblower complaint uncovered that VA hospitals were using various methods of gaming the wait-time metric, including developing different methods for calculating wait times that appeared more favorable. As a result, thousands of veterans were placed in queues without being placed on an official electronic wait list, and dozens of deaths were allegedly tied to these practices.
Similar issues have been noted with the implementation of the Hospital-Acquired Condition Reduction Program (HACRP)—a pay-for-performance model in which hospital reimbursements are adjusted based on rates of various adverse events, such as catheter-associated urinary tract infections. Although the rates of certain adverse events have ostensibly decreased, a closer look reveals that these improvements could be attributed to decreased testing. By making lower rates of infection both the target and the outcome measure, the system may have inadvertently incentivized hospitals to stop conducting urine and blood cultures and penalized the hospitals that did more screening. To make matters worse, the conflation of metrics and outcomes may stop hospitals from caring for patients with more severe illness or more complex cases.
These examples of Goodhart’s Law must be considered as organizations ramp up their efforts to address burnout in health care workers. Appropriately, systems seek to measure their progress by identifying standardized metrics. Although a variety of instruments along with benchmarks to similar institutions now exist to help to guide these efforts, the risks of measurement must also be carefully weighed. At baseline, no burnout or well-being metric is useful unless there is psychological safety for individuals and organizations to measure and report honestly. Honest reporting helps identify areas most in need of intervention within systems. It also takes away organizations’ need to game scores for fear of its impact on organizations’ reputations.
To counterbalance these pitfalls and prevent organizational well-being programs from succumbing to Goodhart’s Law, health care organizations must focus on process rather than outcome. A successful example of this approach can be found in the well-established and respected American Nurses Credentialing Center’s Magnet Recognition Program. Instead of relying on reported satisfaction scores, the designation criteria includes evidence of investments, such as regular surveying of the nursing staff (independent of results), a confidential feedback process for nurses to provide input, the existence of a chief nursing officer position, and attention to leadership development. As a result, emerging evidence points to improved working conditions for nurses employed by designated Magnet hospitals.6
This type of emphasis on practice transformation is necessary to improve burnout among health care professionals. A promising step in that direction can be found in the American Medical Association’s Joy in Medicine Recognition program. This program focuses on implementing process improvement measures rather than achieving a specific threshold score or ranking. The program also incorporates measures that can be independently audited, such as electronic health record metrics (eg, how much time a clinician spends putting in orders, drafting notes, or working outside scheduled work hours), efficiency (eg, operating room turnover times), and total workload measures, outlined by a National Academy of Medicine Consensus Study. These process measures can be important additional metrics to assess whether the practice environment is designed to cultivate burnout or well-being. Concentrating on the structural changes needed to promote physician well-being, the designation recognizes regular commitment to and assessment of well-being as well as initiatives designed to improve teamwork, support, efficiency of practice, and leadership.
Focusing on process and diversifying well-being metrics cannot guarantee that system-level efforts to decrease burnout will avoid the problems associated with Goodhart’s Law. However, they make it more difficult for organizations to game the system and focus organizational attention on addressing key drivers of burnout as a means to improvement.
Corresponding Author: Daniel Marchalik, MD, MA, Department of Urology, MedStar Health, 110 Irving St NW, Ste 3B19, Washington, DC 20010 (email@example.com).
Conflict of Interest Disclosures: Dr Shanafelt reported holding a patent to the Physician Well-being Index, the Medical Student Well-being Index, the Nurse Well-being Index, and the Well-being Index, with royalties paid; holding a patent to Participatory Management Leadership Index, which is licensed; and receiving honoraria for serving as an expert on the well-being of health care professionals for health care organizations. No other disclosures were reported.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.