Customize your JAMA Network experience by selecting one or more topics from the list below.
Knaus WA. APACHE 1978-2001: The Development of a Quality Assurance System Based on Prognosis: Milestones and Personal Reflections. Arch Surg. 2002;137(1):37–41. doi:10.1001/archsurg.137.1.37
The development of APACHE (Acute Physiology and Chronic Health Evaluation) began on a Saturday in late June 1978 when I walked into the intensive care unit (ICU) of George Washington University Hospital in Washington, DC. I had come to Washington in 1972 and, with the exception of a year spent working in the former Soviet Union, had completed all of my internal medicine and critical care training in DC. This Saturday morning, however, was unique. It was my first as an attending physician, the last day of fellowship training being the previous day.
The ICU was very busy but the charge nurse immediately directed me to one bedside, that of a young woman in severe septic shock following a major surgical procedure. Her room was crowded with house staff and nurses, all frantically attempting to resuscitate and reverse her circulatory collapse. I quickly tried to review her flowchart, discover how she had fallen to this precarious point, and select a useful response. As I leafed through the large and confusing clinical record and scattered sheets of the flowchart, I automatically turned to see who in the room might provide me with advice. There was no one. I realized that, early on this Saturday morning, I was most likely the most senior and experienced person in the hospital.
I immediately ordered more fluid. I tried to maximize her combination therapy of vasopressors, but to no avail. She died within a few minutes. I later learned she had been admitted late that night after an extensive surgical procedure in which a large ovarian tumor was resected with associated necrotic tissue. She had been hemodynamically unstable on admission and had required ever-increasing vasopressor therapy and volume support. Had the seriousness of her condition been recognized early enough? Was her course inevitable? How did it compare with others and had I managed her sepsis as well as possible?
As these thoughts went through my mind that morning, they reinforced a belief I had held since entering clinical training; the quality of my care and my confidence in its outcomes would never be better than the quality of the information behind them. The information in the room that morning was detailed and exhaustive but was impossible to organize and interpret within the time required. There was no way to relate this patient to the outcomes of other similar patients outside my experience, and there was also no way to know whether this was an isolated tragic case or was part of a pattern of patients transferred from the operating room to the ICU insufficiently resuscitated.
The search for a better organized and standardized reporting of clinical data for quality assurance and patient care had started long before I became an attending physician and received the first grant to research severity of illness in 1978. One old but far from original reference is from The British Annals of Medicine in 1837, attributed to William Farr:
The enemy of medical progress is the hypothetical or speculative reasoning of physicians. It is time to tell such persons that these vague speculations . . . are worth nothing and that they can only advance science by registering facts . . . and by applying the mighty instrument of natural science-arithmetic and mathematics.1
And well before we began working on the prototypes that would eventually become APACHE, the surgical and critical care literature had numerous references to laboratory measurements such as single serum lactate levels or to more complex scoring approaches that would describe the course or trajectory of severely ill patients.2,3 Many of these modern designs for a prognostic or risk scoring system for critically ill patients had focused on mathematics because it is an explicit and one of the most judgment free of the human cognitive processes.
Our goal in developing APACHE was also to use mathematics more explicatively to monitor the process and evaluate the outcomes of care. We designed APACHE to occupy a middle ground between the attempts to define illness severity by a single number and those requiring extensive and complex mathematical calculations. We also wanted an approach that could be used universally for most patients, but also be precise enough to have relevance for an individual patient. We were fortunate in our early design years in having advice from Jerome Cornfield, MS, a pioneering biostatistician, who advised us to combine existing clinical knowledge with newer computer-based statistical search techniques rather than relying on one or the other. This would help ensure the precision and reliability of our results. With Jerry's help we devised a comprehensive clinically based conceptual model of the key elements that determined severity and that influenced a patient's outcome from severe illness. We collected this information on a sample of 582 patients from a university and community hospital and published the results in 1981. This was the first article to introduce "APACHE," an acronym we choose to stand for the system's major components—acute physiology, age, and chronic health evaluation.4
The initial reaction by the clinical community to APACHE was encouraging. Although the widespread use of "case mix adjustment" in quality assurance studies is well established, the prospect of having the ability to directly compare the utilization and outcomes of various ICUs in the early 1980s was well received. We followed this initial publication with more multi-institutional testing of the system concentrating on its ability to predict ICU and hospital mortality rates among various types of hospitals. We found APACHE was remarkably accurate in its ability to risk-stratify patients and to accurately predict mortality rates across institutions and even international boundaries. There were APACHE studies in France, Spain, and Finland.5 At this early stage of development when we worked with small patient cohorts from highly selected institutions, there were only minor nonsignificant variations between predicted and observed mortality rates. The system seemed to work well but did not challenge directly the appropriateness of patient outcomes.
Even with these initial limited samples, however, we documented other areas of wide variations among ICUs potentially useful in managing patients and controlling the high cost of critical care. One of the most significant was in the types of patients admitted. For some hospitals we found many patients in the ICU were not very severely ill and most of these patients did not receive any of the life-sustaining therapies such as mechanical ventilation that had prompted the development of ICUs. We termed these patients "low-risk monitor" patients to indicate they were being provided intensive monitoring as opposed to intensive life-support services. Many of these patients were individuals admitted immediately after surgery. Their identification having been anticipated influenced the more recent movement to intermediate and step-down units and this eventually led to national admission, discharge, and transfer guidelines. On the basis of these observations we concluded that there was value to the scrutiny of intensive care using a system like APACHE to document and perhaps improve utilization and outcomes. In a JAMA editorial we published in 1983, however, that summarized how measuring the effect of physiology on outcomes could be used to scrutinize utilization and outcomes, we concluded with a statement that would later be prophetic in regard to the future of APACHE: "While investment in intensive care services exceeds 1% of this country's gross national product, research does not receive any formal support."6(p1060)
The next milestone was the publication of the second version of APACHE, APACHE II in 1985.7 APACHE II simplified the system, reducing the number of physiologic measurements required from 33 to 12 and more precisely represented the complex interactions of disease and severity on prognosis. APACHE II was extremely successful. It was concise and easy to use, and by being linked to the ability to store information within a database, it meant comparisons could be rapidly and easily made among institutions. APACHE II spread rapidly through the clinical community. Within a few years it was named a citation classic as one of the most widely cited articles within the critical care literature (Current Contents, 1993).
The single most important application that helped establish the APACHE II system as a potential "gold standard" for evaluating intensive care was a 1986 article titled, "An Evaluation of Outcome From Intensive Care in Major Medical Centers."8 It applied the APACHE II predictions to a cohort of 5030 patients treated at 13 US hospitals with data collected in the early 1980s. It reported that there was one hospital that had significantly fewer than predicted deaths following intensive care and another that had substantially more based on their APACHE-adjusted case mix. Most importantly, this article suggested that the variation in outcome might be ascribable to marked differences in how well care was coordinated in the ICU, specifically the degree of communication and quality control undertaken by the medical, surgical, and nursing staff.
The article was published at a time when variations in surgical moralities were well established but not yet directly targeted for improvement.9 There was also a small emerging literature on how variations in the process of surgical care with burn and trauma patients might be influencing outcomes.10,11 But, because intensive care relied so heavily on coordination of care among many individuals and specialties, our preliminary results that personal interactions affected outcomes attracted substantial attention.
Increased demand for use of APACHE II occurred at the same time that our growing experience with prognostication and database development provided us with more insights regarding how to improve APACHE's predictive accuracy. Unfortunately, providing support both for institutions and individuals using APACHE to study their ICU utilization and outcomes while also writing grants and doing research to improve the accuracy and usefulness of the system were beyond the capacity of our small but dedicated ICU research team. These tensions, combined with the need to raise sufficient research funding to expand the growing APACHE database, improve its accuracy, and develop the substantial software application that would make it easier to use APACHE in daily practice, prompted us to take what was, then, the unusual move of developing a commercial entity. In 1988, a company based on APACHE was founded, APACHE Medical Systems Inc (AMSI); AMSI was designed to support users of APACHE while providing additional support for research.
There was probably no single action that more directly affected APACHE's subsequent development than the establishment of AMSI by the 4 founders: Jack Zimmerman, MD; Elizabeth Draper, MS; Douglas Wagner, PhD; and myself. Technology transfer and explicit academic-commercial intellectual property arrangements were unusual in the late 1980s and mostly concentrated within the basic sciences. The software applications in health care at the time were preponderantly for capturing charges and rendering bills. The kind of medical decision support systems APACHE envisioned were virtually nonexistent and seldom offered for sale. We were clearly breaking new ground in a number of areas.
With the money raised from venture capital, we were able to support the largest and, to date, most detailed analysis of process and outcome from intensive care and, in so doing, develop the third version of APACHE. This work on APACHE would not have been completed without the substantial funds available by private investors. Our grants from the National Institutes of Health and private foundations were insufficient for the magnitude of this research effort.12
Continuing and expanding work on APACHE was essential. We were beginning to see a larger number of reports describing variations between APACHE's predicted outcomes and observed outcomes. In most cases the authors, using data from one or a small number of ICUs, attributed this to the fact that the APACHE system did not "work" for their patients, a sort of "shoot-the-messenger approach" that required substantial effort to address. While we recognized that APACHE needed some adjustments, we became increasingly convinced that it was able to identify important potential variations in outcomes that were not due to differences in the patients treated.
APACHE III was published in 1991.13 It represented an exhaustive effort to evaluate and improve all components of the system from the weighting and interaction of the physiologic elements, to how the physiologic components interacted with different disease states, to the best way to represent the form of each component. This improved the precision of the system. The improvement was modest in terms of relative improvement (the receiver operator characteristics areas—a common summary measure used to summarize the overall accuracy of prediction went from 0.86 with APACHE II to 0.90 with APACHE III). This improvement represented, however, a major advance in precision since incremental improvements at this level of prognostic accuracy are progressively difficult to obtain. We were also able to take this opportunity to address one frequent criticism of APACHE II by our surgical colleagues, the need for differential weighting of neurological assessment for patients suffering from traumatic and nontraumatic causes of coma, a distinction not present in APACHE II. There were numerous other disease-specific improvements that were made in the scoring and prognostic approach.
At this point, we underestimated the magnitude of a fundamental barrier to the use of APACHE III. We assumed that clinical information and management systems designed for ICUs would rapidly become commonplace. These basic clinical systems would electronically capture and store much of the data required for APACHE scoring as a natural part of electronic charting and creating of the electronic medical record. Extensive efforts were made to embed a large number of predictive algorithms with a modern, easy-to-interpret graphical user interface. This would enable ICU directors and clinicians to monitor and manage their units and their individual patients the same way a pilot monitors and directs an aircraft. There was enormous gratification in unveiling the first products of this development. They made real the concept of walking into an ICU or an individual room and instantly being able to recognize the status of a patient and to track their trajectory as well as estimate their future risk of death, their risk of active treatment, or their probability of being readmitted. A major review article described the system to the scientific community.14
In 1991 the first commercial installation of this comprehensive computerized quality assurance and clinical decision support system was put into place at William Beaumont Hospital, Royal Oak, Mich. It provided both retrospective and prospective applications enabling real-time individual clinical decision support. The release gathered substantial media attention, attracted additional private investment, and formed in 1996 the ability for AMSI to make a public offering that raised $25 million for future research, development, and distribution. This is a small amount by today's standards but seemed an enormous amount of money to us at that time, equivalent to all the money that had been previously raised to support development of APACHE I, II, and III.
This investment, however, came at a price. The results of the research prompted reactions that were unanticipated and are still difficult to place into firm historical context. I will offer my personal view on them; others may have different interpretations.
The first reaction was that since we were one of the earliest commercial applications of medical decision support software, questions were raised regarding whether we had inappropriately used federal research dollars to promote APACHE's commercial operations. The federal funding agency, National Center for Health Services Research, now known as the Agency for Health Care Research and Quality, conducted an extensive inquiry with prolonged and detailed legal documentation. The conclusion was that we had acted well within the regulations. A second controversy emerged when the APACHE III paper was published. It omitted the full equation and coefficients needed to calculate all the predictions. These were considered proprietary intellectual property of the company. Today, such practices are common but at that time it raised a storm of controversy among the journal and its editorial board. Again, however, after a major exchange of documents, the conclusion of the editorial board was that our actions were proper and appropriate.
The reaction of some of our clinical colleagues, however, was highly critical. Many individuals demanded that APACHE software be free and that all services they needed to run and support APACHE be provided without direct or at only nominal charge. When we attempted to point out the substantial cost of research, development, and support, and that other support services, from drugs and devices to information and monitoring systems, were not provided free, they responded that APACHE services were somehow different. When we asked how to pay for them, they said we should get more grants. When we approached the granting agencies they said they did not pay for providing services.
The most disappointing outcome of all this controversy was that the field of risk scoring and prognosis in critical care became confused and fragmented. Instead of the single-standard approach to severity and prognosis that we hoped APACHE III would become, derivative systems like SAPS (Simplified Acute Physiology Score) and MPM (Mortality Probability Models) were promoted as alternatives.15,16 The main professional body, the Society for Critical Care Medicine, launched its own effort to compete directly with AMSI to provide quality assurance and clinical decision support services, Project IMPACT.
None of these efforts contributed new insights into prognosis or risk assessment. However, they did succeed in dividing a small professional community into even smaller camps. What happened here? While the controversy over commercial rights for APACHE was clearly the most visible symbol of resistance to promoting a single standard of quality assurance and outcomes assessment in critical care, I believe there were most likely more fundamental underlying considerations.
When we began our work on APACHE, we assumed that prognosis was central to an ICU clinician's role, whether he or she was a medical or surgical specialist. We envisioned APACHE as enabling a common vision of patients and risk-adjusted outcomes and prompting and empowering individuals to take collective action to monitor and improve outcomes. By the time we developed APACHE III and, particularly, when we saw how the combination of the methodology with a modern graphical user interface made the information easy to access and interpret, we were greatly encouraged.
What we did not anticipate, however, was that unlike diagnosis and treatment, the 2 other major intellectual tasks that clinicians undertake, APACHE's explicit presentation of prognosis would have raised sensitive issues of control. Many clinicians and institutions seemed threatened by the concept of a quality assurance report card evaluating their performance. In a recent study detailing many of these concerns Christakis17 from the University of Chicago describes about how when undertaking diagnosis and treatment, clinicians feel they have some control over the future, but that adding a prognostic estimate threatens both their sense of control and, perhaps, questions the quality of their decisions. There is also the belief that prognosis can be inherently dangerous, that it can become a self-fulfilling prophecy. There were also the military descriptions that Christakis mentions and that I frequently heard when people were discussing APACHE; terms such as "hammering" someone with the quality assurance report or "battering and crushing" a clinician or patient with a poor prognostic estimate. Christakis' discussion of the way prognosis also evokes religious connotations also rang true for me. I can recall innumerable times when I was told I was "playing god" by providing prognostic estimates. But perhaps the most important contribution Christakis makes is that he emphasizes how prognosis is not taught in medical education and is largely missing or ignored in all major medical and surgical textbooks. In this regard one thing that I do feel quite good about is that because of our efforts with APACHE, chapters on prognosis and outcomes evaluation are frequently included in major critical care texts. Over time, these efforts should enable us to introduce more of these concepts into undergraduate and graduate medical education.
So where do we find ourselves in 2001? Since we started work on APACHE in 1978, there have been simultaneous biological and information revolutions that will transform all of American society including medical practice. Information is recognized as a primary economic driver for our economy and increasingly as a force for professions. The role of medical entrepreneurship has become much more common. Scientists and clinicians are patenting and gathering returns under the increasingly common intellectual property and technology transfer arrangements supported by university, commercial, and governmental agencies. The biologic revolution has placed the prospect of genetic information and biologic markers as new potential prognostic factors on the horizon. They will certainly expand and accelerate the movement toward personalized medicine with the ability to much more accurately prescribe and monitor the quality of care and predict outcomes.
And where does this leave APACHE 20 years following its initial publication? We have developed, refined, and created a quality-control system for medical and surgical care that has been widely accepted and has stood the test of time. There are more than 2000 peer reviewed scientific articles in which APACHE is a key component of the methods and results sections. New studies appear every month.
We have many institutions that have successfully tackled the logistical, cultural, political, and practical barriers that have prevented systems like APACHE from playing a wider role in the daily management of patient care. The University of Michigan, Ann Arbor; Baptist Hospital, Miami, Fla; Munson Medical Center, Traverse City, Mich; North Shore Hospital, Long Island, NY; Dominican, Santa Cruz, Calif; and Long Beach Memorial Medical Center, Long Beach, Calif, are all great examples. In each of these institutions there is a dedicated and comprehensive team commitment to collect, interpret, and then act on the APACHE information received. In each institution there is also a combination of committed clinicians, and enabling technology monitors, identifies, and corrects shortfalls in quality using state-of-the-art methods and philosophy, that is, identifying best-practice standards and then moving toward them. In these efforts, one individual, Charles Watts, MD, University of Michigan, has greatly advanced the use of the APACHE system for process improvement and institutional advancement and deserves special mention.
Another very inspiring example of the vision for APACHE is the installation at Sarasota Memorial Hospital in Florida. There the APACHE III Critical Care Management System is directly interfaced to a comprehensive clinical information system that captures and downloads most of the data necessary for calculation of APACHE scores and production of the predictions. Using this system, Sarasota Memorial Hospital has produced impressive improvements in utilization and outcomes with a minimum of manual effort.18
From a practical viewpoint, the recent sale of AMSI to the Cerner Corporation provides an ideal platform from which to continue to promote and develop the knowledge business that APACHE represents.19 The Cerner Corporation is a health care information leader that is at the forefront of integrating most health care data for institutions and multihospital systems; as I emphasized, one of the many frustrations faced during the last 2 decades of work on APACHE was the slow pace and fragmented nature of integration of electronic data collection efforts within ICUs. We had anticipated many more installations such as Sarasota Memorial Hospital where APACHE scoring and predictions were automatic. The partnership with the Cerner Corporation should make this much more common.
The need for a system like APACHE is even more prominent today than it was when we began. As medical and surgical care have become more complex and the patients more chronically ill with multiple conditions, the need for coordinated and sophisticated approaches built on information and not only intuition are becoming more prominent. The Institute of Medicine's recent publication, To Err Is Human: Building a Safer Health System20 is a landmark report indicating that medicine can no longer be left to personal judgment alone. APACHE, having survived and matured during its tumultuous initial 2 decades, is well positioned to provide a scientifically valid and time-tested approach to these challenges.
This does not mean that all of the previously mentioned barriers and concerns, from electronic data capture, close vs open systems, or concerns about risk adjustment outcome reports not being appropriate for an institution will disappear, but they will evolve. I am confident that risk assessment and its application for quality assessment and as an adjunct to clinical decision making will become a progressively more accepted component of medical practice.
This historical perspective was reinforced when I read an account recently of medical science in the second century AD. It was then that Galen, the Greek physician, introduced human dissection and anatomical study, but the field stagnated for almost a millennium. Constrained by the view that the interior of the human body was "God's province," physicians and surgeons concentrated on herbal medicine, leaving procedures that required physical contact between healer and patient to barbers. Medical schools taught anatomy only as a theoretical science.21 We have the opposite challenge. Human anatomy is heavily represented in our medical school curriculums, but offerings emphasizing the anatomy of medical information receive minimal emphasis. I am confident that this too will change, and it will take less than 1000 years.
Corresponding author: William A. Knaus, MD, Department of Health Evaluation Sciences, University of Virginia School of Medicine, Box 800717, Charlottesville, VA 22908.