Five-year overall survival. A, Adult Comorbidity Evaluation 27. B, Washington University Head and Neck Comorbidity Index.
Five-year cancer-specific survival. A, Adult Comorbidity Evaluation 27. B, Washington University Head and Neck Comorbidity Index.
Sanabria A, Carvalho AL, Vartanian JG, Magrin J, Ikeda MK, Kowalski LP. Validation of the Washington University Head and Neck Comorbidity Index in a Cohort of Older Patients. Arch Otolaryngol Head Neck Surg. 2008;134(6):603-607. doi:10.1001/archotol.134.6.603
Copyright 2008 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2008
To validate the prognostic ability of the Washington University Head and Neck Comorbidity Index (WUHNCI) relative to 5-year survival in a cohort of older patients with head and neck cancer and to compare it with that of the Adult Comorbidity Evaluation 27 (ACE-27).
Three hundred twenty-one patients older than 70 years with head and neck cancer in a tertiary cancer center. Comorbidity was measured using the ACE-27, WUHNCI, and National Cancer Institute (NCI) comorbidity index.
Main Outcome Measure
Five-year overall survival.
Five-year overall and cancer-specific survival, respectively, were as follows: Using the WUHNCI, 52.2% and 62.9% for a score of 0; 25.1% and 41.7% for a score of 1; 39.3% and 64.9% for a score of 2; and 19.5% and 45.0% for a score of 3 or higher. Using the ACE-27, 54.4% and 61.7% for a score of 0 (no comorbidity); 46.8% and 61.7% for a score of 1 (mild comorbidity); 31.7% and 51.6% for a score of 2 (moderate comorbidity); and 13.8% and 43.7% for a score of 3 (severe comorbidity). The C statistics were 0.641 for the NCI comorbidity index, 0.656 for the ACE-27, and 0.686 for the WUHNCI.
The WUHNCI did not demonstrate good discriminative power compared with the ACE-27 in a cohort of older patients. To be widely used, instruments developed to measure comorbidities must be reliable in any population. We believe that the ACE-27 is still the best index to assess comorbidities and that it should be used in studies evaluating the prognostic effect of comorbidities.
Head and neck cancer is a leading cause of cancer mortality worldwide. In the United States, 39 000 new cases and 12 000 deaths were registered in 2006.1 Head and neck cancer has a higher incidence among older people mainly because of its relationship to chronic tobacco use and alcohol consumption.2 Besides cancer, these exposures are associated with other significant systemic chronic comorbidities such as pulmonary, cardiovascular, hepatic, and metabolic diseases, which can modify treatment tolerance and affect short-term prognosis. The presence of comorbidities could also change the long-term prognosis of patients with head and neck cancer because they often experience postoperative complications and undergo conservative and less radical procedures. 3,4
Various instruments have been developed to assess the effect of comorbidities on patient survival.5,6 The National Cancer Institute (NCI) comorbidity index describes the number of diseases but does not consider the severity. The Charlson comorbidity index and the Adult Comorbidity Evaluation 27 (ACE-27)7 incorporate measures of severity and have been validated in select cancer populations. The ACE-27, a modification of the Kaplan-Feinstein Comorbidity Index,8 has been widely validated in patients with head and neck cancer. This index is simple, requires little training for use, and has been demonstrated to be reliable in different types of studies.8,9 In 2002, Piccirillo et al6 described a new comorbidity index, the Washington University Head and Neck Comorbidity Index (WUHNCI), derived from the ACE-27. In their study, the authors state that this new index has the same predictive characteristics as the ACE-27, with the advantage of fewer items to be assessed. Until now, to our knowledge, no published study has validated the WUHNCI. Because validation in different populations is an indispensable step to corroborate the predictive characteristics of a new instrument, we decided to test the WUHNCI in a cohort of older patients and to compare its performance with that of the ACE-27 (the accepted standard) and the NCI comorbidity index (a commonly used comorbidity index).
For this study, we used a database collected for a previous study.4 The original study was approved by the hospital committee of ethics and included the medical records of all patients older than 70 years with untreated head and neck cancer admitted to the Hospital do Câncer A. C. Camargo, São Paulo, Brazil, between January 1, 1993, and December 31, 2003. The inclusion criteria used in the original study were a histologically confirmed diagnosis of malignant disease, no distant metastasis, no recurrence, and curative treatment intent (exclusively or as part of a multidisciplinary approach). Patients who underwent surgery for thyroid cancer, skin cancer, melanoma, or orbital tumors were excluded.
Data collection from the medical records was performed using a specially designed survey form. Comorbidities were collected individually (ie, the presence or absence of the selected comorbidity), and their severity was retrospectively determined using the ACE-27. Outcome measures were overall and cancer-specific survival. Because these are time-to-event outcomes, the period was calculated from the first treatment date until the date of the last objective information that was registered in the clinical records. Overall survival was defined as the time from initial treatment to death from any cause. Cancer-specific survival was defined as the time of initial treatment to death from the index cancer, including cancer deaths related to treatment.
The information from the forms was entered into a database (Epi Info; World Health Organization, Geneva, Switzerland). For the statistical analysis, commercially available software (STATA 8.0; StataCorp LP, College Station, Texas) was used. Descriptive statistics were used to show the distribution of comorbidities in the population. Continuous and discrete variables were categorized to facilitate data analysis and presentation. Univariate analysis (Kaplan-Meier method) was performed to explore the relationship between comorbidity score and outcome events. Actuarial overall 5-year survival was calculated for each index discriminated by score. The C statistic was calculated for each index to assess the prognostic performance. The C statistic interpretation is similar to that of a receiver operating characteristic curve, and its values range from 0.5 (no discrimination) to 1.0 (perfect discrimination).
Clinical records of 477 patients older than 70 years were examined at the hospital medical archives. Three hundred twenty-one patients fulfilled inclusion and exclusion criteria. The mean (SD) age was 75.8 (5.2) years (age range, 70-93 years; median age, 75 years). Two hundred twenty-eight patients (71.0%) were male, and 286 patients (89.1%) were of white race/ethnicity. Most tumors were located at the oral cavity (36.7%), larynx (25.2%), and oropharynx (19.3%). Most patients (93.8%) had squamous cell carcinoma. Sixty-two patients (19.3%) were classified as having TNM stage I cancer, 62 (19.3%) as having stage II cancer, 77 (24.0%) as having stage III cancer, and 120 (37.4%) as having stage IV cancer. The mean (SD) overall survival was 34.6 (32.9) months, the median survival was 23.8 months, and the 5-year overall survival and cancer-specific survival were 42.3% and 56.3%, respectively.
Two hundred thirty-nine patients (74.5%) had at least 1 comorbidity. The classification of patients' comorbidities by the NCI comorbidity index, ACE-27, and WUHNCI is summarized in Table 1. Actuarial 5-year overall survival and cancer-specific survival for each comorbidity index are given in Table 2. Comparative curves for 5-year overall and cancer-specific survival between the ACE-27 and the WUHNCI are shown in Figure 1 and Figure 2. The C statistics were 0.641 for the NCI comorbidity index, 0.656 for the ACE-27, and 0.686 for the WUHNCI.
Comorbidity has been defined as the presence of disease unrelated to the disease under study.8 For patients with cancer, examining the effect of comorbidities is relevant because known relationships exist between some chronic diseases and a higher risk of cancer development. In addition, comorbidities can frequently influence treatment selection and administration.10- 12 This is specifically important in head and neck cancer, in which smoking and alcohol consumption are the most important risk factors, but they are also risk factors for developing other cardiovascular, pulmonary, and metabolic diseases. Comorbidities can act as confounding factors when prognosis is measured. Certain therapies such as chemotherapy cannot be offered in the presence of specific comorbidities. In addition, physicians may be obligated to administer suboptimal treatments to patients with comorbidities. All of these factors contribute to decreases in disease-free survival.13
Several instruments have been developed for comorbidity measurement. Kaplan and Feinstein7 developed a specific index in a group of patients with diabetes mellitus in 1974. In 1987, Charlson et al14 devised another specific scale to measure comorbidity. This scale was derived and validated in a cohort of medical patients and reliably predicted survival in patients undergoing head and neck cancer surgery.15 In 2000, the Kaplan-Feinstein Comorbidity Index was modified by Piccirillo16 as the ACE-27, which has been widely validated in patients with head and neck cancer.6,8,9,16- 20 This test is reliable and easy to use, requires little training on the part of the test administrator, and can be calculated retrospectively. All of these variables make the ACE-27 one of the best instruments to assess comorbidity in patients with head and neck cancer.8,9
In 2002, Piccirillo et al6 developed a new comorbidity index, the WUHNCI. This index was derived from the ACE-27 after a statistical analysis, had fewer variables, and assigned a score to each variable, giving an overall score that can predict 5-year survival. The authors stated that the WUHNCI predicted survival better than the ACE-27, despite the use of fewer variables. This suggested an important contribution to the analysis of comorbidities in patients with head and neck cancer because fewer variables facilitate completion of the scale and may decrease the variability associated with an instrument containing many variables, as occurs with the ACE-27.
However, as stated by Justice et al,21 an instrument must be validated in populations different from those in which the original scale was developed to demonstrate its reliability and to assess its prognostic ability. Therefore, we decided to assess the validity of the WUHNCI in a cohort of older patients with head and neck tumors, comparing it with that of the ACE-27 and NCI comorbidity index. Our study could not reproduce the results of Piccirillo et al6 mainly because of an inability of the WUHNCI to discriminate adequately the prognostic groups. The survival curves produced by the ACE-27 and the WUHNCI were not comparable, and the survival probability obtained for patients with a high score on the WUHNCI was better than that obtained for patients with a low score on the WUHNCI, which is contradictory. Although the C statistic was higher for the WUHNCI compared with the ACE-27, this isolated statistical condition could not support the discriminative prognostic power of the WUHNCI. We believe that the C statistic assesses the index globally and in a discriminative way (it is a test similar to the receiver operating characteristic curve) in which individual cut points with better performance could skew the global value of the test as occurs with the WUHNCI for levels 0 and 2. However, when individual cut points are evaluated independently in a predictive way, as occurs with the Kaplan-Meier test, results are contradictory.
We believe that there are reasons to explain these conflicting results. First, in the development of the WUHNCI, the authors first eliminated 13 of 31 ailments originally included in the ACE-27 because of their low frequency in the database.6 Although less frequent, some eliminated variables such as stroke, cardiomyopathy, diabetic organ disease, and moderate or severe chronic liver disease are so severe that they could independently cause a worse prognosis. In our database, 3 patients had diabetes mellitus with end-organ damage and 16 patients had sequelae of a cerebrovascular event, which would not be adequately assessed by the WUHNCI. Second, the final index as shown in the original article6 does not establish ailment severity and simply describes the illness. For example, pulmonary disease could correspond to well-controlled or uncontrolled chronic obstructive pulmonary disease, and/or cardiac arrhythmia could represent second- or third-grade blockade or well-controlled atrial fibrillation, which ultimately corresponds to the same score of 1 (pulmonary disease) or 2 (cardiac disease). As occurs with the NCI comorbidity index, this removes the influence of severity of illness (which is adequately assessed by the ACE-27) from the WUHNCI, resulting in the loss of an important prognostic effect. Third, in the population used for the development of the WUHNCI, only 38% of the patients were older than 65 years.6 As we suggest in another study,4 geriatric populations have specific characteristics, and their prognosis depends on other variables (such as functional status) in addition to the simple evaluation of the presence and severity of comorbidity.
To be widely used, an instrument developed to assess comorbidity should have similar outcomes in different populations independent of age or sex. The results of this study demonstrate that the WUHNCI does not offer predictive ability similar to that of the ACE-27 in a cohort of older patients. We recommend the use of the ACE-27 to assess comorbidities until new instruments demonstrate better performance.
Correspondence: Alvaro Sanabria, MD, MSc, PhD, Department of Surgery, School of Medicine, Universidad de La Sabana, Fundación Abood Shaio, Campus Puente del Comun, Km 21 Autopista Norte, Bogotá Colombia (email@example.com).
Submitted for Publication: April 17, 2007; final revision received June 19, 2007; accepted July 3, 2007.
Author Contributions: Drs Sanabria and Vartanian had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Sanabria, Carvalho, Vartanian, Magrin, Ikeda, and Kowalski. Acquisition of data: Sanabria and Vartanian. Analysis and interpretation of data: Sanabria. Drafting of the manuscript: Sanabria, Vartanian, and Magrin. Critical revision of the manuscript for important intellectual content: Sanabria, Carvalho, Ikeda, and Kowalski. Statistical analysis: Sanabria. Obtained funding: Kowalski. Administrative, technical, and material support: Vartanian, Magrin, and Ikeda. Study supervision: Sanabria, Carvalho, Vartanian, and Kowalski.
Financial Disclosure: None reported.
Funding/Support: This study was supported by Universidad de La Sabana (Dr Sanabria) and by Fundação Antonio Prudente.