Customize your JAMA Network experience by selecting one or more topics from the list below.
Walling AM, Tisnado D, Asch SM, et al. The Quality of Supportive Cancer Care in the Veterans Affairs Health System and Targets for Improvement. JAMA Intern Med. 2013;173(22):2071–2079. doi:10.1001/jamainternmed.2013.10797
Characterizing the quality of supportive cancer care can guide quality improvement.
To evaluate nonhospice supportive cancer care comprehensively in a national sample of veterans.
Design, Setting, and Participants
Using a retrospective cohort study design, we measured evidence-based cancer care processes using previously validated indicators of care quality in patients with advanced cancer, addressing pain, nonpain symptoms, and information and care planning among 719 veterans with a 2008 Veterans Affairs Central Cancer Registry diagnosis of stage IV colorectal (37.0%), pancreatic (29.8%), or lung (33.2%) cancer.
Main Outcomes and Measures
We abstracted medical records from diagnosis for 3 years or until death among eligible veterans (lived ≥30 days following diagnosis with ≥1 Veterans Affairs hospitalization or ≥2 Veterans Affairs outpatient visits). Each indicator identified a clinical scenario and an appropriate action. For each indicator for which a veteran was eligible, we determined whether appropriate care was provided. We also determined patient-level quality overall and by pain, nonpain symptoms, and information and care planning domains.
Most veterans were older (mean age, 66.2 years), male (97.2%), and white (74.3%). Eighty-five percent received both inpatient and outpatient care, and 92.5% died. Overall, the 719 veterans triggered a mean of 11.7 quality indicators (range, 1-22) and received a mean 49.5% of appropriate care. Notable gaps in care were that inpatient pain screening was common (96.5%) but lacking for outpatients (58.1%). With opioids, bowel prophylaxis occurred for only 52.2% of outpatients and 70.5% of inpatients. Few patients had a timely dyspnea evaluation (15.8%) or treatment (10.8%). Outpatient assessment of fatigue occurred for 31.3%. Of patients at high risk for diarrhea from chemotherapy, 24.2% were offered appropriate antidiarrheals. Only 17.7% of veterans had goals of care addressed in the month after a diagnosis of advanced cancer, and 63.7% had timely discussion of goals following intensive care unit admission. Most decedents (86.4%) were referred to palliative care or hospice before death. Single- vs multiple-fraction radiotherapy should have been considered in 28 veterans with bone metastasis, but none were offered this option.
Conclusions and Relevance
These care gaps reflect important targets for improving the patient and family experience of cancer care.
Preserving quality of life is an important but often difficult goal for patients and families living with advanced cancer, impaired by their illness as well as its treatment.1 Strong evidence supports many specific care processes to improve quality of life, and research suggests that such care promotes better patient and family outcomes.2-7 Recent studies show shortfalls in even the fairly routine areas of cancer pain and clear communication about goals of care.1,8-11 Given the prevalence of symptomatic and communication concerns and our increasing capacity to address them through intervention, understanding practice shortcomings would highlight opportunities for improvement.4,12-15
Veterans Affairs (VA) has gone further than most health care systems in building palliative care and hospice programs16-18 to address these needs. Critics might worry that this development could come at the expense of standard treatments, but recent studies found that the VA provides at least equivalent cancer care compared with fee-for-service Medicare using the standard metrics of guideline-concordant treatment and survival. Analyses of VA–Surveillance, Epidemiology, and End Results Medicare propensity-matched cohorts with prostate, lung, and colorectal cancer; multiple myeloma; and lymphoma demonstrated that, with few exceptions, the VA provided more appropriate care and equivalent survival.19-21 Data on supportive care for patients with advanced cancer in the VA are sparser.
The nature of VA financing might influence some dimensions of VA cancer care. For example, VA patients have a lower likelihood of receiving newer modalities of radiotherapy for prostate cancer,19,22 perhaps from fewer financial incentives for adoption. Conversely, in supportive care, the VA financing structure may incentivize better quality. For example, chemotherapy in the last 14 days and intensive care unit and emergency department visits in the last month of life were less inappropriately aggressive in the VA than in Medicare.21 Understanding how an integrated system like the VA performs should shed light on trends elsewhere as Medicare increasingly moves away from pure fee-for-service financing. Furthermore, demonstrating the feasibility of patient- and family-centered oncologic measurement would inform the effort of the Centers for Medicare & Medicaid Services to develop cancer care measures.23
We therefore conducted a nationwide, representative retrospective cohort study of VA supportive care quality using a recently developed set of process-of-care quality measures.
We selected a national cohort of veterans with stage IV cancer to represent common solid tumors distinguished by varying prognostic and clinical features. We identified a sampling frame of 424 veterans with pancreatic, 3184 with lung, and 628 with colorectal cancers diagnosed in 2008 from the VA Comprehensive Cancer Registry.24,25 We randomly sampled from each cancer type in equal proportions and confirmed eligibility on the basis of advanced stage and documentation that the veteran was alive for at least 30 days and had at least 1 VA hospitalization of 3 days or more or at least 2 VA encounters of any type (eg, emergency department visit or outpatient visit). Eligibility criteria ensured that veterans had experienced enough VA care to estimate quality. The VA Greater Los Angeles Healthcare System institutional review board approved the study.
Medical record abstraction from diagnosis in 2008 for up to 3 years or until death was used to obtain the data components required to measure quality using the Cancer Quality–Assessing Symptoms and Side Effects of Supportive Treatment (ASSIST) quality indicators (QIs).26 We also collected patient characteristics, including demographics (age, sex, race or ethnicity, and marital status), and clinical variables, including comorbidity, as measured by the Adult Comorbidity Evaluation–27 and receipt of hospice or palliative services.27 We supplemented missing medical record documentation for race or ethnicity with VA administrative data in 7.8% of cases. Urban vs rural residence location was obtained from VA administrative data.
Information regarding age, race or ethnicity, cancer type, and the presence of a hospice referral was collected from the VA Comprehensive Cancer Registry or the medical record for ineligible patients to understand bias related to eligibility criteria.
Medicare-linked files were obtained to conduct sensitivity analyses on how Medicare use affected VA supportive cancer care quality.
Cancer Quality–ASSIST QIs are evidence-based measures of process quality designed to evaluate the supportive care received by patients with cancer, established using the method for QI development, including literature and expert panel review, from RAND Health and the University of California, Los Angeles.26,28-31 We used 42 indicators covering 3 main domains (pain, nonpain symptoms, and information and care planning) demonstrated as valid, reliable, and feasible for medical record abstraction in approximately 500 patients at 3 diverse academic medical centers, including the VA Greater Los Angeles Healthcare System, Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins, and University of California, Los Angeles.9,10,32,33
Three experienced oncologic nurse reviewers (A.P.Z. and 2 others) conducted medical record abstractions after a 3-day training session using a detailed abstraction protocol and computer-aided tool to allow data entry, error checks, and secure data storage. The abstraction team had access to real-time consultation with a senior nurse reviewer (A.P.Z.), met biweekly, and used a log to document common questions and answers to facilitate consistency. They accessed each veteran’s full electronic medical record from local workstations using VA software that allows secure national access of the clinical patient record system. Each medical record abstraction took a mean 2 hours 15 minutes to complete.
A 5% reabstraction sample was studied for all but 2 QIs (which depended only on laboratory data) to ensure the reliability and validity of our data. Seven QIs were dropped because of reliability and/or validity concerns. Of the 35 unique QIs included in the final analysis, 5 had both inpatient and outpatient versions; therefore, scores for 40 QIs are reported. For the final indicator set, interrater agreement was 91.9% (pooled κ, 0.68) for the numerator and 98.8% (pooled κ, 0.92) for the denominator, suggesting good to excellent agreement.34
If a veteran was eligible for a QI, we scored it as 1 or 0 based on adherence or nonadherence to the recommended process. If a veteran was eligible for a QI several times, then a mean score between 0 and 1 was possible. We calculated both event-level (eg, management of each occurrence of pain) and patient-level (eg, rates of adherence to all occurrences of pain aggregated by patient) quality scores, and because they were similar, we report patient-level scores. If patients had a documented contraindication to the care process or a preference against it, we conservatively considered the veteran to have received appropriate care.
We determined adherence to each QI and then the mean patient-level percent adherence to the Cancer Quality–ASSIST indicators in the entire cohort to ascertain the overall quality of supportive care. For example, if a patient was eligible for 6 QIs and received recommended care for 3, his or her overall score was 50%. The overall score for the cohort is the mean of these patient-level scores. To evaluate relative gaps in performance, we determined domain-level ratings of pain, nonpain symptoms, and information and care planning indicators.
We stratified overall and domain-level quality scores by age, comorbidity, cancer type, marital status, and whether the veteran lived in an urban or rural location. Because the quality outcome deviated slightly from normality, we used parametric (t test or analysis of variance with the Tukey-Kramer method for the multiple-comparisons test) and nonparametric (Wilcoxon rank sum or Kruskal-Wallis) tests to evaluate differences.35 Because of similar results, we present information derived from using the parametric tests.
We evaluated differences in the quality scores between veterans who were concurrent users and nonusers of Medicare services to ensure that scores based entirely on the VA medical record did not misrepresent the overall quality of care for veterans with substantial Medicare use. We did not consider hospice use in our definition of Medicare use because patients who primarily receive care in the VA may use their Medicare benefit when referred for hospice services by VA providers through the Hospice-Veteran Partnership Program.36
We conducted a sensitivity analysis with t tests comparing the QI scores for Medicare-enrolled veterans in our sample who had substantial fee-for-service Medicare use (based on the Medicare Provider Analysis and Review, outpatient, and carrier files) with veterans who were enrolled in Medicare but for whom we found little to no Medicare fee-for-service use. We used SAS, version 9.3 (SAS Institute), for all data management and analyses.
Of 719 veterans with stage IV cancer diagnosed in 2008 included in our final sample, 266 (37.0%) had colorectal cancer, 239 (33.2%) had lung cancer, and 214 (29.8%) had pancreatic cancer (Table 1). These 719 veterans were treated at 116 geographically distributed VA facilities with a mean of 6.2 patients per site (range, 1-28). Many were older men (97.2%) with a mean age of 66.2 years. Similar to national veteran demographics,37 most were white (74.3%), with 19.9% black and 4.2% Hispanic, and many resided in urban locations (66.8%). Half were either married (44.7%) or lived with a significant other (6.1%). Almost all veterans had some comorbidity (29.6% severe, 23.6% moderate, and 39.5% mild), and three-fourths received chemotherapy and/or radiation therapy for their disease. Most veterans obtained both inpatient and outpatient care (85.0%), with 8.3% receiving only inpatient care and 6.7% having only outpatient care. All but 54 patients (7.5%) died before the end of the study period, and nearly three-fourths received some form of hospice or palliative care consultation.
Study patients were eligible for a mean of 11.7 QIs (range, 1-22) and received a mean 49.5% of appropriate care overall, regardless of setting (Table 2).
Veterans received appropriate care in the pain domain a mean 68.5% of the time. Inpatient assessment of pain, as well as use of bowel regimens and short-acting pain medications with long-acting medications, had higher adherence compared with similar indicators applied in the outpatient setting. Patients with spinal cord compression received timely treatment only half the time and follow-up 65.0% of the time. Notably, of 28 patients in our sample (all with advanced cancer) who received radiation for painful bone metastases, none received single-fraction treatment or had documentation of a contraindication to this evidence-based treatment of choice.
Veterans received appropriate care in the nonpain symptom domain a mean 36.8% of the time. Similar to patterns found within the pain domain, inpatient QIs measuring nausea or vomiting assessment and pleural effusion treatment and follow-up performed better than their outpatient counterparts. Dyspnea assessment in the outpatient setting (15.8%) and dyspnea treatment (10.8%) scored poorly. Chemotherapy-associated diarrhea had low rates of assessment (12.2%) and treatment (24.2%). Fatigue assessment for patients with advanced cancer was low in the outpatient setting (38.0%) and even lower for patients receiving chemotherapy (31.3%). There was often a lack of assessment for the presence or absence of dysphagia (36.9%), and only 4.3% of veterans who received parenteral or enteral nutrition had documentation regarding indications for such feeding. Depression was diagnosed in 14 patients: 78.6% had a treatment plan documented, and 36.4% had follow-up regarding response to therapy within 6 weeks.
Veterans received appropriate care in the information and care planning domain a mean 45.8% of the time. Timely palliative care was received by most veterans who died in the study (86.4%), and 72.4% had an advance directive or surrogate decision maker documented in the medical record. Sixty percent receiving mechanical ventilation had timely goals of care discussions documented, and 63.7% had them on admission to the intensive care unit. Documentation of a discussion about prognosis and goals of care within 1 month of diagnosis of advanced cancer occurred in 17.7% of patients. This was increased to 35.5% if the patient had brain metastasis. A discussion of the intent of chemotherapy (palliative vs curative) was documented 15.9% of the time. The 1 patient who had an implantable cardioverter defibrillator had its deactivation discussed prior to death (Table 3).
Overall quality scores and quality scores by pain and nonpain symptom domains did not vary between patients 75 years and older and those younger than 75 years, although patients 75 years and older did receive higher-quality information and care planning scores (49.7% vs 44.5%, P < .05). A similar pattern was seen among veterans living in urban areas in comparison with their rural-dwelling counterparts. Patients did not have significantly different overall or domain-specific quality scores by level of comorbidity. In general, supportive care quality scores were higher for veterans with pancreatic cancer compared with patients with colorectal cancer (52.9% vs 46.4%, P < .05). This was largely explained by the fact that patients with pancreatic cancer received higher-quality information and care planning (52.4% vs 40.5%, P < .05). Pain and nonpain symptom management quality was similar across cancer types. Patients who were married or lived with a significant other vs those who did not had lower overall palliative care quality (47.1% vs 51.9%, P < .05), and this was consistent across all domains (Table 4).
We found no significant difference in overall QI scores comparing patients with substantial Medicare fee-for-service use (47.0%) with those having little or no Medicare use (50.2%) (P = .21).
The 360 patients who were sampled but ineligible had no statistically significant difference in terms of race but were older (69.6 vs 66.2 years, P < .001), and almost half (46.9%) were referred directly to a hospice. Among ineligible patients, those who were older (71.2 vs 68.3 years, P = .01) and patients with pancreatic cancer (47.3%) compared with those with lung (32.5%) and colorectal (20.1%) cancer were more likely to have a hospice referral documented (P = .03).
Overall, veterans received only about half of recommended care as measured by Cancer Quality–ASSIST QIs. Performance gaps of this magnitude are unfortunately common.8,10,38 Moreover, the observed deficits in this VA population may underestimate those found in the wider health care system because the VA has emphasized palliative and supportive care. Our findings agree with our more limited previous estimates of supportive care quality.9,10,33 A recent study of lung cancer showed low rates of cancer pain screening in some VA facilities39 similar in magnitude to our results. In addition, estimates of the suboptimal rates of bowel prophylaxis with long-term opioid use and inpatient pain assessments from a national academic collaborative outside the VA were similar to our findings.40
Our study has important strengths and identified many actionable gaps for improving care using clinically detailed quality measures. We found low rates of detection of certain symptoms and lower adherence to follow-up than initial management, suggesting that ensuring screening and follow-up for common symptoms among patients with advanced cancer, such as fatigue, depression, and nausea or vomiting, may need to be targeted much like pain has been (ie, pain as the fifth vital sign). The fact that the Cancer Quality–ASSIST inpatient pain screening QI had almost perfect adherence suggests that such efforts have been successful in changing practices around pain assessment. Such efforts for other high-impact symptoms may be considered.
We found low rates of discussion of goals and preferences for patients with advanced cancer during the month following diagnosis. Patients with pancreatic cancer, the cancer with the shortest expected survival duration, received higher quality care in the information and care planning domain compared with patients with lung and colorectal cancer. This is consistent with previous literature that shows that physicians are more likely to wait to discuss end-of-life issues until late in the disease trajectory, when patients have significant symptom burden and/or no more treatment options.41,42 This may also explain why older patients received higher-quality information and care planning. Our findings support a need for more proactive incorporation of palliative care services in cancer care.4
Urban veterans also received higher-quality care in the information and care planning domain. We know that rural patients have to travel farther to receive care at VA facilities and may have a higher threshold for seeking care, which may lead to fewer opportunities to engage in care planning. Being married or living with a significant other was associated with receiving lower-quality palliative care across all measured domains. This unexpected association warrants further study and may reflect the challenges of broadening patient-centered supportive care to the family unit.
We used robust methods to develop the Cancer Quality–ASSIST QIs published previously.9,10,26,28-33 Our current study had complete ascertainment of medical records from a representative sample drawn from a national cancer registry. We evaluated a large health care system recently shown in comparison with fee-for-service Medicare to deliver better guideline-concordant cancer care, with similar mortality but less aggressive use in the last month of life,19-21 and high-quality medical care in many other chronic conditions.43,44
Limitations of our study include the possibility of overestimating VA performance if non-VA care affects it favorably (eg, prognostic communication outside the VA could facilitate transition to hospice care within the VA); however, most of our indicators (eg, symptom related) directly reflect the process at the point of care. We also limited eligibility to veterans who met a minimum threshold of VA use and confirmed the similarity of ineligible patients in terms of age, race, and hospice use. Next, a sensitivity analysis comparing veterans with and without fee-for-service Medicare use found no statistically significant difference in overall QI scores. Last, because our remote medical record retrieval system did not allow us to review actual advance directives, we relied on clinical note documentation and possibly underestimated quality in that area.
The VA is moving to meet the challenges outlined in our findings, including development and implementation of a practice toolkit to support improvement in symptomatic cancer care and a palliative care national clinical template.45 What are the implications of our findings for the rest of the health care system? One important implication is that tools are available to assess the extent to which supportive care quality might be better or worse outside the VA. Given the National Quality Forum endorsement of Cancer Quality–ASSIST indicators as well as other similar measures, such as those from the Quality Oncology Practice Initiative in recent standards, our results support the application of the Cancer Quality–ASSIST indicators more widely.46-48
Accepted for Publication: July 21, 2013.
Corresponding Author: Anne M. Walling, MD, PhD, David Geffen School of Medicine, University of California, Los Angeles, 911 Broxton Ave, Los Angeles, CA 90095 (firstname.lastname@example.org).
Published Online: October 14, 2013. doi:10.1001/jamainternmed.2013.10797.
Author Contributions: Drs Walling and Lorenz and Mr Pantoja had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Walling, Tisnado, Malin, Dy, Lorenz.
Acquisition of data: Walling, Tisnado, Asch, Malin, Pantoja, Zisser, Schreibeis-Baum, Lorenz.
Analysis and interpretation of data: Walling, Tisnado, Pantoja, Dy, Ettner, Lee, Lorenz.
Drafting of the manuscript: Walling, Tisnado, Schreibeis-Baum, Lorenz.
Critical revision of the manuscript for important intellectual content: Walling, Tisnado, Asch, Malin, Pantoja, Dy, Ettner, Zisser, Lee, Lorenz.
Statistical Analysis: Walling, Tisnado, Pantoja, Ettner, Lee, Lorenz.
Obtained funding: Lorenz.
Administrative, technical, and material support: Walling, Tisnado, Asch, Pantoja, Schreibeis-Baum, Lorenz.
Study supervision: Walling, Tisnado, Dy, Zisser, Lorenz.
Conflict of Interest Disclosures: Dr Malin reported being employed by and having stock ownership with WellPoint. Dr Lorenz reported serving as a consultant to Otsuka Pharmaceuticals for data monitoring and safety in the evaluation of a phase II trial of Sativex, a novel cannabinoid analgesic. No other disclosures were reported.
Funding/Support: This study is based on work supported by the US Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research & Development Service (project IIR 09-097).
Role of the Sponsors: The funding sources had no role in the design and conduct of the study; in the collection, analysis, and interpretation of the data; or in the preparation, review, or approval of the manuscript.
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the US government.
Previous Presentation: The results of this study were presented at the American Society of Clinical Oncology Quality Symposium; December 1, 2012; San Diego, California.
Additional Contributions: We thank our medical record abstraction team, including Anna Dickey and Jennifer Larkin, for their contributions to the VA Cancer Quality–ASSIST national study.
Correction: This article was corrected online September 10, 2014, to fix a quality indicator statement in Table 3.
Create a personal account or sign in to: