Delirium in Mechanically Ventilated Patients: Validity and Reliability of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU)

Context Delirium is a common problem in the intensive care unit (ICU). Accurate diagnosis is limited by the difficulty of communicating with mechanically ventilated patients and by lack of a validated delirium instrument for use in the ICU.

Objectives To validate a delirium assessment instrument that uses standardized nonverbal assessments for mechanically ventilated patients and to determine the occurrence rate of delirium in such patients.

Design and Setting Prospective cohort study testing the Confusion Assessment Method for ICU Patients (CAM-ICU) in the adult medical and coronary ICUs of a US university-based medical center.

Participants A total of 111 consecutive patients who were mechanically ventilated were enrolled from February 1, 2000, to July 15, 2000, of whom 96 (86.5%) were evaluable for the development of delirium and 15 (13.5%) were excluded because they remained comatose throughout the investigation.

Main Outcome Measures Occurrence rate of delirium and sensitivity, specificity, and interrater reliability of delirium assessments using the CAM-ICU, made daily by 2 critical care study nurses, compared with assessments by delirium experts using Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, criteria.

Results A total of 471 daily paired evaluations were completed. Compared with the reference standard for diagnosing delirium, 2 study nurses using the CAM-ICU had sensitivities of 100% and 93%, specificities of 98% and 100%, and high interrater reliability (κ = 0.96; 95% confidence interval, 0.92-0.99). Interrater reliability measures across subgroup comparisons showed κ values of 0.92 for those aged 65 years or older, 0.99 for those with suspected dementia, or 0.94 for those with Acute Physiology and Chronic Health Evaluation II scores at or above the median value of 23 (all P<.001). Comparing sensitivity and specificity between patient subgroups according to age, suspected dementia, or severity of illness showed no significant differences. The mean (SD) CAM-ICU administration time was 2 (1) minutes. Reference standard diagnoses of delirium, stupor, and coma occurred in 25.2%, 21.3%, and 28.5% of all observations, respectively. Delirium occurred in 80 (83.3%) patients during their ICU stay for a mean (SD) of 2.4 (1.6) days. Delirium was even present in 39.5% of alert or easily aroused patient observations by the reference standard and persisted in 10.4% of patients at hospital discharge.

Conclusions Delirium, a complication not currently monitored in the ICU setting, is extremely common in mechanically ventilated patients. The CAM-ICU appears to be rapid, valid, and reliable for diagnosing delirium in the ICU setting and may be a useful instrument for both clinical and research purposes.

Delirium is a disturbance of consciousness characterized by an acute onset and fluctuating course of impaired cognitive functioning so that a patient's ability to receive, process, store, and recall information is strikingly impaired. It is associated with poor outcomes in hospitalized patients, including increased length of stay, the need for subsequent institutionalization, and higher mortality rates.^1-9 Although the frequency of delirium varies from 15% to 50% among general medical or surgical patients,^1,10,11 these rates apply to patients who are not in the intensive care unit (ICU), and few data exist concerning delirium in the ICU.^12-16

Mechanically ventilated ICU patients are at high risk for the development of delirium due to multisystem acute illnesses, comorbidities, medications, and numerous other risk factors.^1,7,16-20 In this population, cognitive impairment has been reported to negatively effect key outcome indicators such as liberation from the ventilator, the development of nosocomial pneumonia, and length of stay.^21-25 However, the incidence and implications of cognitive impairment manifesting as delirium in critically ill ICU patients have been poorly studied, in part because of the lack of a validated instrument to reliably diagnose delirium in mechanically ventilated patients.

Major limitations exist in current methods to identify delirium in the ICU. Standard delirium assessment instruments either require special psychiatric training or have not been validated for use by health care professionals for use in nonverbal patients who are mechanically ventilated.^26-31 In fact, such patients have been excluded from most studies of delirium to date because of the inability to carry out cognitive assessment, which usually required verbal communication.^{1,10,11,17,18,29,31,32}

The Confusion Assessment Method (CAM)³¹ is the most widely used instrument for diagnosing delirium by internists and other nonpsychiatrists^10,17,18 and has been found to have the best combination of ease, speed, reliability, and validity.³³ CAM provides a standardized rating of delirium, which was validated against expert opinion and Diagnostic and Statistical Manual of Mental Disorders, Revised Third Edition (DSM-III-R) definitions of the American Psychiatric Association. The instrument was originally developed for use by nonpsychiatrically trained clinicians to identify delirium in patients able to communicate verbally.³¹ After reviewing the literature and consulting with numerous experts, we chose to adapt the CAM for use in nonverbal, mechanically ventilated patients. The development phase of this investigation led to a small pilot study of the CAM for ICU patients (CAM-ICU)³⁴ that included 22 mechanically ventilated patients. Based on this work, the CAM-ICU was further refined to standardize all elements.³⁵ In keeping with the original intent of the CAM, the adapted CAM-ICU tested in this investigation was designed for use by ICU personnel who have no formal psychiatric training. The purposes of this investigation were to carry out a validation study of the CAM-ICU, to incorporate the improvements from the pilot study, and to determine the occurrence rate of delirium in mechanically ventilated patients.

The study population, none of whom have been included in previous investigations, included mechanically ventilated adult medical and coronary ICU patients admitted to Vanderbilt University's 641-bed academic medical center with 96 adult ICU beds. The institutional review board approved this study, and informed consent was obtained from the patient or surrogate. The study interval was from February 1, 2000, to July 15, 2000. During this time, 158 mechanically ventilated patients were admitted to the ICU, of whom 111 (70.3%) were enrolled and 47 (29.7%) met exclusion criteria. A priori exclusion criteria included a history of psychosis or neurologic disease (eg, cerebrovascular accident) that would confound the diagnosis of delirium (n = 16), inability to communicate with assessors (ie, did not speak or understand English or was deaf) (n = 5), admitted to the ICU but extubated before study nurses' assessments (n = 10), previously enrolled in the study (n = 5), patient or family refusal to participate (n = 6), or died before study nurses' assessments (n = 5). In addition, 15 (13.5%) of the 111 enrolled patients remained comatose throughout the investigation and were excluded from further analysis. The final sample size was 96.

Two critical care study nurses enrolled patients and performed daily, independent CAM-ICU ratings during the patients' ICU stay. The reference standard evaluations were performed independently by the delirium experts (described below), who applied DSM-IV criteria for delirium. The delirium experts demonstrated excellent reliability (κ = 0.97) using standardized methods to complete the DSM-IV ratings during prestudy training period. All cognitive assessments by the nurses and delirium experts were conducted independently in a blinded fashion in the ICU between 1 and 5 PM. None of the raters had access to any of the other's evaluations or ratings. From these ratings, reliability and validity measures were calculated as described below.

Delirium, as defined by the CAM,³¹ has 4 features: (1) an acute onset of changes or fluctuations in the course of mental status, (2) inattention, (3) disorganized thinking, and (4) an altered level of consciousness (ie, other than alert). The patient is determined to be delirious (ie, CAM positive) if he/she manifests both features 1 and 2, plus either feature 3 or 4 (Figure 1). The development of the CAM-ICU began by adapting the patient assessment (without modifying the actual CAM instrument) using nonverbal, objective tests derived through a comprehensive literature review and consultation with numerous delirium experts. This resulted in a pilot investigation that yielded promising results.³⁴ Based on the pilot study, the CAM-ICU wasrefined to improve face validity and further standardize the evaluation, as described below.

The CAM-ICU was completed by the study nurses as described in Table 1. To aid in the assessment of acute onset or fluctuation of mental status changes, patients were followed up daily with the Glasgow Coma Scale³⁶ and an agitation/sedation scale called the Richmond Agitation Sedation Scale (RASS) (See Table 1 footnotes).^37,38 Any fluctuation during the prior 24-hour period in the patient's Glasgow Coma Scale or RASS scores was used by the study nurse to indicate a positive mental status, and as such these scales became a standardized part of the rating of the CAM-ICU by the nurses.

To aid the assessment of inattention, we used an Attention Screening Examination (ASE), which included a visual recognition component and an auditory recognition component. The ASE visual was validated previously by Hart et al^39,40 in the Cognitive Test for Delirium and the ASE auditory was validated as the Vigilance A random letter test.^39,41 Both of these tools are presented in the pilot publication of the CAM-ICU,³⁴ but during refinement it was decided to administer both visual and auditory components of the ASE and limit the number of letters in the sequence of auditory testing to only 10 letters. Data from the pilot study³⁴ were used to establish the cut point for presence of inattention, which was rated as present if the patient scored less than 8 correct answers on either the visual or auditory components of the ASE.

Assessment of disorganized thinking was refined after the pilot study by adding the standardized questions and commands included in Table 1. Disorganized thought was said to be present if the patient was unable to answer at least 3 of the 4 questions correctly and could not complete commands such as those described in Table 1. An altered level of consciousness, was rated as being vigilant or lethargic or in a stupor or coma using the standardized definitions in Table 1.

Reference standard evaluations were performed by a geriatrician delirium expert (J.F.),^42-44 a board certified geriatric consult-liaison psychiatrist (R.M.), or a neuropsychologist (S.G.). The delirium experts, basing evaluation on interviews with family members, the patient's nurse, and chart review for lab data and nursing notes, standardized their approach to DSM-IV delirium ratings during the pilot study.³⁴ Patients were rated as either normal, delirious, stuporous, or comatose using the DSM-IV criteria for delirium or standardized definitions for stupor and coma.³⁴ These experts also rated the patient for the presence or absence of dementia using standard DSM-IV criteria. To increase our sensitivity for detection of dementia, the experts could rate a patient as demented if at any time during his/her course it became clear (eg, through interviews with the patient, family members, the patient's nurse, or chart review) that the patient's baseline cognitive state met DSM-IV criteria. For cases in which the experts rated their own confidence level of DSM-IV ratings for either dementia or delirium as low or intermediate rather than high, 2 raters would form a consensus opinion that day about the rating in question.

Criterion validity was determined by comparing the 2 CAM-ICU raters to the delirium expert rating of delirium using the DSM-IV criterion as the reference standard. Interrater reliability was determined by comparing the CAM-ICU ratings of nurse 1 with those of nurse 2 using the κ coefficient. To determine validity and reliability, we used the first alert or lethargic evaluation of each patient (as rated by the reference standard delirium expert) for the comparison evaluation. This allowed us to avoid repeat observer bias since patients had variable numbers of evaluations. For these evaluations the patient had to be aroused with verbal stimulation, demonstrate eye contact, and follow some commands.³⁸ The a priori decision to include this level of consciousness as the first comparison for validation and reliability testing was made to include only interactive patient evaluations, avoiding comatose and stuporous evaluations because they lack characteristic delirium features and their relative ease of rating might falsely elevate the test characteristics of the CAM-ICU.

Subgroup analyses were conducted for patients who would likely pose the greatest challenge in delirium assessment: those 65 years or older, those with possible dementia, and those with more severe illness (an Acute Physiology and Chronic Health Evaluation II [APACHE II]⁴⁵ score above the median). The first 2 subgroups (older age and possible dementia) were prospectively determined while the subgroup based on higher level of illness severity was not.

Some training is required for optimal use of the CAM-ICU. As an indicator of ease of use, we measured time for completion of the CAM-ICU testing and rating, and completion rates in patients with and without delirium.

In the pilot study of the CAM-ICU,³⁴ the test sensitivity of mechanically ventilated patients was 95% averaging across raters and test specificity was 88%. A group of 12 intensivists agreed a priori that the instrument's sensitivity was the critical feature so as not to miss the diagnosis, stating that it would be important to confirm the lower limit of the 95% confidence interval (CI) to be 85% or higher while an acceptable specificity would be 75% or higher. Sample size was calculated to ensure the appropriate number of patients necessary to achieve the expected lower limit of the 95% CI for the CAM-ICU test sensitivity and specificity in alert or lethargic patients. Assuming an incidence of delirium of 50%, it would require 50 mechanically ventilated patients to ensure that the lower range of the CI for sensitivity in the entire study population was 85% and another 45 patients to ensure that specificity was at least 75%. Thus, the planned total sample size for the investigation was 95 patients. The study was not powered for subgroup analyses.

Patient demographics and the severity of illness measurement using the APACHE II⁴⁵ were collected at time of enrollment.^17,19 Baseline visual or auditory deficits were recorded if patients wore corrective lenses (glasses, bifocals, or contacts) or had a hearing aid, as well as if the family reported that the patient had any documented impairment in vision or hearing. The modified Blessed Dementia Rating scale (mBDRS),⁴⁶ originally validated against brain pathological specimens, was used to screen for dementia using family interviews (an additional surrogate question asked them to rate on a 5-point scale whether they believed the patient was demented). Because baseline dementia could serve as a confounder in rating the CAM-ICU, we chose to increase our sensitivity for detecting dementia by defining patients as having suspected dementia if they met any of the following 3 criteria: (1) delirium expert rated them as having dementia, (2) modified Blessed Dementia Rating scale score of at least 3, or (3) rating by the surrogate of at least 3 out of 5 as possibly having dementia. At the time of hospital discharge, the patients completed a Folstein Mini-Mental State Examination (MMSE)⁴⁷ and CAM-ICU rating. The MMSE is a standard method of screening for dementia or related cognitive impairment with a scale ranging from 0 to 30 points. Scores below 24 indicate cognitive impairment.

To compare demographic variables and other baseline characteristics between enrolled and excluded patients, the t test was used. To compare proportions and rates, χ² tests were used when sample sizes were large and Fisher exact tests when appropriate.⁴⁸ The performance test characteristics for the CAM-ICU were calculated using standard definitions: sensitivity, specificity, predictive values, overall accuracy (true positives + true negatives/true positives + false positives + true negatives), and likelihood ratios (sensitivity/1 − specificity) were estimated from simple 2 × 2 tables. Exact 95% CIs for these test characteristics were calculated using SAS software version 8.02 (SAS Institute Inc, Cary, NC). The CIs for the likelihood ratios were calculated using the bootstrapping technique using STATA 7.0 (STATA Corp, College Station, Tex). All statistical tests were 2 sided, and a P value of <.05 was considered statistically significant.

The reference standard delirium experts and 2 critical care study nurses completed evaluations in 111 patients. Comparison of the demographic variables of the 111 enrolled patients with the 47 patients excluded based on criteria designated a priori showed no significant differences in age, sex, race, severity of illness, or other baseline characteristics (all P>.25). After excluding 15 enrolled patients because they remained comatose throughout the investigation, 96 (86.5%) of 111 enrolled patients remained and comprised the final study population. Characteristics of these 96 patients at the time of enrollment are presented in Table 2. The mean (SD) age of the study population was 55.3 (17.4) years (range, 18-92 years). The study patients, all of whom were mechanically ventilated, had high severity of illness with mean (SD) APACHE II scores of 22.9 (7.2) (median, 23; interquartile range [IQR], 18-29], and a wide spectrum of illnesses. The presence of visual or auditory deficits were reported at baseline in 61.5% of patients, supporting the need to have both visual and auditory means to assess cognitive function as part of the CAM-ICU.

Patients were evaluated during their entire hospital stay and had a total of 471 paired observations between the nurses and delirium experts. Using the first alert (n = 91) or lethargic (n = 93) paired evaluation of each patient, the test performance of the CAM-ICU was determined (Table 3). The 2 nurses' sensitivities using the CAM-ICU compared with the reference standard were 100% for nurse 1 and 93.5% for nurse 2. Their specificities were 97.8% and 100%, respectively. The likelihood ratios for the CAM-ICU for the 2 nurses were 50 and greater than 100, respectively, and the accuracy of the CAM-ICU was 98.4% (95% CI, 92%-100%; P<.001).

In reviewing the 2 misclassified CAM-ICU ratings by the nurses (of which there was 1 false-positive and 1 false-negative), the 2 best explanations for these discordant ratings were: (1) a dose of sedative or analgesic drug had been given between the CAM-ICU rating and the reference standard rating, and (2) a lapse of more than 3 hours between ratings.

In the 84-paired assessments of the first alert or lethargic evaluation for each patient, the CAM-ICU was completed with excellent interrater reliability between nurse 1 and nurse 2 (κ = 0.96, 95% CI, 0.92-0.99).

We conducted 3 subgroup analyses (Table 4) to evaluate the performance of the CAM-ICU in patient groups that could pose particular challenges in delirium assessment: (1) those 65 years or older, (2) those with suspected dementia as defined in the "Methods" section, and (3) those with higher severity of illness. Interrater reliability measures across comparisons showed high agreement with κ values: 0.92 for those 65 or older, 0.99 for those with suspected dementia, and 0.94 for those with APACHE II scores at or above the median value of 23 (all P<.001). Comparing sensitivity and specificity between patient subgroups according to age, suspected dementia, and severity of illness showed no significant differences (all P>.56).

The CAM-ICU instrument and rating were completed in a mean (SD) of 2 (1) minutes, demonstrating the relative ease with which it could be incorporated into nurses' daily routine. When patients were either alert or lethargic, they were able to complete the visual ASE in 167 (69.6%) of 240 cases and able to complete the auditory ASE in 176 (73.3%) of 240 cases. Delirious patients were unable to complete both ASE tests and were considered nonresponsive in 82% of cases while nondelirious patients could complete both ASE tests in 91% of testing attempts.

Delirium occurred in 80 patients (83.3%) during their ICU stay for a mean (SD) duration of 2.4 (1.6) days (median, 2; IQR, 1-3 days). Of 471 daily evaluations completed during this study, a diagnosis by the reference standard of delirium was made in 25.2%, stupor in 21.3%, and coma in 28.5% of all observations (with the remaining 25.0% rated as normal). The level of confidence of the delirium experts was recorded for each evaluation. Overall, the level of confidence was scored as high in 414 observations, moderate in 54, and low in 3. Each of these 57 observations of moderate-to-low confidence was reviewed by 2 delirium experts, who rendered a consensus opinion using DSM-IV criteria. Although the exact duration of the reference standard evaluations was not recorded, a review of notes taken by the delirium experts during the study showed that their delirium evaluations frequently lasted 30 to 45 minutes.

When patients were alert or easily aroused, able to make eye contact, and able to follow commands (as measured objectively using RASS^37,38), they were still found to be delirious in 39.5% of observations as rated by the reference standard and in 42.5% of observations as rated using the CAM-ICU.

The mean (SD) length of stay for the patients in the ICU was 8.3 (6.9) days (median, 6; IQR, 4-11 days) and in the hospital was 17.9 (15.6) days (median, 8; IQR, 8-23 days). In-hospital mortality was 30.2% (29/96) and 6-month mortality was 47.7% (43/90), with 6 of the patients lost to follow-up after hospital discharge. Of the 67 patients (69.8%) surviving to hospital discharge, disposition was as follows: 36 (53.7%) went home, 17 (25.4%) went to a subacute facility, 9 (13.4%) were transferred to another hospital, and 5 (7.5%) went to a nursing home.

At the time of hospital discharge, the mean (SD) MMSE score was 22.2 (7.8). The discharge MMSE was abnormal (a score of <24 out of possible 30 points) in 33 (49.3%) of 67 survivors. Even after excluding 3 patients with suspected dementia at baseline, 44.8% were rated as cognitively impaired by the MMSE at hospital discharge. In addition, the CAM-ICU performed by the study nurses fulfilled full delirium criteria in 10.4% of patients and partial delirium criteria in 20.5% at the time of hospital discharge. Among patients fulfilling at least partial delirium criteria at discharge, 70.3% were judged to require a level of care other than home.

In this investigation, we have validated the CAM-ICU, a 2-minute assessment instrument, which demonstrated a sensitivity of 93% to 100%, a specificity of 98% to 100%, and high interrater reliability (κ = 0.96) in the detection of delirium. In 96 consecutive mechanically ventilated patients, delirium occurred in 83.3% while they were in the ICU. In the 3 subgroups expected to pose the greatest challenges for the CAM-ICU (ie, those 65 years or older, those with suspected dementia, and those with the highest severity of illness), the instrument retained excellent sensitivity, specificity, and interrater reliability.

We simultaneously assessed ICU patients for level of sedation and delirium, finding that 40% of patients who were at a neutral level (neither agitated nor overly sedated) were delirious by both the reference standards' (DSM-IV) and nurses' evaluations (CAM-ICU). Notably, 10% of patients met full delirium criteria, 20% met partial delirium criteria, and nearly 50% demonstrated substantial cognitive impairment (MMSE score, <24) at the time of hospital discharge.

The strengths of this study include the challenging study population of medically diverse but severely ill mechanically ventilated patients, the large number of patient evaluations, and the use of recognized delirium experts for the reference standard ratings. Another important strength of the study design was the use of a standardized, easily performed nursing assessment, which should allow the CAM-ICU to be readily implemented in both academic and community hospitals.

Limitations of this investigation warrant comment. In developing the CAM-ICU, we sought to develop a tool for detecting delirium, not dementia.⁴⁹ However, it is commonplace for mildly demented patients to be cared for in the ICU setting. Because such patients could pose a challenge for the CAM-ICU, we used liberal inclusion of patients with suspected dementia in our subgroup analysis to verify performance of the CAM-ICU in patients with dementia. The findings of 100% sensitivity and specificity in this group are based on a small number of patients but are consistent with our data in another cohort.³⁴ This investigation also represents a selected population at a single site, and future studies will need to evaluate the generalizability of performance across other patient populations including those with a lower prevalence of delirium.

The validation of a delirium instrument for the ICU opens a new frontier for investigation: to evaluate the impact of this important problem in the ICU, especially as it relates to the outcomes of older, mechanically ventilated patients.^50-52 Important areas for future investigation include determination of risk factors for delirium in the ICU²⁰ and the impact of delirium in mechanically ventilated patients on clinical outcomes such as reintubation, nosocomial pneumonia, as well as broader outcomes such as quality of life.^53-57 Although the use of psychoactive medications such as sedatives and analgesics in mechanically ventilated patients is intended to relieve anxiety and suffering, recent studies have suggested that these medications may be prescribed overzealously.^58-61 Interventional trials designed to reduce overuse of these medications and their attendant contributions to delirium and long-term cognitive deficits are greatly needed.

Cognitive impairment in the ICU may be independently related to prolonged neuropsychological deficits,^53,62-66 but studies specifically analyzing the interactions between delirium in the ICU and long-term neurocognitive function are lacking. A significant percentage of individuals developing delirium in the hospital continue to demonstrate symptoms of delirium after discharge.¹¹ Such patients demonstrate decreased cerebral activity and increased cognitive deterioration⁶⁷ and are more likely to develop dementia than patients without delirium.⁶⁸ Finally, patients who develop delirium have a greater rate of decline on cognitive tests than nondelirious patients.⁴³ Importantly, future studies are needed to determine the prognostic significance of delirium in the ICU on long-term cognitive outcomes.

In conclusion, delirium occurred in more than 8 of 10 mechanically ventilated adult medical ICU patients in this cohort, and it was present in 4 of 10 alert or easily aroused patients who are usually assumed to be cognitively intact by ICU personnel. Unfortunately, delirium is often not recognized by clinicians^1,32,69; when it is noted in the ICU, it is often considered an "expected" occurrence attributed to ICU psychosis.^14,70-79 The most common type of delirium, hypoactive or quiet delirium, may be associated with a worse prognosis than hyperactive or agitated delirium.^{2,42,44,80-83} We documented that bedside nurses in the ICU with no formal psychiatric training can reliably detect delirium in mechanically ventilated patients with a high degree of sensitivity and specificity using the CAM-ICU. Requiring only a modest degree of training, this instrument is rapid and easy to use. Incorporation of the CAM-ICU into clinical practice and future investigations may lead to a more precise understanding of the incidence, predictors, and consequences of delirium among critically ill patients.