Wisnivesky JP, Henschke C, Balentine J, Willner C, Deloire AM, McGinn TG. Prospective Validation of a Prediction Model for Isolating Inpatients With Suspected Pulmonary Tuberculosis. Arch Intern Med. 2005;165(4):453-457. doi:10.1001/archinte.165.4.453
Copyright 2005 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2005
Current guidelines for the control of nosocomial transmission of tuberculosis (TB) recommend respiratory isolation for all patients with suspected TB. Application of these guidelines has resulted in many patients without TB being isolated on admission to the hospital, significantly increasing hospital costs. This study was conducted to prospectively validate a clinical decision rule to predict the need for respiratory isolation in inpatients with suspected TB.
A cohort of 516 individuals, who presented to 2 New York City hospitals between January 16, 2001, and September 29, 2002, and who were isolated on admission for clinically suspected TB, were enrolled in the study. Face-to-face interviews were conducted to determine the presence of clinical variables associated with TB in the prediction model, including TB risk factors, clinical symptoms, and findings from physical examination and chest radiography.
Of the 516 patients, 19 were found to have TB (prevalence, 3.7%; 95% confidence interval [CI], 2.2%-5.7%). The prediction rule had a sensitivity of 95% (95% CI, 74%-100%) and a specificity of 35% (95% CI, 31%-40%). Using a prevalence of TB of 3.7%, the positive predictive value was 9.6% and the negative predictive value was 99.7%.
Among inpatients with suspected active pulmonary TB who are isolated on admission to the hospital, a prediction rule based on clinical and chest radiographic findings accurately identified patients at low risk for TB. Approximately one third of the unnecessary episodes of respiratory isolation could have been avoided had the prediction rule been applied. Future studies should assess the feasibility of implementing the rule in clinical practice.
Nosocomial transmission of tuberculosis (TB) has become a major concern in the United States, especially transmission of strains that are resistant to antituberculous agents.1- 7 In response to this threat to health care workers and patients, the Centers for Disease Control and Prevention issued guidelines for controlling the transmission of TB in heath care institutions.8 These guidelines are based on early identification and isolation of all patients considered to be at risk for the disease. Specifically, the Centers for Disease Control and Prevention recommendations dictate that patients should be placed in single-bed, negative-pressure rooms until the results of 3 acid-fast bacilli (AFB) smears are negative.
Although these policies have been shown to decrease the rate of TB transmission in certain institutions,9,10 delayed recognition and isolation of patients with active TB stills occurs.11- 15 Delayed diagnosis often arises because clinicians vary in their experience with and ability to recognize TB.16,17 Current guidelines have also resulted in many patients at low risk for TB being isolated unnecessarily.18- 21 In a low-endemic area, for example, TB was confirmed in only 1 of 92 patients who were isolated.21 As the incidence of TB in the United States continues to decline,22 the problem of excessive isolation may become even more significant. Thus, the present strategy of systematic isolation results in the mismanagement of many patients and generates unnecessary expenses for hospitals.
To address this problem, we developed a decision rule that allows physicians to assess TB risk.20 We used a case-control design to identify clinical and chest radiographic findings associated with the presence of TB; these findings were then used to construct a decision rule. The resulting rule comprises 6 clinical findings: TB risk factors or symptoms (exposure to an individual with TB, institutionalization [prison, shelter, or nursing home] in the past 3 years, homelessness, weight loss [≥10% of body weight], night sweats for ≥3 weeks, symptoms of malaise or weakness for >3 months, and persistent fever), self-reported positive purified protein derivative test results, fever (<38.5°C, 38.5°C-39.0°C, and >39.0°C), shortness of breath, crackles on physical examination, and upper lobe disease on chest radiographs (Table 1). Shortness of breath and crackles on examination are negative predictors in the model because they are associated with the absence of TB. When applied to the derivation population, the decision rule had a sensitivity of 98% and a specificity of 46%.
Clinical prediction rules frequently do not perform as well when tested in patients other than those from whom the rule was derived.23,24 The objective of the present study is to prospectively validate the decision rule. The model was applied to a new set of patients to determine its classification accuracy and its potential to reduce the number of unnecessary episodes of respiratory isolation.
The study was conducted at Mount Sinai Hospital, a 964-bed tertiary care teaching institution on the Upper East Side of Manhattan, NY, and at St Barnabas Hospital, a referral center and primary care facility in the Bronx that serves a population with a high prevalence of TB. The institutional review boards of both hospitals approved the study.
Between January 16, 2001, and September 29, 2002, patients who were admitted to the 2 institutions and who were isolated because of suspicion of pulmonary TB were assessed for enrollment in the study. The decision to isolate these patients was not based on the prediction rule. Patients were not eligible for enrollment if they were receiving antituberculous medications at the time of admission, were younger than 18 years, or were admitted to the hospital with the knowledge of an AFB smear with positive results. Diagnosis and treatment decisions were made by the admitting team according to local standards of care.
Study personnel conducted face-to-face interviews using a standardized questionnaire to collect information on each patient at the beginning of isolation. Demographic information included age, sex, race, and immigration status. Risk factors for TB included a history of exposure to an individual with TB, institutionalization, homelessness, human immunodeficiency virus status, and human immunodeficiency virus risk factors. Potential clinical predictors included significant weight loss (≥10% of body weight), night sweats for a minimum of 3 weeks, persistent fever, presence of cough, a history of shortness of breath, hemoptysis, and a history of positive purified protein derivative test results. Data from physical examination included body temperature on hospital admission, oxygen saturation level, and the presence of crackles during chest examination. The following laboratory data were recorded: total white blood cell count, CD4 cell count, and arterial blood gas values.
Chest radiographs were reviewed by investigators unaware of the patient’s clinical data to determine the presence of the following findings: infiltrates, consolidation, cavities, pleural effusion, lymphadenopathy, and fibrotic changes. Finally, the admitting working diagnosis and treatment were obtained from the patient’s medical record. For this study, a patient was considered to have a diagnosis of TB if the admitting team stated in their initial notes that the main working diagnosis was TB and not pneumonia, chronic obstructive lung disease or congestive heart failure exacerbation, or another clinical diagnosis.
The criterion standard that the decision rule was developed to identify was active pulmonary TB. A patient was considered to have TB if at least 1 sputum culture was positive for Mycobacterium tuberculosis. Sputum samples were concentrated and stained with auramine O fluorescent stain to screen for TB. The samples were prepared in Löwenstein-Jensen culture medium and Middlebrook 7H11 selective agar. Specimens obtained at Mount Sinai Hospital were also inoculated into mycobacteria growth indicator tubes. Cultures were maintained for at least 6 weeks to detect the presence of growing organisms.
The mean ± SD value was calculated for each variable. Univariate analysis was performed using the χ2 test for proportions and the pooled t test or Wilcoxon rank sum test for continuous variables to compare the baseline characteristics of patients with and without TB. The classification performance of the decision rule for identifying patients with TB was assessed by calculating sensitivity and specificity, with 95% confidence intervals (CIs).25 The potential reduction in the number of patients unnecessarily isolated was estimated by comparing the actual number of isolation episodes with the theoretical number of isolation episodes had the prediction rule been applied. All analyses were performed using a statistical software program (SAS; SAS Institute Inc, Cary, NC).
Of 753 patients assessed for eligibility during the study, 237 were not included in the study for the following reasons: 103 (43%) refused to participate, 19 (8%) because of language barriers, 4 (2%) could not give consent, and 111 (47%) were discharged from the hospital before the investigators obtained permission from the treating physician to approach the patient. There were no statistically significant differences between patients who did not participate and those enrolled in the study regarding sex and race distribution; enrolled patients, however, were younger (mean age, 46.3 vs 49.8 years; P = .001).
Of the 516 patients enrolled in the study, 19 were found to have culture-proven pulmonary TB (3.7%; 95% CI, 2.2%-5.7%). Of the 19 patients with TB, 14 had positive AFB smear findings (74%; 95% CI, 49%-90%). Mean ± SDage was 46.3 ± 11.9 years, 231 patients were women (45%), and 362 patients (70%) were human immunodeficiency virus positive. Baseline demographic information and clinical symptoms of patients with and without TB are given in Table 2. A reported history of a positive purified protein derivative tuberculin test result and higher oxygen saturation were associated with the presence of TB in univariate analysis (Table 2). History of shortness of breath was associated with the absence of TB (P = .01). There were no statistically significant differences between patients with and without TB in the presence of TB risk factors, hemoptysis, body temperature on hospital admission, the presence of crackles on physical examination, white blood cell count, CD4 cell count, or the alveolar-arterial oxygen level. The working diagnosis by the admitting team was significantly associated with the presence of TB (P = .04). Antituberculous medications were started on hospital admission in 10% of patients with TB compared with 2% of those without TB (P = .005).
Upper lobe consolidation (defined as the presence of any consolidation above the third rib posteriorly, excluding diffuse consolidation consistent with congestive heart failure) was associated with the presence of TB (Table 3). The proportion of patients with upper lobe cavities and reticular and reticulonodular infiltrates was not significantly different between patients with and without TB. Almost 16% of patients with TB had lymphadenopathy (hilar, mediastinal, or paratracheal) on chest radiography compared with 2% of patients without TB (P = .001). Of 19 patients with TB, 18 had scores of 1 or greater (1 patient with TB had a score of 0), indicating the need for respiratory isolation (sensitivity, 95%; 95% CI, 74%-100%). All patients with a score of 10 or higher had TB. Of the 497 patients without TB, 176 had scores of less than 1 (specificity, 35%; 95% CI, 31%-40%). Using a prevalence of TB of 3.7%, the positive predictive value was 9.6% and the negative predictive value was 99.7%. Application of this decision rule to the study population would have led to a 35% reduction in the number of patients without TB who were isolated on admission to the hospital.
Our results suggest that among patients who were isolated for suspected pulmonary TB, those at very low risk could be discriminated from those at higher risk based on data immediately available from the medical history, physical examination, and chest radiographs. Infection control policy that incorporates the prediction model could decrease the frequency of unnecessary episodes of isolation, thereby reducing the cost of hospitalization.
The current recommendations issued by the Centers for Disease Control and Prevention to control the nosocomial spread of TB call for the immediate isolation of any patient suspected of having or known to have infectious TB.8 Because effective respiratory isolation has been shown to reduce the nosocomial transmission of TB,9,10 this measure should be routinely used to protect other patients and health care workers. However, standard criteria for early identification of patients at risk for pulmonary TB have not been developed. The lack of specific criteria has resulted in the delayed isolation of patients with TB and the unnecessary isolation of many patients at low risk for the disease. This pattern could be partially corrected if patients who require isolation were more accurately identified.
The goal of the model is to allow adequate respiratory precautions on hospital admission of all contagious patients while minimizing the isolation of patients without TB. Had the decision to isolate patients been made using the prediction rule, 35% of the TB-negative patients in the study might not have been isolated. The model, however, would have missed 1 of the 19 patients with TB. Given the false-negative rate of 5% observed in the study, clinicians may consider obtaining sputum samples for AFB from patients who are not isolated based on the prediction rule but who are still considered to be at considerable risk for TB by the admitting team.
Although use of the prediction rule may lead to a reduction in the number of patients who were isolated unnecessarily, it is also important to consider the potential public health implications of failing to isolate up to 5% of the patients with TB. These patients can transmit the disease to health care workers and other inpatients, particularly those who are immunocompromised. In some cases, a single index case can initiate a nosocomial outbreak of the disease.3,5 The evaluation of possible contacts of patients with TB who are not isolated is complex and costly and involves many steps.26 Thus, contact investigation and treatment of new cases could potentially offset any cost savings attained by decreasing the number of unnecessary isolation episodes. Future studies should formally evaluate the benefits and potential unwanted consequences of using the prediction rule.
Almost 30% of the patients with TB in this study had negative findings on AFB smears. Of these culture-positive/AFB smear–negative patients, 3 had scores higher than 10. Because all patients with scores of 10 or greater had TB, further diagnostic tests (eg, additional AFB smears, aerosol-induced sputum testing, nucleic acid amplification testing,27 and fiberoptic bronchoscopy in selected cases)28- 30 could be performed before isolation is discontinued to avoid the exposure of health care personnel and other patients. Using the prediction rule to identify culture-positive/AFB smear–negative patients before isolation is discontinued can partially compensate the consequences of failing to isolate some patients with TB. Given the few culture-positive/AFB smear–negative patients in our study, this potential use of the prediction model needs to be further explored.
Several potentially eligible patients were not enrolled in the study; thus, the results may not apply to them. In addition, the clinical prediction rule was validated among individuals who were already isolated on admission to the hospital. Thus, physicians should limit its use to patients already considered for isolation to further assess their probability of TB. Alternatively, the model could be applied to early discharge of isolated patients with a low risk of TB. Future studies should evaluate the performance of the prediction model in all patients with respiratory complaints for whom pulmonary TB is considered in the differential diagnosis.
Decreasing the number of patients with low likelihood of TB who are isolated unnecessarily may reduce hospital costs. These costs, estimated to be as high as $200 per episode of respiratory isolation,31- 33 include the costs of diagnostic tests (AFB smears and cultures), personal protection masks, personnel, and environmental control measures.31- 36 Thus, approximately $5000 was spent per case of TB identified during the study. These estimates are conservative, however, because costs of prolonged hospitalization and indirect medical costs (including lost workdays) are not considered. Conversely, if many patients with TB are missed, the costs associated with late treatment and contact investigation may balance these savings. Formal economic analyses should evaluate the impact of implementing an isolation policy that incorporates the prediction model.
As an alternative to the model validated in the present study, other clinical rules can be used for the prediction of active pulmonary TB. In a study of 563 patients considered to be at risk for TB at hospital admission, El-Solh et al18 derived a decision tree to help clinicians make decisions regarding respiratory isolation. Redd and Susser37 evaluated the exposure histories and clinical findings of patients visiting an emergency department with suspected TB and used these data to develop a rapid decision instrument to predict culture-positive TB. Similarly, Tattevin et al38 derived a TB prediction model using data from 277 patients admitted to several hospitals in France. Overall, the sensitivity and specificity of these rules are similar to those of our model. In addition, there is considerable overlap among the predictors in all the models, further supporting the validity of our findings.
Clinical prediction rules attempt to reduce the uncertainty of medical decision making by standardizing the collection and interpretation of clinical data. Methodological standards for the validation of decision rules have been proposed.24,39,40 Following these standards, the outcome identified by the prediction rule—infectious pulmonary TB—was explicitly defined and determined without knowledge of the predictor variables. Predictor variables were assessed in a standardized manner. The study patients represented the full spectrum of patients at risk for pulmonary TB, and the statistical techniques used to assess the accuracy of the prediction rule were identified. The rule may be considered practical for use in the clinical setting; its purpose is clear and relevant, and the rule is concise. One advantage of the prediction rule is that the information required to determine each patient’s score is routinely obtained at hospital admission, so it can be applied at no additional cost. The rule can also be used in patients with human immunodeficiency virus infection or AIDS. Thus, the classification performance of the rule seems to be adequate based on the results of this study.
The sensitivity of the prediction rule in the validation cohort was similar to that in the derivation sample (95% vs 98%, respectively). The specificity, however, was lower (35% vs 46%). These findings are not surprising because prediction rules often do not perform as accurately when used in populations other than the one in which they are established.24 Even when applied to patients from a similar population, models tend to perform less well when tested prospectively.23
Our study had some methodological limitations. Due to the decreasing prevalence of TB in the United States, there were relatively few patients with TB identified at both institutions during the study. Consequently, the validity and precision of our estimates may be limited. In addition, we did not attempt to evaluate the reliability or reproducibility of the predictor variables, and neither did we assess physician acceptance of the prediction rule. Both hospitals participating in the study were located in New York City. Because the demographic and clinical characteristics of patients with TB may be different in other geographic areas, the model should be validated in other settings before it can be implemented in clinical practice. The prediction rule was not validated among outpatients. Therefore, it should be used only to guide decisions about the isolation of inpatients. Another limitation of the prediction rule is that it has not yet been demonstrated to have an impact on actual clinical practice. The true test of its usefulness will be whether it reduces the number of patients without TB who are isolated without increasing nosocomial transmission of the disease. This will require acceptance of the decision rule by physicians and institutions. The rule has not been tested in patients younger than 18 years; thus, it should be applied only to adults.
The decision rule should be used to facilitate physician decision making but not to replace clinical judgment. There is no need for guidelines if the patient is admitted to the hospital because of a positive AFB smear result or clinically obvious pulmonary TB. Physicians should be cautious with patients who have an upper lobe cavity suggestive of TB on the admission chest radiograph.
In summary, among inpatients who underwent respiratory precautions for suspected pulmonary TB, a prediction rule accurately identified those at low risk for TB who do not require respiratory isolation. The study showed that the prediction rule is sensitive in identifying patients with TB and has the potential to reduce the number of unnecessary episodes of respiratory isolation by 35%. Implementation studies should be performed to assess the actual impact of the rule on clinical practice.
Correspondence: Juan P. Wisnivesky, MD, MPH, Divisions of General Internal Medicine and Pulmonary, Critical Care, and Sleep Medicine, 1470 Madison Ave, Box 1087, New York, NY 10029 (firstname.lastname@example.org).
Accepted for Publication: October 11, 2004.
Financial Disclosure: None.
Funding/Support: This study was supported by grant R03 HS10649-01 from the Agency for Healthcare and Research Quality, Rockville, Md.
Previous Presentation: This study was presented in part at the 26th Annual National Meeting for the Society of General Internal Medicine; May 3, 2003; Vancouver, British Columbia.
Acknowledgment: We thank Albert Siu, MD, MSPH, and David Yankelevitz, MD, for helpful discussions.