Receiver operator characteristic (ROC) curves for sample assignment to the PCR+CLI+ vs the PCR−CLI− groups based on interleukin (IL) 6 concentration (A) and on the IL-5/IL-6 ratio (B). The values increase from left to right. Note that the IL-6 concentrations are higher for the PCR+CLI+ group while the IL-5/IL-6 ratios are higher for the PCR−CLI− group. Best discrimination is achieved at an IL-6 concentration and IL-5/IL-6 ratio that maximizes the vertical difference between the ROC curve and the line of identity. PCR indicates polymerase chain reaction, the test used to determine the presence (+) or absence (−) of virus; CLI, coldlike illness present (+) or absent (−).
Customize your JAMA Network experience by selecting one or more topics from the list below.
Alper CM, Li-Korotky H, Lo C, Doyle APC, Winther B, Doyle WJ. Nasal Secretion Concentrations of IL-5, IL-6, and IL-10 in Children With and Without Upper Respiratory Tract Viruses. Arch Otolaryngol Head Neck Surg. 2010;136(3):281–286. doi:10.1001/archoto.2010.14
To determine if levels of interleukin (IL) 5, IL-6, and IL-10 or their ratios in nasal secretion are diagnostic of viral upper respiratory tract infections (vURTIs) and coldlike illnesses (CLIs) in children.
Longitudinal study of children for vURTIs, CLIs, and concentrations and ratios of nasal cytokines.
Outpatient assessments of children.
A total of 224 children, aged 1 to 9 years.
Main Outcome Measures
Concentrations of IL-5, IL-6, and IL-10 in nasal secretions, vURTIs diagnosed by polymerase chain reaction (PCR) detection of upper respiratory tract viruses, and concurrent CLIs diagnosed by parents.
Of 1269 secretion samples, 552 (43.5%) were collected during a vURTI (PCR findings positive for an assayed virus [PCR+]). A concurrent CLI was diagnosed for 34% of the PCR+ samples and for 18% of the samples found to be negative by PCR analysis (PCR−). Cytokine concentrations and ratios were highly variable and skewed to the lower values. The significance of the cytokine concentrations and ratios as discriminators of groups defined by the presence or absence of virus and of subgroups defined by the presence or absence of a CLI was evaluated using receiver operating characteristic curves. All measures were significant discriminators of the PCR+ vs PCR− groups, and most were significant discriminators of the paired CLI subgroups. The concentration of IL-6 and the IL-5/IL-6 ratio were the best discriminators across all groups and subgroups. However, the sensitivities and specificities of those discriminators at the best cutoff values were on the order of 0.7 for the most extreme pairwise comparison (PCR+CLI+ vs PCR−CLI−) and lower for the other comparison groups.
The low sensitivities and specificities for cytokine-based assignment of specimens to the paired groups and subgroups limit their usefulness for diagnosis of infection or illness.
Viral upper respiratory tract infection (vURTI) presenting as a coldlike illness (CLI) is extremely common in humans. Causal viruses include rhinovirus, adenovirus, coronavirus, enterovirus, influenza virus, parainfluenza virus, and respiratory syncytial virus (RSV), among others.1 While usually self-limited and of short duration, the primary illness expressions of a vURTI impose significant morbidity in affected persons and are associated with large expenditures of health care dollars.2,3 Moreover, vURTI complications that include otitis media, sinusitis, acute bronchiolitis, exacerbations of asthma, and pneumonia prolong and enhance morbidity, increase the total costs of “treating” the infection, and contribute to excess mortality in select segments of the population.4-7
In the population of individuals with a vURTI, the expressed degree of illness is variable such that a proportion of vURTIs is not associated with a CLI,1,8 and the presence of a CLI is not prerequisite for the development of vURTI complications.9 These observations suggest that extant vURTIs and complications attributable to a vURTI may go undetected by observers in a significant percentage of cases.
Studies have documented the presence of specific cytokines in nasal secretions and wash fluids obtained from individuals with a vURTI.10 These potent intercellular signaling molecules orchestrate and coordinate the host immune and inflammatory responses to noxious stimuli including viral pathogens. Studies of natural vURTIs show that the nasal concentrations for some of these cytokines are greater (1) during vURTIs vs disease-free periods11-13; (2) in vURTIs accompanied by CLIs vs vURTIs without CLIs14; and (3) in vURTIs with airway complications.15-17 Thus, local cytokine levels may serve as markers for the presence of a vURTI and for the degree of illness during a vURTI.18
Herein, we focus on the nasopharyngeal concentrations of 3 cytokines: interleukin (IL) 6, a proinflammatory cytokine the nasopharyngeal levels of which are known to track sign and symptom expression during the course of a vURTI; IL-10, an anti-inflammatory cytokine known to be produced during recovery from a vURTI; and, as a control, IL-5, which is not produced during a vURTI.10,19,20 Concentrations and ratios of these cytokines were assayed in nasopharyngeal secretions and wash fluids collected periodically from children over the course of the typical “cold season.” The samples were grouped by the presence or absence of one or more common upper respiratory viruses, as detected by polymerase chain reaction (PCR) and subgrouped by the presence or absence of a concurrent CLI on the day of sample collection for both virus-positive (PCR+) and virus-negative (PCR−) specimens. The hypothesis tested is that nasal secretion concentrations of IL-6 and/or IL-10 and/or ratios of cytokine concentrations are diagnostic of a vURTI and of a CLI in children.
Nasopharyngeal secretion samples were collected during the course of a 2-center longitudinal study of the relationship between otitis media and vURTIs. Families with at least 2 children aged between 1 and 5 years were enrolled. The 2 index children and any older sibling younger than 10 years who assented to participation were included as subjects. Approximately 30 families in each of 4 consecutive years were enrolled during the month of October and observed through April of that year. Families were reimbursed $100/mo for their participation, and the study was approved by the institutional review boards at the 2 study sites.
The general protocol included initially collecting demographic and related information on the enrolled children, followed by the following regular activities: (1) daily parental assignment of their children's status with respect to the presence or absence of a CLI and of 7 illness signs (ie, runny nose, nasal congestion, sore throat, cough, fever, irritability, and earache); and (2) weekly assessments of the presence or absence of otitis media by pneumatic otoscopy and periodic collection of nasal secretion samples during parent-identified CLI episodes in the child or in an enrolled sibling, at the onset of a new otitis media episode in the child or in a sibling, and at random times during illness-free periods. The data analyzed herein are the results of nasal secretion sample assays for the presence or absence of common upper respiratory viruses; IL-5, IL-6, and IL-10 protein concentrations and the ratios of these concentrations; and the presence or absence of a CLI on the day of secretion collection.
Nasal secretions were collected by aspiration of free secretions or, in their absence, by introducing sterile saline solution into the nasal cavity and aspirating the nasal fluid into a collection trap. All samples were diluted to a volume of 5 mL with normal saline. One milliliter of the sample was added to an aliquot containing 100 μL of viral broth9 and frozen at −70°C for later viral assay by PCR, and the remainder was subdivided into 1-mL aliquots, each frozen at −70°C for later biochemical assays.
The aliquot containing viral broth was thawed and assayed by PCR to detect adenovirus, coronavirus, influenza virus, parainfluenza virus, rhinovirus (picornavirus), and RSV using a protocol previously described.9 One (or 2) of the other aliquots was chosen at random, thawed, and assayed for IL-5, IL-6, and IL-10 using commercially available enzyme-linked immunosorbent assays (BioSource, Camerillo, California), as previously described.21 The lower detection limits for the IL-5, IL-6, and IL-10 assays were 4.0 pg/mL, 0.1 pg/mL, and 0.2 pg/mL, respectively. Samples with nondetectable cytokine concentrations were assigned a concentration equal to one-half the lower detection limit for the respective cytokine. Cytokine ratios were constructed by dividing the cytokine concentration with the larger average value by the cytokine concentration with the lesser average value.
The presence or absence of a concurrent CLI on the day of secretion collection was determined using an algorithm that assigned a CLI to all days when 2 or more of the following parent-recorded signs were present: runny nose, nasal congestion, and/or cough; or to days when runny nose and/or congestion were present and the previous day had been assigned by algorithm to a CLI.18
Samples were assigned to paired groups defined by the presence or absence of a detected virus (PCR+ vs PCR−) and also assigned to subgroups defined by the presence or absence of a clinical illness for both virus-positive (with or without CLI) (PCR+CLI+ vs PCR+CLI−) and virus-negative samples (PCR−CLI+ vs PCR−CLI−). The efficiency of the cytokine concentrations and cytokine ratios to discriminate between paired groups and subgroups was evaluated using receiver operating characteristic (ROC) curves constructed using the NCSS 2004 statistical package (Statistical Systems, Kaysville, Utah). The ROC curve is a plot of the sensitivity vs 1 − specificity for the paired data set over increasing values for the discriminator and evaluates the efficiency of the discriminator for assigning a given sample to the correct group and subgroup. The area under the ROC curve (AUC) for a given discriminator and set of paired groups and subgroups is used to determine the significance of the discriminator for making accurate group and subgroup assignments. The AUC is compared with a value of 0.5 indicative of random sample assignment to the groups and subgroups using a 2-tailed Z test and evaluated at P = .05. That test determines if the assignment of samples to the true paired group and subgroup is significantly better for the discriminator compared with random chance assignment. Note that positive AUCs indicate better than chance assignment of samples to the true group or subgroup, while negative AUCs indicate better than chance assignment of samples to the opposite group or subgroup. Thus, for positive AUCs, the average value of the discriminator is greater for the true group and subgroup, while for negative AUCs, the average value of the discriminator is lower for the true group and subgroup compared with the opposite group.
The ROC curve also allows for the specification of a discriminator cutoff value that maximizes the sensitivity and specificity of the group and subgroup assignments. The value of the discriminator at the maximal vertical distance between the ROC curve and the line of identity (0.5) is projected onto the x and y axes to determine specificity and sensitivity of the discrimination.
Samples were collected from 224 children (118 boys, 190 white, 26 black, and 8 mixed race or other) aged 1 to 8.6 years (mean [SD] age, 3.6 [1.6] years). A total of 1269 samples were assayed for virus and the 3 cytokines; 552 (43.5%) of these were PCR+ for at least 1 of the assayed viruses, and 717 were PCR− for all assayed viruses. The detected virus was rhinovirus in 367 samples (66.5%), RSV in 37 (6.7%), coronavirus in 36 (6.5%), influenza virus in 27 (4.9%), parainfluenza virus in 15 (2.7%), adenovirus in 15 (2.7%), and multiple viruses in 55 (10.0%). Assignments of CLI for the day of sample collection could be made for 502 of the PCR+ samples (90%) and 645 of the PCR− samples (90%) (missing assignments resulted from incomplete parental diary data). A concurrent CLI was present for 171 of the 502 PCR+ specimens (34%) and 118 of the 645 PCR- samples (18%) (P < .001 for the χ2 test).
Table 1 summarizes data for the IL-5, IL-6, and IL-10 concentrations in secretion samples and their ratios for groups defined by the presence or absence of virus detection (PCR+ vs PCR−) and, for both groups, by the presence or absence of illness (CLI+ vs CLI−) on the day of sample collection. The cytokine concentrations and their ratios for all groups were not normally distributed and showed a large variability with significant skewing to the lower values. The median and range of the basal (PCR−CLI−) IL-5 concentrations were much higher than those for either IL-6 or IL-10. In contrast to the other cytokines, the median IL-10 concentration for all comparison groups was equal to the minimum concentration, indicating that more than half of the samples assayed for IL-10 were below the level of detection for that assay.
Table 2 summarizes the statistical data for the ROC curves (AUC [standard error (SE)], Z value, and 2-tailed, P value) constructed for each of the 3 cytokine concentrations and their ratios with respect to assigning samples to the appropriate paired group and subgroup. For the PCR+ vs PCR− comparison, all 3 cytokines and their ratios were significant discriminators. However, while the IL-6 and IL-10 concentrations and the IL-6/IL-10 ratio assigned the samples to the appropriate group, the IL-5 concentration and the IL-5/IL-10 and IL-5/IL-6 ratios assigned the samples to the opposite group (eg, PCR+ samples assigned to PCR− group). For the PCR+CLI+ vs the PCR+CLI− subgroups, the IL-6 and IL-10 concentrations and the IL-5/IL10 and IL-5/IL-6 ratios were significant discriminators, with the 2 cytokine concentrations assigning samples to the correct subgroup and the 2 ratios assigning samples to the opposite subgroup. For the PCR−CLI+ vs PCR−CLI− subgroups, the IL-6 and IL-10 concentrations and all 3 cytokine ratios were significant discriminators. The IL-6 and IL-10 concentrations and the IL-6/IL-10 ratio assigned samples to the correct subgroup, while the IL-5/IL-10 and IL-5/IL-6 ratios assigned samples to the opposite subgroup.
The AUCs (SEs) for all 3 comparisons show that the IL-6 concentration and the IL-5/IL-6 ratio are the best between-group and between-subgroup discriminators, but the sensitivities and specificities of the discriminations were low. This is exemplified in the Figure, which shows the ROC curves for those discriminators as applied to the most extreme paired subgroup assignments: PCR+CLI+ vs PCR−CLI−). Sample assignment to the PCR+CLI+ group based on IL-6 concentration was significantly better than chance (AUC = 0.787 [0.022]; z = 13.23) (P < .001), and sample assignment to the PCR−CLI− group based on the IL-5/IL-6 ratio was significantly greater than chance (AUC = 0.772 [0.020]; z = 13.67) (P < .001). An approximate best IL-6 concentration for the discrimination is 77.0 pg/mL, but the sensitivity and specificity at that cutoff were only 0.73 and 0.72, respectively. An approximate best IL-5/IL-6 ratio was 12.8, but the sensitivity and specificity at that cutoff were both 0.70. The ROC curves for the other cytokines and comparison groups were of similar form, but the accuracies for sample assignment to the paired groups at the best cutoffs were much lower. These results show that while the cytokines and cytokine ratios can discriminate between the various groups and subgroups, the accuracy of the discriminations defined by the respective specificities and sensitivities is less than optimal.
A large number of cytokines could be assayed in this type of study,10 and so it is important that judicious choices be made regarding those selected for assay. Studies that have captured the temporal dynamics for a variety of cytokines elaborated during the course of experimental vURTIs provide some guidance.10,19,20,22 Across studies and viruses, postinfection changes in the concentrations of IL-6, IL-8, IL-10, interferon alfa, and tumor necrosis factor in nasal secretions have been reported, but only IL-6 and IL-8 were consistently detected. In the present study, IL-6 was chosen for assay because of (1) its low basal nasal secretion concentration (as opposed to IL-8, which, like IL-5, is constitutively produced at relatively high levels); (2) its significant-fold increases postinfection; and (3) its direct relationship to CLI signs and symptoms. Interleukin 10 was chosen because it is an anti-inflammatory cytokine produced during the late period of a vURTI and has been related to the development of complications. Interleukin 5 was chosen as a control cytokine because its concentration is not increased during vURTIs.10
Researchers disagree on how to measure and report cytokine concentrations in nasal secretions. Some investigators favor the use of raw cytokine levels,19,23 while others recommend that the concentrations be corrected for saline dilution by dividing by the urea concentration in the sample.24 In the analyses presented herein, the raw cytokine concentrations were used, but in separate analyses, the concentrations were corrected for urea, as previously described.25 The results of the ROC analyses for the 2 data sets were similar, but the raw cytokine concentrations yielded higher AUCs for IL-6 and IL-10 compared with the adjusted concentrations (data not shown).
The findings of our ROC analyses supported the expectation that nasal secretion concentrations of IL-6 and IL-10, but not IL-5, are higher during times of virus detection and when symptoms and/or signs are expressed on the day of secretion collection. Specifically, the results show that the IL-6 concentration was significantly greater in PCR+ nasal secretions than in those without evidence of virus. For PCR+ and PCR− samples, IL-6 concentrations were significantly greater in samples collected on an illness day than those collected on a day without illness. These results suggest that IL-6 concentration is a marker of virus infection during a vURTI and of local inflammation reflected as nasal symptoms and signs, regardless of an identified vURTI. While IL-10 concentration was also significantly greater for those groups and subgroups, that protein was not detected in most samples assayed, which limits its usefulness as a discriminator.
In contrast, IL-5 concentration was measureable in a large percentage of samples, which suggests that it is constitutively produced, but its concentration was lower in the PCR+ samples than in the PCR− samples; its concentration did not differ among subgroups defined by the presence or absence of concurrent illness.
While the measured concentrations of the assayed cytokines and their ratios were capable in most cases of discriminating groups defined by the presence or absence of viral infection and the presence or absence of a CLI, the sensitivities and specificities of the discriminations were low. Indeed, while IL-6 concentration and the IL-5/IL-6 ratio were the best overall between-group discriminators based on their AUCs, the specificities and sensitivities of the discrimination for the 2 most extreme comparisons, PCR+CLI+ vs PCR−CLI−, were only about 0.7. The specificities and sensitivities of the other tested comparisons based on the cytokines and cytokine ratios were much lower.
The moderate to low sensitivities and specificities of the various cytokine discriminators for group and subgroup assignments reflect the observed high variability in the concentrations and ratios of these cytokines for the different groups and subgroups. This large variability was not unexpected. Reports of experimental vURTIs caused by different viruses have described the temporal kinetics for cytokine production during the course of infection.19,20,22 Those results document well-defined virus- and cytokine-specific postexposure changes in cytokine levels (curves) characterized by successive temporal periods of increase, plateau, and decrease that are phase locked to the time of virus exposure.10 In natural vURTIs, the time of exposure and onset of infection are unknown; therefore, sampling covers all times represented by these curves and all cytokine concentrations associated with those times.
In addition, we did not assay all potential viruses (eg, metapneumovirus, bocaviruses), and this may have contributed to the observed variability in the main groups and biased our assignment of samples to the PCR− group. However, the low natural incidence of these excluded viruses implies that such biases would exert a minimal effect on the analyses. Also, parent-assigned signs and symptoms used to determine the presence or absence of a CLI might have varied between episodes and among subjects, thus introducing errors in the CLI assignments. Finally, other natural illnesses not associated with a vURTI (eg, allergy, rhinitis, asthma) can affect the nasal cytokine concentrations, and the presence or absence of these illnesses was not systematically quantified in this study.
In summary, none of the cytokine markers examined in this study are expected to be very useful for assigning nasal secretion samples to groups defined by the presence or absence of virus infection or to subgroups defined by the presence or absence of the signs and symptoms of a CLI. Because we chose cytokines that are arguably the best makers of infection and inflammation, it is unlikely that a single cytokine or panel of cytokines can be used with a high specificity and sensitivity to prospectively identify patients with a vURTI who are “at risk” for complications.
Correspondence: Cuneyt M. Alper, MD, Children's Hospital of Pittsburgh, 3705 Fifth Ave @ DeSoto Street, Pittsburgh, PA 15213 (Alperc@pitt.edu).
Submitted for Publication: June 11, 2009; final revision received August 6, 2009; accepted October 6, 2009.
Author Contributions: Dr Doyle had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Alper, Li-Korotky, Winther, and Doyle. Acquisition of data: Alper, Lo, Cullen Doyle, and Winther. Analysis and interpretation of data: Alper, Li-Korotky, Lo, and Doyle. Drafting of the manuscript: Alper, Cullen Doyle, and Doyle. Critical revision of the manuscript for important intellectual content: Alper, Li-Korotky, Lo, Cullen Doyle, Winther, and Doyle. Statistical analysis: Doyle. Obtained funding: Alper and Doyle. Administrative, technical, and material support: Alper, Lo, and Cullen Doyle. Study supervision: Alper, Li-Korotky, Lo, and Winther.
Financial Disclosure: None reported.
Funding/Support: This work was supported in part by grant DC005832 from the National Institutes of Health and the Eberly endowment to the Children's Hospital of Pittsburgh (Dr Alper).
Previous Presentation: This article was presented at the American Society for Pediatric Otolaryngology Annual Meeting; May 25, 2009; Seattle, Washington.
Additional Contributions: Kathleen Ashe, BA, and J. Owen Hendley, MD, assisted with the virologic assays; Margaretha L. Casselbrant, MD, PhD, Ellen Mandel, MD, Harriette Wheatley, RN, and Ellen Reynolds, MSN, CPNP, assisted with sample procurement; and Brendan M. Cullen Doyle and James T. Seroky, MA, assisted with data entry.