Figure. Patient flow diagram.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Fine AM, Nizet V, Mandl KD. Large-Scale Validation of the Centor and McIsaac Scores to Predict Group A Streptococcal Pharyngitis. Arch Intern Med. 2012;172(11):847–852. doi:10.1001/archinternmed.2012.950
Author Affiliations: Department of Medicine, Division of Emergency Medicine, Children's Hospital Boston, Boston, Massachusetts (Drs Fine and Mandl); Department of Pediatrics (Drs Fine and Mandl) and Center for Biomedical Informatics (Dr Mandl), Harvard Medical School, Boston; Department of Pediatrics and Skaggs School of Pharmacy and Pharmaceutical Sciences, University of California, San Diego, La Jolla (Dr Nizet); and Children's Hospital Informatics Program, Harvard-MIT Health Sciences and Technology, Boston (Dr Mandl).
Background The Centor and McIsaac scores guide testing and treatment for group A streptococcal (GAS) pharyngitis in patients presenting with a sore throat, but they were derived on relatively small samples. We perform a national-scale validation of the prediction models on a large, geographically diverse population.
Methods We analyzed data collected from 206 870 patients 3 years or older who presented with a painful throat to a United States national retail health chain from September 1, 2006, to December 1, 2008. Main outcome measures were the proportions of patients testing positive for GAS pharyngitis according to the Centor and McIsaac scores (both scales, 0-4).
Results For patients 15 years or older, 23% (95% CI, 22%-23%) tested positive for GAS, including 7% (95% CI, 7%-8%) of those with a Centor score of 0; 12% (95% CI, 11%-12%) of those with a Centor score of 1; 21% (95% CI, 21%-22%) of those with a Centor score of 2; 38% (95% CI, 38%-39%) of those with a Centor score of 3; and 57% (95% CI, 56%-58%) of those with a Centor score of 4. For patients 3 years or older, 27% (95% CI, 27%-27%) tested positive for GAS, including 8% (95% CI, 8%-9%) of those testing positive with a McIsaac score of 0; 14% (95% CI, 13%-14%) of those with a McIsaac score of 1; 23% (95% CI, 23%-23%) of those with a McIsaac score of 2; 37% (95% CI, 37%-37%) of those with a McIsaac score of 3; and 55% (95% CI, 55%-56%) of those with a McIsaac score of 4. The 95% CIs overlapped between our retail health chain–derived probabilities and the prior reports.
Conclusion Our study validates the Centor and McIsaac scores and more precisely classifies risk of GAS infection among patients presenting with a painful throat to a retail health chain.
Quiz Ref IDGroup A streptococcal (GAS) pharyngitis is the most common cause of bacterial pharyngitis affecting over half a billion people annually worldwide.1 GAS pharyngitis is both the antecedent for invasive streptococcal infections such as necrotizing fasciitis and the postinfectious immunologic complication of rheumatic fever and/or rheumatic heart disease, a leading cause of cardiovascular morbidity and mortality in many developing parts of the world. Physical examination of the posterior oropharynx is an inaccurate method to distinguish GAS from other causes of acute pharyngitis,2 so Snow and others,3 most importantly Quiz Ref IDthe Centers for Disease Control and Prevention (CDC) and the American College of Physicians–American Society of Internal Medicine (ACP-ASIM), endorse applying the 4-point Centor clinical scoring scale4 to classify the risk of GAS and guide management of acute pharyngitis in adults (Table 1). Developed 3 decades ago and based on the evaluation of 286 adults at a single emergency department, the Centor score helps clinicians to distinguish GAS from viral pharyngitis and thereby to appropriately prescribe antibiotics to alleviate symptoms and decrease the rates of acute rheumatic fever, suppurative complications, missed school and work days, and disease transmission.5
The McIsaac score,6,7 derived from 521 patients from a university-affiliated family practice in Toronto, Ontario, Canada, and validated on 621 patients from 49 Ontario communities, adjusts the Centor score for the patient's age. Since younger patients are more likely to have GAS than older patients, Quiz Ref IDthe McIsaac score is calculated by adding 1 point to the Centor score for patients aged 3 to 14 years and subtracting 1 point for those 45 years or older. Because clinical prediction models may perform poorly when applied to new settings, it is important to validate them on different populations and over time.8,9 Furthermore, despite endorsement from CDC and ACP-ASIM, the clinical scores have gained poor traction in clinical practice,10 perhaps in part owing to the perception that the scores were derived from a relatively small sample. Herein, we analyze a geographically diverse population of patients who presented with sore throat to MinuteClinic, a large retail health chain, to perform the largest validation studies of the Centor and McIsaac scores.
We analyzed retrospective data collected from patients tested for GAS pharyngitis when they presented with a painful throat from September 1, 2006, to December 1, 2008, to MinuteClinic, a large, national retail health chain with over 500 sites in 26 states.11-14 From the retail clinic's 581 sites, the data set included 238 656 patient encounters across 25 states. In this setting, physician assistants or nurse practitioners collect standardized historical and physical examination information based on algorithm-driven care. The clinicians enter these codified data in real time, and the information is stored in a common database across all clinic locations. MinuteClinic providers have demonstrated greater than 99% adherence to an established acute pharyngitis protocol, the “Strep Pharyngitis Algorithm,”15 from the Institute for Clinical Systems Improvement.16 According to this algorithm, medical providers collect structured information about patients' relevant signs and symptoms, obtain rapid antigen testing on all patients with pharyngitis (with confirmatory testing used for patients whose rapid test is negative), and treat only those patients with a positive test for GAS. The data set included only patient visits where there was complete information about age, all signs and symptoms included in the Centor and McIsaac scores, and test results. The Children's Hospital Boston committee on clinical investigation approved this database analysis.
We included patient visits if a patient presented with a chief complaint of painful throat and was tested for GAS pharyngitis or if a patient had symptoms of pharyngitis and was tested for GAS pharyngitis. Patient visits were excluded if the patient reported having been treated for GAS within the 1 month prior to the visit. Patients younger than 3 years were excluded because neither the Centor score nor the McIsaac score is intended for use in those patients. For patients with multiple visits during the study period, we included the first visit only. Patients were not excluded if they were pregnant or had comorbid conditions. MinuteClinic practice is to not care for patients with septic appearance but to refer them to emergency department care.
All MinuteClinic locations used the Clinical Laboratory Improvement Amendments–waived QuickVue In-line Strep A test (Quidel Corp). The confirmatory test was a streptococcal DNA probe (74%) or throat culture (26%). Patients were categorized as GAS positive if the finding of either test (rapid or confirmatory) was positive.
Predictor variables and covariates were developed for age, sex, history of fever in previous 24 hours, history of exposure to someone with GAS pharyngitis, presence of cough, duration of pharyngitis symptoms (days), presence of erythematous tonsils, presence of tonsillar exudates, presence of swollen tonsils, presence of swollen anterior cervical lymph nodes, presence of swollen posterior cervical lymph nodes, and presence of rhinorrhea. Streptococcal test results were extracted for each patient.
Data from all patients fulfilling the inclusion and exclusion criteria were used to validate the McIsaac score, and data from all patients 15 years or older were used to validate the Centor score. The Centor score was calculated by summing the following clinical factors: history of fever, presence of tonsillar exudates, presence of swollen anterior cervical lymph nodes, and absence of cough. The McIsaac score was calculated for all patients 3 years or older by adding 1 point to the Centor score for those younger than 15 years and by subtracting 1 point from the Centor score for those 45 years or older.17 McIsaac scores of −1 and 5 were normalized to 0 and 4.7
Two approaches were taken to validate the scores. First, we compared the likelihood of GAS pharyngitis by clinical score in the MinuteClinic patients to the likelihood of GAS pharyngitis by clinical score in the published data. Second, we applied logistic regression to the MinuteClinic data to derive new prediction models, maintaining the same parameter that they be limited to no more than 4 clinical variables. The 4 chosen variables derived from the cohort of patients 15 years or older were then compared with the 4 variables that compose the Centor score.
The percentage of patients 15 years or older in the retail health data who tested positive for GAS by Centor score (0-4) was calculated and compared with the original report by Centor et al4 and with the validation study by Wigton et al.18 The percentage of patients 3 years or older in the retail health data who tested positive for GAS by McIsaac score (0-4) was calculated and compared with the McIsaac studies.6,7 Ninety-five percent CIs were calculated for the proportion of patients testing positive at each score. The 95% CIs around the proportion testing positive by score in the retail health data were compared with the 95% CIs in the Centor4,18 and McIsaac6,7 studies.
Variables included in the Centor and McIsaac scores as well as variables not included in the scores were examined to determine the best predictors of GAS pharyngitis among the MinuteClinic patients. Univariate and multivariate analyses were performed to identify predictors of GAS pharyngitis. Significance of association of categorical variables with GAS pharyngitis was tested by the χ2 test. In the multivariate analysis, candidate predictors were entered into a stepwise logistic regression to identify independent predictors of patients with GAS pharyngitis. P value cutoffs for entry and departure for the multivariate regression models were .25 and .10, respectively. For the purpose of simplicity and usability and to facilitate comparison with the prior studies, the final model was limited to 4 predictor variables and assessed by area under the receiver operator characteristic curve (AUC). Statistical analyses were performed using JMP Pro software, version 9.0.2 (SAS Institute).
Of 238 656 patient visits, 5653 were excluded owing to treatment for GAS within the prior month, and an additional 1399 were excluded because the patient age was younger than 3 years, leaving 231 604 patient visits. For patients with multiple visits, only the first visit was included, leaving 206 870 patients to validate the McIsaac score. Of these, 64 789 visits occurred for patients younger than 15 years (31%), leaving 142 081 visits for the validation of the Centor score (Figure).
Among the 142 081 retail health visits for patients 15 years or older, 23% (95% CI, 22%-23%) tested positive for GAS, compared with17% (95% CI, 14%-23%) in the original Centor et al4 article and 26% (95% CI, 24%-32%) in the validation study of the Centor score.18 Two-thirds of the patients in the retail health data set were female, and the average age was 34 years. Table 2 lists the age, sex, and clinical signs and symptoms of pharyngitis by GAS result for those 3 years or older and for those 15 years or older. In both groups, patients who tested positive for GAS pharyngitis were more likely to present with tonsillar exudates, swollen anterior cervical lymph nodes, tonsillar swelling, history of fever in the previous 24 hours, absence of cough, lack of rhinorrhea, swollen posterior cervical lymph nodes, exposure to GAS, and temperature above 101°F (38.3°C) at the time of presentation.
Table 3 lists the percentage of patients testing positive for GAS by clinical score in the retail health data, compared with the published literature by Centor et al,4 Wigton et al,18 and McIsaac et al.6,7 Patients in the retail health population had GAS positivity rates in an intermediate range between the Centor et al4 and Wigton et al18 reports and were more likely to have GAS pharyngitis than were patients in the McIsaac et al6,7 studies. The 95% CIs around the percentage of patients testing positive in the retail health cohort overlapped with the 95% CIs around the percentages testing positive by score in the Centor et al4 and Wigton et al18 studies. Table 4 lists the risks of GAS pharyngitis according to the number of predictors present and stratified by the patient ages used in the McIsaac classification.
In the multivariate logistic regression model, the same 4 candidate predictors were selected from the retail health data, as in the original Centor et al4 report. Quiz Ref IDPresence of tonsillar exudates conferred the highest odds of having streptococcal infection (3.1 [95% CI, 3.0-3.2]) followed by swollen anterior cervical lymph nodes (2.2 95% CI, 2.1-2.3]), history of fever (1.7, [95% CI, 1.7-1.8]) and absence of cough (1.6, [95% CI, 1.5-1.6]).
The overall performance of the model as applied to the retail health data was evaluated by comparing the AUCs. For patients 15 years or older, applying the Centor score to the retail health data yielded an AUC of 0.72. For patients 3 years or older, applying the McIsaac score to the retail health data achieved an AUC of 0.71.
We evaluated 2 commonly used prediction models to classify risk of GAS pharyngitis among patients presenting with a painful throat. The purpose of a clinical prediction model is to provide clinicians with a practical and applicable tool to improve medical decision-making, the health of individual patients, and the public health. The Centor score is 1 model that is particularly robust; it has withstood 30 years of changes in diagnostic testing, information technology, and population dynamics.19 Our study validated the Centor score in a clinical setting (retail health chain) with a less acutely ill population than is seen in the emergency department setting from which the score was derived. While the Centor score was derived from a relatively small number of patients (n = 286) seen in 1 setting during a single 2-month period, we analyzed data from multiple locations spanning more than 1 calendar year, mitigating the potential impact of seasonality because the data are collected throughout the normal peaks and ebbs of GAS pharyngitis incidence. Logistic regression selected, from among the candidate predictors listed inTable 3, the same 4 predictors that were chosen in the landmark article by Centor et al.4
With data from over 140 000 patients, our analyses provide precise interpretations of risk for each Centor score category that still lie within the 95% CIs of the original by Centor et al,4 which was based on fewer than 300 patients. As our research group20 has shown previously, the recent local incidence of GAS pharyngitis further improves the accuracy of estimating an individual patient's risk of GAS pharyngitis. Troughs and peaks of GAS pharyngitis outbreaks will occur naturally throughout the year, so the retail health data in our analyses collected over more than 1 year average over those variations and should provide more reliable characterization of the score than did the original study by Centor et al,4 which was conducted over only 2 months.
The AUC is a metric widely used to reflect the overall accuracy of a diagnostic test or overall performance of a clinical prediction model. The AUC of the Centor score in the present retail health population (0.72) was lower than that found in the original 1981 study by Centor et al4 (0.78) but the same as that found in the validation study by Wigton et al18 (0.72), arguing for the discriminating validity of this score. Clinical prediction models tend to perform less well in validation studies, but our data are consistent with the model's performance in other validation studies.21 While McIsaac et al6,7 did not report an AUC with their original data, the McIsaac score performed similarly to the others in the present large data set.
The observed proportion of MinuteClinic patients testing positive according to clinical scores fell within the 95% CIs of the Wigton et al18 and McIsaac et al6,7 validation studies (except for McIsaac score 0), supporting the calibration validity of the Centor and McIsaac scores.
Leveraging codified data from retail health clinics where uniform, algorithm-driven care is provided and data are captured in a single electronic medical record, our study demonstrates the strengths of the Centor and McIsaac scores as useful tools in clinical decision making. Though many clinicians in the primary care or emergency medicine setting do not routinely test adult patients who are either very likely or very unlikely to have GAS pharyngitis (ie, those with Centor scores of 0, 1, and 4), because MinuteClinic protocol mandates testing for all patients presenting with a painful throat, a further unique strength of our large validation study is ascertainment of GAS status on all subjects.
Though all clinical and laboratory data were collected prospectively, the analyses were conducted retrospectively. There may be some variability in clinical interpretation of the Centor criteria by the nurse practitioners in the MinuteClinic setting; whether anterior cervical nodes are enlarged, for example, might be more subjective than other criteria such as temperature above 101°F (38.3°C).22Quiz Ref IDFurthermore, data are not available for calculating interobserver or intraobserver reliability.
Though very useful for diagnosing the presence of GAS, retail health data would be unlikely to detect group C Streptococcus and most other bacterial causes of pharyngitis, including Fusobacterium necrophorum, which may cause severe disease especially in adolescents and young adults.23
All patients in the data set were symptomatic with sore throat, so our analyses do not address the important issue of the asymptomatic streptococcal carrier state. Serologic testing was not performed, so symptomatic patients with a positive GAS test finding were assumed to be true positives, not carriers.
Because these data were collected recently, we could not quantify potential changes in antibiotic uses attributable to the 2002 Infectious Diseases Society of America guideline24 and to the 2001 American College of Physicians guideline.25
Using national-scale and uniform data electronically captured from a retail clinic chain, we have validated the Centor and McIsaac scores as useful and valid tools for diagnosis and treatment of patients with acute pharyngitis.
Correspondence: Andrew M. Fine, MD, MPH, Division of Emergency Medicine–Main 1, Children's Hospital Boston, 300 Longwood Ave, Boston, MA 02115 (firstname.lastname@example.org).
Accepted for Publication: February 17, 2012.
Published Online: May 7, 2012. doi:10.1001/archinternmed.2012.950
Author Contributions:Study concept and design: Fine, Nizet, and Mandl. Acquisition of data: Fine and Mandl. Analysis and interpretation of data: Fine, Nizet, and Mandl. Drafting of the manuscript: Fine, Nizet, and Mandl. Critical revision of the manuscript for important intellectual content: Fine, Nizet, and Mandl. Statistical analysis: Fine. Obtained funding: Fine and Mandl. Administrative, technical, and material support: Fine. Study supervision: Fine and Mandl.
Financial Disclosure: None reported.
Funding/Support: This study was funded by CDC Mentored Public Health Research Scientist Development Award K01HK000055 (Dr Fine); National Library of Medicine, National Institutes of Health grants R01 LM007677 and G08LM009778 (Dr Mandl); and CDC Public Health Informatics Center of Excellence Award P01HK000088 (Dr Mandl).
Additional Contributions: CVS/Caremark and MinuteClinic provided the data used in this study. However, they had no role in the study design, data analysis, data interpretation, manuscript drafting or revision, or the decision to submit for publication.