Association Between Laboratory Calibration of a Serum Bilirubin Assay, Neonatal Bilirubin Levels, and Phototherapy Use

Importance  The American Academy of Pediatrics treatment recommendations for neonatal jaundice are based on age-specific total serum bilirubin (TSB) levels. In May 2012, Ortho Clinical Diagnostics adjusted the calibrator values for Vitros Chemistry Products BuBc Slides (Ortho Clinical Diagnostics), a widely used method to quantify TSB, after concerns of positively biased results.

Objective  To investigate the association between recalibration of a reflectance spectrophotometry serum bilirubin assay and TSB levels and phototherapy use among newborns.

Design, Setting, and Participants  Descriptive study comparing TSB levels and phototherapy use before and after recalibration at Kaiser Permanente Northern California, a large, integrated health care delivery system. The study evaluated live births at or after 35 weeks’ gestation at 12 facilities that used universal serum bilirubin screening before (January 1, 2010, through April 30, 2012; n = 61 677) and after (July 1, 2012, through December 31, 2013; n = 42 571) recalibration. The analysis took place in December 2015.

Intervention  Recalibration of bilirubin testing instruments.

Main Outcomes and Measures  Proportions of newborns with (1) at least 1 TSB value at or above 15 mg/dL; (2) at least 1 TSB level exceeding the American Academy of Pediatrics phototherapy threshold; (3) phototherapy during the birth hospitalization; and (4) at least 1 readmission for phototherapy.

Results  In 104 420 infants (61 677 in the prerecalibration period and 42 511 in the postrecalibration period), a TSB was obtained in 99.2% of infants during birth and in 99.5% of infants within the first 30 days after birth. The postrecalibration period was associated with a 1.25 mg/dL (95% CI, 1.19-1.31; P < .001) decrease in mean maximum TSB, which led to a 39% relative reduction (from 20.4% to 12.4%) in infants with a TSB level of 15 mg/dL or more and a 51% relative reduction (from 9.3% to 4.5%) in infants with a TSB level that was at or above the American Academy of Pediatrics phototherapy threshold. Phototherapy during birth hospitalizations was reduced by 59% (absolute risk reduction, 5.5%; 95% CI, 4.7%-6.1%) and readmissions for phototherapy by 53% (absolute risk reduction, 1.8%; 95% CI, 1.4%-2.3%).

Conclusions and Relevance  Modest recalibration-induced reductions in mean TSB concentrations was associated with a significant reduction in the percentage of infants with clinically significant hyperbilirubinemia. Current laboratory accuracy standards are insufficient to detect biases that can have significant clinical effect. These data underline the need for increased integration of laboratory expertise into clinical guidelines and to support international initiatives to standardize laboratory measurements.

The American Academy of Pediatrics (AAP) treatment recommendations for neonatal jaundice are based on age-specific total serum bilirubin (TSB) levels.¹ In the 1960s, variability in TSB concentrations across methods led to the formation of a joint committee between the AAP, the College of American Pathologists, the American Association for Clinical Chemistry, and the National Institutes of Health to develop recommendations for a uniform bilirubin standard.^2-5 A standard reference material, 916 bilirubin, was developed by the National Institute of Standards and Technology in the 1970s, and a reference method for the determination of TSB was published in 1985.^6-8

The Vitros BuBc Neonatal Bilirubin assay (Ortho Clinical Diagnostics) is one of only a few commercially available reflectance spectrophotometry assays for measuring serum bilirubin in the clinical laboratory.^9,10 The instrument provides 2 measured and 1 calculated result per sample: unconjugated bilirubin, conjugated bilirubin, and the sum of the 1 (unconjugated bilirubin + conjugated bilirubin = TSB). On May 21, 2012, Ortho Clinical Diagnostics notified customers that it had “received customer complaints of positively biased results using Vitros … BuBc Slides for CAP [College of American Pathologists] proficiency testing.” Based on customer input and an internal investigation, Ortho Clinical Diagnostics responded by adjusting the calibrator values for Vitros BuBc Slides. The company predicted an average reduction in unconjugated bilirubin values of 0.1 to 2.16 mg/dL (to convert to micromoles per liter, multiply by 17.104), depending on the concentration range of the unconjugated bilirubin.

The objective of this study was to investigate the clinical effect of this recalibration on maximum TSB levels and phototherapy use in Kaiser Permanente Northern California (KPNC), a large, integrated health care delivery system.

Box Section Ref ID

Quiz Ref IDThis was a descriptive study of all live births at or after 35 weeks’ gestation between January 2010 and December 2013 at 12 KPNC facilities that used universal bilirubin screening with a TSB prior to discharge.¹¹ The analysis took place in December 2015.

From existing KPNC databases, we obtained all TSB values from each infant’s first month after birth by using previously described methods.^11,12 We used each infant's maximum TSB value and also compared each TSB value with the 2004 AAP phototherapy guideline, based on hour-specific TSB value, gestational age, and direct antiglobulin test result^13,14 and used each infant’s highest TSB value in relation to the guideline. The KPNC institutional review board and the University of California, San Francisco Committee on Human Research approved this study. Informed consent was waived because this study had no direct patient contact and minimal risk to the patients.

Quiz Ref IDPhototherapy was defined by a birth hospitalization or readmission with either an International Classification of Diseases, Ninth Revision code for phototherapy (99.83) and an order for phototherapy in the electronic medical record or an electronic medical record flowsheet entry for phototherapy. Total serum bilirubin (unconjugated bilirubin + conjugated bilirubin) was quantified using either the Vitros Fusion 5.1 or Vitros 250 according to manufacturer instructions using the BuBc slide. Testing was performed across 21 hospital laboratories using 42 individual instruments. The hospital laboratories implemented the manufacturer-adjusted calibrators between May 24, 2012, and June 13, 2012. We defined the prerecalibration cohort as births from January 1, 2010, through April 30, 2012, and the postrecalibration cohort as births from July 1, 2012, through December 31, 2013. One facility transitioned to a different method of measuring TSB on November 19, 2013. Thus, we excluded infants born at this facility after the change. Quiz Ref IDThe outcomes compared between the cohorts were (1) the percentage of infants with at least 1 TSB value above 15 mg/dL; (2) the percentage of infants with at least 1 TSB level at or above the AAP phototherapy threshold; (3) the percentage of infants receiving phototherapy during the birth hospitalization; and (4) the percentage of infants readmitted for phototherapy.

We compared basic demographics between the 2 cohorts using the χ² test and mean maximum TSB values using the t test. We compared outcomes between the 2 cohorts using an autoregressive integrated moving average time-series model with Stata version 14 (StataCorp). The autoregressive integrated moving average command fits linear regression models that can accommodate correlated error terms. We fit separate models for each of the dependent variables (monthly percentage of TSB levels ≥15 mg/dL, a TSB level ≥AAP phototherapy threshold, phototherapy during the birth hospitalization, and readmission for phototherapy) with an independent variable of prerecalibration vs postrecalibration. In sensitivity analyses, we explored the effect of increasing the autoregressive lag in the autoregressive integrated moving average model up to order 4 and the effect of including a secular trend modeled using restricted cubic splines. Additionally, we investigated including May 2012 and June 2012 and setting the recalibration date as June 1, 2012.

The cohort consisted of 104 428 infants born at or after 35 weeks’ gestation: 61 677 in the prerecalibration period and 42 571 in the postrecalibration period. Infants in the 2 periods were similar in terms of birth weight and percentage born before 37 weeks’ gestation. In the postrecalibration cohort, there were more Asian infants and infants of “other” race/ethnicity. In addition, a lower percentage of infants who were tested were direct antiglobulin test–positive in the postrecalibration cohort (Table 1). A TSB was obtained in 99.2% of infants during birth hospitalization and in 99.5% of infants within the first 30 days after birth. The mean (SD) maximum TSB was 10.1 (4.9) mg/dL in the prerecalibration period and 8.8 (4.5) mg/dL in the postrecalibration period (absolute difference, 1.25 mg/dL; 95% CI, 1.19-1.31; P < .001).

Monthly rates of phototherapy administration and hyperbilirubinemia are shown in the Figure. Hyperbilirubinemia defined by either a maximum TSB level of 15 mg/dL or higher or 1 TSB value at or above AAP phototherapy thresholds dropped dramatically after recalibration in June 2012. Similarly, both the monthly rates of phototherapy administration during the birth hospitalization and readmissions for phototherapy dropped precipitously in June 2012 and remained lower. Quiz Ref IDTable 2 shows the TSB values and rates of phototherapy before and after recalibration. With recalibration, phototherapy use during the birth hospitalization and readmissions for phototherapy dropped by more than 50%.

In the autoregressive integrated moving average models, the postrecalibration period was associated with an 8.0% (95% CI, 7.1%-8.8%) absolute reduction in infants with a TSB level at or above 15 mg/dL, a 4.8% (95% CI, 4.1%-5.4%) absolute reduction in infants with a TSB level at or above the AAP phototherapy threshold, a 5.5% (95% CI, 5.1%-6.0%) absolute reduction in infants receiving phototherapy during the birth hospitalization, and a 2.0% (95% CI, 1.7%-2.3%) absolute reduction in phototherapy readmissions. Sensitivity analyses did not give appreciably different results.

Adjusting for delivery mode, the postrecalibration period was associated with a small but significant reduction in overall length of stay by 0.08 days during the birth hospitalization (95% CI, 0.05-0.1 days). However, in both periods, phototherapy during the birth hospitalization increased length of stay by more than 2 days (prerecalibration, 2.2 days; 95% CI, 2.1-2.3 days vs postrecalibration, 2.5 days; 95% CI, 2.4-2.7 days). Phototherapy readmission lengths of stay were approximately 1 day (prerecalibration, 1.30 days; 95% CI, 1.26-1.33 days vs postrecalibration, 1.17 days; 95% CI, 1.11-1.24 days). Quiz Ref IDThe number of TSB tests sent per patient dropped from 2.65 tests/infant to 2.20 tests/infant, a difference of 0.45 tests/infant (95% CI, 0.43-0.48).

We observed a large-scale clinical effect of the Vitros BuBc Neonatal Bilirubin assay recalibration. Ortho Clinical Diagnostics predicted an average drop in unconjugated bilirubin values of approximately 0.1 to 2.16 mg/dL. Our data confirmed that the mean maximum TSB concentration decreased by 1.25 mg/dL. We hypothesized that the effect of this decrease on phototherapy rates would be clinically significant because many infants have TSB concentrations near the AAP thresholds for treatment. In the prerecalibration period, 7.3% of infants had 1 or more TSB levels of 0 to 2 mg/dL above the AAP phototherapy threshold. Indeed, the use of phototherapy in birth hospitalizations dropped 59% and readmissions for phototherapy dropped 53%. These data demonstrate that seemingly modest systematic reductions in TSB concentrations resulted in a major reduction in the percentage of infants with clinically significant hyperbilirubinemia. The infants in the 2 times differed slightly, with more Asian infants and infants of “other” race/ethnicity in the postrecalibration cohort and a lower percentage of infants with a positive direct antiglobulin test result. Because these race/ethnicity categories were risk factors for hyperbilirubinemia, this would only serve to attenuate our results. The difference in DAT positivity percentage was small.

Recalibration led to a significant reduction in use. We estimate for every 10 000 deliveries, recalibration resulted in a reduction of 1300 patient-days/year (1100 days during the birth hospitalization and 200 phototherapy readmission days) and 4500 fewer TSB tests/year. Although costs are specific to each hospital, estimating an inpatient hospitalization day at $1000 and the cost of a TSB test and blood draw at $30, recalibration saved $1.4 million for every 10 000 deliveries. Thus, from January 2010 to April 2012, the excess cost for KPNC was more than $8 million. At the time of recalibration, approximately 380 laboratories across the United States were enrolled in College of American Pathologists proficiency testing for neonatal total bilirubin using the Vitros assay; our hospital laboratories represented about 6% of those enrolled. Thus, the overall health care resources lost to analytical inaccuracy far exceed what was calculated for our health care system. Beyond cost, phototherapy often means separating the mother and infant. This can interfere with breastfeeding and bonding and cause parental anxiety.^15-18

Calibration shifts in laboratory assays can have major clinical implications. While theoretical models can predict these effects, it is rare to capture the consequential extent of the changes. This study provides a powerful example of how small changes in measurement methods can lead to large changes in diagnosis and treatment. These data highlight 2 main limitations. First, when there is significant interinstrument variability, guidelines should specify the analytical assays that were used to determine treatment thresholds. Second, current laboratory accuracy standards (proficiency testing and quality control monitoring) are insufficient to detect biases that can have significant clinical effect. The positive bias imposed by the Vitros BuBc assay was ultimately identified by comparing neonatal bilirubin proficiency testing peer group means with those of other instrument/assays, but the problem was undetected and under investigation for at least 2 years (Figure) before the recalibration was implemented.

The data from the Vitros BuBc assay recalibration underline the need for increased integration of laboratory expertise into clinical guidelines and to support international initiatives to standardize laboratory measurements.¹⁹

Corresponding Author: Michael W. Kuzniewicz, MD, MPH, Division of Research, Kaiser Permanente Northern California, 2000 Broadway Ave, Oakland, CA 94612 (michael.w.kuzniewicz@kp.org).

Accepted for Publication: December 22, 2015.

Published Online: April 11, 2016. doi:10.1001/jamapediatrics.2015.4944.

Author Contributions: Dr Kuzniewicz had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Kuzniewicz, Newman.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Kuzniewicz.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Kuzniewicz, McCulloch, Newman.

Obtained funding: Kuzniewicz, Newman.

Administrative, technical, or material support: Kuzneiwicz, Greene, Walsh.

Study supervision: Kuzniewicz.

Conflict of Interest Disclosures: None reported.

Funding/Support: This project was supported by grant R01HS020618 from the Agency for Healthcare Research and Quality.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.