Copyright 1998 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.1998
To assess the proficiency of commercial laboratories in analyzing lead in clinical blood samples from subjects without overt lead exposure.
We submitted masked duplicate blood lead specimens to 8 masked laboratories. Each laboratory received blood aliquots immediately following drawing (time 1) and 2 weeks later (time 2) from 7 human subjects and 3 bovine blood samples with known lead levels of 0.26, 0.57, and 0.79 µmol/L (5.4, 11.8, and 16.4 µg/dL). Of the 8 laboratories, 5 were commercial laboratories, 1 was a state laboratory, 1 was a research laboratory, and 1 was the Centers for Disease Control and Prevention reference laboratory.
Correlation coefficients were calculated, and differences within and between laboratories were assessed by analysis of variance.
Results were obtained for all specimens, with all the human subjects' overall mean lead levels being less than 0.48 µmol/L (<10 µg/dL). Each laboratory reported all human blood specimens appropriately, as having lead levels less than 0.48 µmol/L (<10 µg/dL) and within 0.14 µmol/L (3 µg/dL) of the overall mean for that subject. All internal reproducibilities were very high (range, 0.92-1.00) except for one (0.60), possibly lower because of 1 pair of specimens. Mean differences between blood samples analyzed at time 1 and time 2 ranged from −1.4 to 1.2, with only 2 laboratories having significant differences (P<.01).
Overall, there was strong reproducibility within and among laboratories, with no overall time trend or interlaboratory or intralaboratory variance. The storage conditions did not seem to affect the aggregate results. The data suggest that through implementation of the Centers for Disease Control and Prevention/Wisconsin Blood Lead Proficiency Testing Program, the Centers for Disease Control and Prevention's Blood Lead Laboratory Reference System, and mandated federal and state proficiency programs, laboratories in this geographic region have improved their performance as compared with previous published studies and an unpublished study.
BECAUSE OF the subclinical symptoms of chronic lead exposure, diagnosis has traditionally relied heavily on laboratory criteria (blood lead levels, protoporphyrin levels, and, more recently, bone lead levels), rather than observable symptoms. Blood lead level is generally accepted as the most valid biomarker of human exposure to lead.1 In 1965, the "normal" blood lead range for children in the United States was 0.72-1.21 µmol/L (15-25 µg/dL)2; whereas today it is generally considered that even levels of 0.48 to 0.72 µmol/L (10-15 µg/dL) can produce developmental delays.3 A population-based study4 has shown that blood lead levels have decreased over time as the use of leaded gasoline and lead-based paint has decreased. Also, the demand for blood lead testing has increased since the Centers for Disease Control and Prevention (CDC)5 recommended, in 1991, that children living in communities with high risks for lead exposure should have repeated blood lead determinations every 6 months until 2 or more consecutive blood lead values are less than 0.48 µmol/L (<10 µg/dL). Consequently, laboratories are now working closer to their limits of detection, and their ability to consistently reproduce valid results at levels less than 0.48 µmol/L (<10 µg/dL) is under scrutiny. Several standardization and proficiency programs have been implemented to control such error; the most noteworthy of these is the CDC/Wisconsin Blood Lead Proficiency Testing Program (currently known as the Federal/Wisconsin Blood Lead Proficiency Testing Program), a nonregulatory program that tests participating commercial laboratories periodically by submitting marked samples of known lead concentration for analysis.6 The analysis of blood lead specimens is regulated by the Clinical Laboratory Improvement Amendments Act of 1988,7 which states that acceptable error is within 0.19 µmol/L (4 µg/dL) when blood lead values are less than 19.3 µmol/L (<40 µg/dL). Although these marked samples are intended to be received without extra attention and care, they are easily identified by supervisors and technicians. To our knowledge, the only prior blinded study of laboratory proficiency in blood lead determination was conducted by Sargent et al8 in 1996. They reported that there was wide disparity in laboratory performance, with some laboratories meeting or exceeding the CDC/Wisconsin Blood Lead Proficiency Testing Program guidelines and others failing to do so. They also believed that further testing to determine if "order-of-magnitude errors"8 occurred frequently enough to warrant the use of blinded quality control or quality assurance samples at routine intervals was necessary. The present study provides a second assessment of routine laboratory performance on blood lead determination and compares interlaboratory and intralaboratory precision by assessing known and unknown lead levels.
Vacutainer blood collection tubes (Becton Dickinson Labware, Franklin Lakes, NJ), which contain liquid EDTA anticoagulant, and 23-gauge butterfly blood-collection systems were provided by CDC's National Center for Environmental Health. All materials were prescreened for lead content, which was not substantial (<0.003 µmol/L [<0.07 µg/dL] for the collection tubes). One milliliter of an aqueous mixture of 1% vol/vol nitric acid (HNO3) and 0.001% vol/vol Triton X-100 (Aldrich Chemical Inc, Milwaukee, Wis), an emulsifier, was drawn through the butterfly blood-collection systems (capillary collection) into a clean syringe, then dispensed and measured for lead content. The collection tubes were screened with the addition of 1.0 mL of an aqueous mixture of 1% vol/vol nitric acid (HNO3) and 0.001% vol/vol Triton X-100 solvent, followed by soaking for approximately 12 to 14 hours. Half of the tubes were inverted to maximize contact with the stoppers (or caps). The lead level of the solvent was measured after contact with all the containers to be used in the blood collection. The total amount of lead from all of these sources was added and evaluated so that the total amount of lead contamination that would be caused by the collection devices was less than 5% of the expected blood lead mean value (0.24 µmol/L [5 µg/dL]) of the patient samples. All laboratories, except laboratory 7, used the EDTA-containing tubes with lavender tops. Laboratory 7 only conducts analyses on certified trace metal–free tubes with sodium heparin anticoagulant (tan topped). To maintain anonymity, we complied with this requirement. Butterfly blood-collection systems were used to facilitate the drawing of multiple samples with a minimum number of venipunctures.
The National Center for Environmental Health, through the CDC Blood Lead Laboratory Reference System, provided a set of in vivo dosed bovine blood samples at 3 target values to be used as quality control specimens. The target values of 0.26, 0.57, and 0.79 µmol/L (5.4, 11.8, and 16.4 µg/dL) were determined using inductively coupled plasma–isotope-dilution mass spectroscopy.9 The bovine blood samples were collected with 1.5 mg of the anticoagulant Na2 EDTA per milliliter of whole blood. The collection tubes (lavender or tan topped) were washed with 18-megaohm deionized water to remove the EDTA that was normally placed in the tubes during tube production, thus eliminating the problems that twice the amount of EDTA had with specific analytical methods. The bovine blood samples from the CDC Blood Lead Laboratory Reference System were then placed into the tubes, and the tops replaced. All of the manipulations of cleaning, drying, and filling were handled in a class-100 clean room (<100 particles/m3) to eliminate possible contamination of the samples. All reference samples were refrigerated before and during shipment by overnight express carrier.
All human blood samples were drawn in the first quarter of 1995 at the Clinical Center for Environmental and Occupational Health at the Environmental and Occupational Health Sciences Institute in Piscataway, NJ. After receiving informed consent, blood to fill sixteen 2-mL vials were drawn from each of 7 healthy adult volunteers with no history of occupational exposure to lead. The subject pool consisted of the 3 bovine blood samples, labeled as human blood, along with the samples from 7 human subjects. Eight laboratories were used: 5 large commercial laboratories, 1 state laboratory, 1 research laboratory, and the CDC reference laboratory. All 5 commercial laboratories were large national laboratories operating in 1995. All 5 processed large numbers of specimens and were selected because of their involvement in blood lead level determination. Each laboratory was sent 2 specimens from each subject, 1 immediately following the drawing of the blood (time 1) and 1 specimen 2 weeks later (time 2), for a total of 80 per batch. Each laboratory analyzed 20 samples, 10 per batch. The specimens were assigned fictitious labels to facilitate blinding. Throughout the 2 weeks before the submission of the second batch, the specimens were stored in polyethylene vials at 4°C, a temperature shown to provide stable storage conditions for whole blood.10- 12 Specimens were either picked up by laboratory personnel or shipped overnight in insulated packages.
All laboratories used graphite-furnace atomic absorption spectroscopy for analysis of the specimens. The CDC reference laboratory performed all analyses as previously described.13,14 Unknown blood samples, bovine quality control materials established by inductively coupled plasma–isotope-dilution mass spectroscopy (4 levels, different from the 3 target values of bovine blood used as subjects H-J), and aqueous standards were diluted 10-fold (1:10) with a matrix modifier, placed into clean autosampler cups, and analyzed. The matrix modifier was prepared with 0.2% vol/vol high-purity nitric acid, 0.5% vol/vol Triton X-100, and 0.2% wt/vol ammonium phosphate diluted with ultrapure 18-megaohm water. Stock lead standards for instrument calibration were prepared using National Institute of Standards and Technology Standard Reference Material 3128. Quality control materials, including 4 levels of bovine blood quality control materials, were analyzed before and after all samples to verify the accuracy of each analytical run. An atomic absorption spectrophotometer (model 4100ZL, Perkin-Elmer Corp, Norwalk, Conn) with Zeeman-effect background correction was used for all CDC analyses of study blood specimens.
Blood samples were matched by patient code, batch number, and laboratory identification number. Descriptive statistics were generated, and correlation analyses and analyses of variance were performed. Laboratory 1 and laboratory 2 had minimum detection levels of 1.0 mg/dL. Those values that were reported as nondetectable were assigned a value of 0.02 µmol/L (0.5 µg/dL). The minimum detection level for laboratory 6 was 3.0 µg/dL, and, therefore, all values reported from laboratory 6 as nondetectable were assigned a value of 0.07 µmol/L (1.5 µg/dL). None of the other laboratories reported levels below detection limits. Data are reported as mean ± SD.
Table 1 presents the within-laboratory reproducibility, which is a measure of the correlation between matched pairs of time 1 and time 2 samples. Figure 1 shows this graphically with reported values at time 1 (micromoles per liter) plotted against those at time 2 (micromoles per liter). All laboratories have Pearson correlation coefficients (r) between 0.92 and 1.00 that were significant at P<.001, except laboratory 2, which had an r of 0.60 (P=.07). Laboratory 4 and laboratory 8, the reference laboratory at the CDC, ranked highest in reproducibility at r=1.00 (P<.001). Laboratory 2 performed poorly on the samples from subject J in both batches (Table 2 and Figure 2), accounting for the low Pearson coefficient and suggested excessive variability within that laboratory. Mean values obtained for the 10 specimens analyzed at time 1 and at time 2 are shown separately for each laboratory in Table 2. At time 1, the mean blood lead level of all subjects by laboratory ranged from 0.15 to 0.30 µmol/L (3.2-6.2 µg/dL). Mean levels at time 2 ranged from 0.13 to 0.25 µmol/L (2.7-5.2 µg/dL). The mean difference (time 1 − time 2) ranged from −1.4 to 1.2 (Table 1). Only laboratories 4 and 7 showed a significant negative time trend between the second and first specimen batches (P=.001 and P =.003, respectively). All other laboratories did well, as the overall reproducibility is acceptable. Subject mean blood lead values per laboratory are shown in Table 2.
Reported blood lead values for all laboratories at time 1 and time 2. To convert lead values from micromoles per liter to micrograms per deciliter, divide by 0.0483.
Reported blood lead values by laboratory for subject J at time 1 and time 2. Dotted line represents the blood lead value (0.79 µmol/L) determined by inductively coupled plasma–isotope-dilution mass spectroscopy. This was used as the target value. (To convert lead values from micromoles per liter to micrograms per deciliter, divide by 0.0483.)
Analysis of variance showed that there was a borderline significant difference among laboratories (P=.06) when comparing the reported blood lead values from each laboratory. Further analysis showed that there was a significant difference between 2 laboratories (P<.05), most likely caused by a single outlier. Because of the small sample size, this outlier has considerable effect on the aggregate result. Overall laboratory mean levels ranged from 0.32 to 0.54 µmol/L (6.6-11.3 µg/dL), with all but 1 mean level between 0.48 and 0.54 µmol/L (10.0 and 11.3 µg/dL).
Table 2 shows the mean result for the bovine subjects (subjects H, I, and J) for each laboratory. Each laboratory analyzed 2 samples from subjects H, I, and J. Although the target value for subjects H, I, and J were known (0.26, 0.57, and 0.79 µmol/L [5.4, 11.8, and 16.4 µg/dL], respectively), many of the laboratories underreported values at both time 1 and time 2 (Figure 2, Figure 3, and Figure 4). Laboratory 2 reported the blood lead level for subject J as 0.15 µmol/L (3.1 µg/dL) for time 2. The overall laboratory mean measurements were 0.2 ± 0.04 µmol/L (5.0 ± 0.8 µg/dL) for subject H, 0.5 ± 0.06 µmol/L (11.0 ± 1.2 µg/dL) for subject I, and 0.68 ± 0.14 µmol/L (14.0 ± 3.0 µg/dL) for subject J. Although all mean blood lead levels, except that obtained by laboratory 2 for subject J, are within the 0.2-µmol/L (4-µg/dL) deviation required by the Clinical Laboratory Improvement Amendments Act of 1988 proficiency testing standard for blood lead levels of less than 1.93 µmol/L (<40 µg/dL), it is not clear why these laboratory means are consistently less than the given target values.
Reported blood lead values by laboratory for subject H at time 1 and time 2. Dotted line represents the blood lead value (0.26 µmol/L) determined by inductively coupled plasma–isotope-dilution mass spectroscopy. This was used as the target value. (To convert lead values from micromoles per liter to micrograms per deciliter, divide by 0.0483.)
Reported blood lead values by laboratory for subject I at time 1 and time 2. Dotted line represents the blood lead value (0.57 µmol/L) determined by inductively coupled plasma–isotope-dilution mass spectroscopy. This was used as the target value. (To convert lead values from micromoles per liter to micrograms per deciliter, divide by 0.0483.)
We examined the ability of each laboratory to correctly classify the individual specimens as greater or less than the CDC action level of 0.48 µmol/L (10 µg/dL). To calculate sensitivity and specificity, it is necessary to have a criterion of "truth" for each specimen. We used the target values for the bovine specimens and assumed, on the basis of our results and the absence of known exposures, that all volunteers had true values less than 0.48 µmol/L (<10 µg/dL) (Table 2). Since the target values of 2 bovine blood specimens, subjects I and J, are greater than 0.48 µmol/L (>10 µg/dL), as determined by inductively coupled plasma–isotope-dilution mass spectroscopy, laboratory sensitivity for values greater than or equal to 10 µg/dL was 87.5% (28/32 samples). Specificity for values less than 0.48 µmol/L (<10 µg/dL) was 100% (128/128 samples), although all values less than 0.48 µmol/L (<10 µg/dL) were well below 10 µg/dL (Table 2). The highest human mean blood level was 0.2 µmol/L (4.4 µg/dL) at time 1 and 0.2 µmol/L (4.1 µg/dL) at time 2, levels that are well below the action threshold of 0.48 µmol/L (10 µg/dL). Laboratory 2 showed excessive variability and produced substantially lower results than the other laboratories.
Correlations between values at time 1 and time 2 were high in 7 of the 8 laboratories. Only 2 of 8 laboratories had significant differences between a subject's reported blood lead values at time 1 and time 2 (Table 1). Because the laboratories were unaware that these samples were part of a proficiency study and blinding was believed to have been maintained, the results suggest adequate accuracy and precision in the analysis of blood specimens at low blood lead levels. Only the performance of laboratory 2 was unsatisfactory. It consistently reported values less than the overall means for subjects A through G and underestimated the blood lead levels in subjects H through J. No other laboratory had such a substantial underestimation. Although laboratories 4 and 7 had significant differences between reported blood lead values at time 1 and time 2, these differences were not strong enough to affect the overall time trend. Only a larger sample size with more samples per laboratory would permit individual investigation.
The variance found between laboratories seems to be driven by 1 laboratory with a markedly different overall mean level for all specimens. That laboratory's overall mean blood lead level seems to be driven by 1 low outlier. Since the sample size is not large enough to minimize the effects of such outliers, the borderline significance of the variance between laboratories must be considered carefully. Another indicator of adequate accuracy and precision was that all laboratory results were within 4 µg/dL of the overall mean for each subject at each time. Although Figures 2 through 4 showed substantial differences between reported blood lead values at time 1 and time 2 for subjects H through J, only 1 pair had values more than 4 µg/dL greater and 0.2 µmol/L (4 µg/dL) less than the mean of the pair, and only 4 of 48 reported blood lead values (time 1 or time 2) were more than 0.2 µmol/L (>4 µg/dL) greater or less than the target value. Three of these unacceptable results were on samples from subject J, 2 of them from laboratory 2. Figure 1 showed that subjects A, D, and F had somewhat higher mean reported blood lead values across laboratories than the other human subjects. It also showed that the mean subject values from all laboratories, excluding subject J from laboratory 2, were within 0.2 µmol/L (4 µg/dL), which limited interlaboratory variance.
Several previous studies15- 19 of laboratory proficiency in blood lead determination found that performance was unsatisfactory. In an unpublished study done by our group at the Environmental and Occupational Health Sciences Institute in 1990, laboratories did not perform satisfactorily in analyzing samples with blood lead levels lower than 0.48 µmol/L (<10 µg/dL). There were substantial differences between replicate samples, whether submitted 1 week apart or on the same day. There was obvious disparity between individual laboratories (P<.01). Although some laboratories fared better than others, none performed satisfactorily, and we consequently arranged to have blood lead determinations, for a randomized trial of the effect of home cleaning, done at the CDC. Published studies, with results such as these, coupled with the decrease in children's blood lead levels, lead the CDC to promote proficiency testing programs and initiate the CDC Blood Lead Laboratory Reference System.
Sargent et al8 concluded that there was wide variation in the ability of laboratories to meet the criteria set forth by the CDC/Wisconsin Blood Lead Proficiency Testing Program, even in 1995 after proficiency programs were in place. The authors believed that further study of laboratory performance on clinical specimens was required to determine if such samples should be sent periodically to test laboratories. This study used such clinical specimens, along with bovine blood samples with known lead levels, with all those involved in the analysis blinded as to the purpose of the blood samples and found that clinical laboratories have improved their performance on the analysis of lead in blood specimens, but still have room to improve. Sargent et al8 reported that 11% of specimens with a known lead level of 0.3 µmol/L (6.9 µg/dL) were misclassified as having a level of more than 0.48 µmol/L (>10 µg/dL), and 42% of specimens with a known lead level of 0.43 µmol/L (9.0 µg/dL) were misclassified. They also reported that 11% of the specimens with levels known to be 0.89 µmol/L (18.4 µg/dL) were misclassified as having levels less than 0.48 µmol/L (<10 µg/dL), and 6% of specimens with known lead levels of 1.59 µmol/L (32.9 µg/dL) were misclassified as having levels less than 0.48 µmol/L (<10 µg/dL). Our study compared favorably, in that all laboratories reported lead levels for subjects A through G as less than 10 µg/dL, and 0% of subject H specimens, known to be 5.4 µg/dL, were misclassified. Also, only 19% of subject I specimens, with a known lead value of 0.57 µmol/L (11.8 µg/dL), were misclassified, and 6% of subject J specimens, with a known value of 0.79 µmol/L (16.4 µg/dL), were misclassified.
An initial problem in those early studies of laboratory proficiency was the absence of a definitive method that could be used to assign certified values to reference materials. In this study, the use of a definitive method (inductively coupled plasma–isotope-dilution mass spectroscopy) to assign target values to the bovine quality control materials eliminated this problem. Consequently, we were able to assess the accuracy of laboratory performance by using the bovine blood specimens as standards in the interpretation of human blood lead results. Although it is not known why 6 of the 8 laboratories had means less than the target value both at time 1 and time 2, several causes seem possible, including human error and the inability of the analytical method used (graphite-furnace atomic absorption spectroscopy) to accurately reproduce the lead values generated by the inductively coupled plasma–isotope-dilution mass spectroscopy. Laboratory 2 substantially underreported the blood lead levels of all specimens, both at time 1 and time 2. Further study with sufficient sample size is needed to more fully explore the problems with the analyses of lead in blood specimens by laboratory 2. Overall, however, the laboratories in this study performed well in relation to what we potentially expected judging from our unpublished study and a published study.8
Other studies10,12,20 have addressed the issue of storage time and its effects on lead determinations in human and bovine blood. These studies recommended the use of original collection tubes, a storage temperature of 4°C, and EDTA or heparin anticoagulant. Our results support these recommendations since we show no overall time trend, even after 2 weeks of such storage. Although there were individual time trends in 2 of the 8 laboratories, they were not large enough to affect the overall time trend.
A limitation to this study is the small sample size. Because only 160 samples were analyzed, we cannot decisively comment on the current status of blood lead testing in individual commercial laboratories; we can, however, suggest that overall the laboratories are performing well in analyzing samples with these low blood lead levels and seem to have improved their ability to provide more accurate and consistent results over time (since the unpublished assessment by our group in 1990). Although the commercial firms used were large national companies, the analyses could have been performed at regional laboratories. This geographic limitation implies the need for caution in generalizing the results to the nation. This limitation, as well as the inconsistent results found for laboratory 2, suggest the need for a more comprehensive study with a greater number of samples per laboratory to assess regional differences and individual laboratory performance, rather than aggregate performance.
Accepted for publication December 29, 1997.
This work was funded by grant 18152 from the Robert Wood Johnson Foundation, Princeton, NJ; cooperative agreement CR 820235 from the US Environmental Protection Agency, Washington, DC; and interagency agreement from the National Institute of Child Health and Human Development, Bethesda, Md; and by center grant 2P30ES05022 from the National Institute of Environmental Health Sciences, Research Triangle Park, NC.
Editor's Note: What a nice study this is! In some matters, it helps to have Big Sister watching.—Catherine D. DeAngelis, MD
Reprints: George G. Rhoads, MD, MPH, Environmental Health Division, Environmental and Occupational Health Sciences Institute, Room 234A, 681 Frelinghuysen Rd, Piscataway, NJ 08855-1179.
Jobanputra NK, Jones R, Buckler G, Cody RP, Gochfeld M, Matte TM, Rich DQ, Rhoads GG. Accuracy and Reproducibility of Blood Lead Testing in Commercial Laboratories. Arch Pediatr Adolesc Med. 1998;152(6):548-553. doi:10.1001/archpedi.152.6.548