Clinical manifestations in patients with positive findings for antinuclear antibodies (positive-ANA patients) and the subset treated with corticosteroids. A, Percentage of 118 patients in the positive-ANA diagnostic criteria meeting 1, 2, or 3 of the American College of Rheumatology (ACR) criteria for systemic lupus erythematosus (SLE). B, Clinical manifestations of positive-ANA patients meeting 3 of the ACR criteria for SLE. Positive serologic findings include anti–double-stranded DNA, anti-Sm, or antiphospholipid antibodies. Other manifestations are as defined in the 1982 ACR criteria.1,10 C, Prednisone dosages of the 39 positive-ANA individuals treated with corticosteroids. D, Number of ACR criteria fulfilled by the 39 positive-ANA individuals treated with corticosteroids. Fourteen patients had a positive ANA test as the sole manifestation of SLE. E, Clinical manifestations of the 39 positive-ANA individuals treated with corticosteroids. CNS indicates central nervous system.
Narain S, Richards HB, Satoh M, Sarmiento M, Davidson R, Shuster J, Sobel E, Hahn P, Reeves WH. Diagnostic Accuracy for Lupus and Other Systemic Autoimmune Diseases in the Community Setting. Arch Intern Med. 2004;164(22):2435-2441. doi:10.1001/archinte.164.22.2435
Copyright 2004 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2004
Most individuals with autoimmune and other immune disorders undergo initial evaluation in the community setting. Since misdiagnosis of systemic autoimmune diseases can have serious consequences, we evaluated community physicians’ accuracy in diagnosing autoimmune diseases and the consequences of misdiagnosis.
We studied the patients referred to our Autoimmune Disease Center for 13 months (n = 476). We estimated the degree of agreement with the final diagnosis (κ statistic) and the accuracy indexes (sensitivity, specificity, and predictive values) of the referring physicians’ diagnoses.
We found a 49% agreement between the referring and final diagnoses (κ = 0.36). Of 263 patients referred with a presumptive diagnosis of systemic lupus erythematosus (SLE), 125 received a diagnosis of other conditions (κ = 0.34). Of those referred with SLE, 76 (29%) were seropositive for antinuclear antibodies but did not have autoimmune disease. The degree of agreement for referring rheumatologists (κ = 0.55) was better than that for nonrheumatologists (κ = 0.32). Stepwise logistic regression indicated that rheumatologists were 4 times more likely to make an accurate diagnosis of SLE than were nonrheumatologists (P<.003). Thirty-nine patients who were seropositive for antinuclear antibodies but had no autoimmune disease had been treated with corticosteroids at dosages as high as 60 mg/d.
Many patients with a positive antinuclear antibody test are incorrectly given a diagnosis of SLE and sometimes treated with toxic medications. The data support the importance of continuing medical education for community physicians in screening for autoimmune diseases and identifying patients who may benefit from early referral to a specialist.
The diagnoses of systemic lupus erythematosus (SLE), progressive systemic sclerosis, and Sjögren syndrome are aided by well-established clinical criteria.1- 5 Since many patients with these potentially serious diseases undergo evaluation and treatment initially by community physicians, it is important that they be recognized so that affected individuals can be referred to specialists in a timely manner. Misdiagnosis can have a serious negative impact on health as well as economic and emotional consequences. The risks of overdiagnosis include inappropriate treatment with potentially dangerous medications, unnecessary referrals6,7 and laboratory tests,8 and difficulty obtaining health or life insurance. Conversely, underdiagnosis can lead to a delay in appropriate therapy, culminating in irreversible complications such as renal failure or pulmonary fibrosis. Because an accurate and timely diagnosis can reduce health care costs, the need to train primary care physicians in rheumatological diagnosis has been increasingly recognized.9
The objective of this study was to evaluate the accuracy of community physicians in diagnosing autoimmune disorders. We also examined the factors associated with a misdiagnosis of these conditions and the question of how frequently an inaccurate diagnosis led to inappropriate therapy.
Four hundred seventy-six patients referred between September 1, 2001, and September 30, 2002, to the University of Florida Autoimmune Disease Center (ADC), Gainesville, consented to participate in this institutional review board–approved study. The referring physician was asked to complete a prescreening questionnaire indicating the reasons for referral and a working diagnosis.
Date of birth, ethnicity, education, referring diagnosis, and final diagnosis were obtained from medical records and data collected at the ADC. Median annual household income was used as an indicator of socioeconomic status. Block group information was obtained using the home ZIP code and matched with income data provided in the Inforum software (Thomson Medstat, Franklin, Tenn). The referring physician’s specialty and year of graduation from specialty training were obtained from the American Medical Association Web site (available at: http://www.ama-assn.org/aps/amahg.htm; last accessed August 31, 2002). Referring physicians were classified as rheumatologists or nonrheumatologists.
Subjects were interviewed and examined by 1 of 4 autoimmune disease subspecialists (H.B.R., E.S., P.H., or W.H.R.). Diagnostic criteria were reviewed after the evaluation by 2 subspecialists (H.B.R. and W.H.R.). Systemic autoimmune diseases were classified using validated criteria for SLE,1,10 progressive systemic sclerosis,2 Sjögren syndrome,3 rheumatoid arthritis,4 and polymyositis/dermatomyositis.5
Diagnoses were categorized as follows: (1) SLE, (2) progressive systemic sclerosis, (3) Sjögren syndrome, (4) polymyositis/dermatomyositis, (5) positive for antinuclear antibodies (hereafter referred to as positive ANA; ANA titer ≥1:40 using Hep-2 substrate [The Binding Site Limited, Birmingham, England] in the absence of autoimmune disease), (6) fibromyalgia, and (7) other, including undifferentiated connective tissue disease (≥2 clinical manifestations suggestive of connective tissue disease and the presence of ≥1 non–organ-specific autoantibody),11 Raynaud phenomenon, Wegener granulomatosis, antiphospholipid antibody syndrome, interstitial lung disease, or sarcoidosis.
All patients underwent extensive autoantibody testing on their first visit. Antinuclear antibodies were detected by means of indirect immunofluorescence using Hep-2 cell substrate. Anti–double-stranded (ds) DNA antibodies were detected using the Crithidia luciliae kinetoplast-staining assay (DiaSorin, Stillwater, Minn). Antiphospholipid antibodies were detected using a commercial IgG and IgM anticardiolipin antibody enzyme-linked immunosorbent assay (The Binding Site Limited) and/or lupus anticoagulant activity. Anti-Sm antibodies were detected by means of enzyme-linked immunosorbent assay (Varelisa; Pharmacia Diagnostics, Uppsala, Sweden), and their presence was confirmed by means of immunoprecipitation.12 Anti–ribosomal P autoantibodies were detected by means of immunoprecipitation of the P0, P1, and P2 proteins.12
Agreement between the referring and final diagnoses was analyzed using the κ statistic, which compares the observed with the expected number of agreements when the rater randomly places the cases, constrained only by the number of diagnoses in each category. The observed-to-expected difference is normalized so that 0.0 represents pure guesswork and 1.0 represents complete agreement. Referring physicians’ diagnoses were compared with final ADC diagnoses that were used as the gold standard and were based on formal diagnostic criteria. Prevalence of an autoimmune disease was calculated as the proportion of patients with a final diagnosis of that condition divided by the total number of patients undergoing analysis.
Sensitivity, specificity, and positive and negative predictive values were calculated as indicators of the ability of the referring physician to diagnose autoimmune disorders. Sensitivity was defined as the probability that the referring physician correctly diagnosed an autoimmune disease. Specificity was the probability that the referring physician was able to exclude the presence of a particular autoimmune disease in the population found not to have that disease by the subspecialists at the ADC. Positive predictive value was the probability that the referring diagnosis was confirmed.
Stepwise forward logistic regression for patients with a referring diagnosis of SLE was conducted, with age, years of education, type of insurance, median annual household income, physician’s specialty, and physician’s year of graduation as in dependent variables. A diagnosis of SLE by the referring physician was scored as 1 (SLE final at ADC) or 0 (not SLE final diagnosis). Similarly, 12 or fewer years of patients’ education was scored as 0, and greater than 12 years was scored as 1; median annual household income of $40 000 or less was scored as 0 and greater than $40 000 as 1. Type of insurance was scored as 1 for privately insured and 0 for those on Medicare/Medicaid; referring physician was scored as 1 if referred by a rheumatologist and 0 if referred by a nonrheumatologist. Based on the results of the regression procedure, we calculated the odds ratio (OR) estimates of correct diagnosis and 95% confidence intervals (CI). A P value of less than.05 was accepted as significant.
Demographic characteristics of the cohort are shown in the following tabulation.
Mean ± SD age of the cohort was 44.5 ± 14.0, and the ethnicity of 1 patient was unknown. Most patients were referred with the diagnosis of SLE (56%) (Table 1). Other referring diagnoses included progressive systemic sclerosis (8%), Sjögren syndrome (4%), polymyositis/dermatomyositis (3%), fibromyalgia (1%), positive ANA (7%), and other nonrheumatological diagnoses (12%). Forty-seven patients with no referring diagnosis and 8 patients without a final diagnosis were excluded from further analysis.
Agreement between the referring physician’s diagnosis and the final ADC diagnosis was evaluated using the κ statistic. Interrater agreement for all diagnoses was 0.36 (95% CI, 0.30-0.42); 49% of the referring diagnoses matched the final diagnoses, and 129 patients received an incorrect diagnosis of SLE. The κ statistic for SLE was 0.35 (95% CI, 0.27-0.42), suggesting only a fair agreement between raters. Many of the patients receiving an incorrect diagnosis of SLE received a final diagnosis of positive ANA (76/263 with SLE as a referring diagnosis, κ = 0.20).
For all diagnoses, agreement was fair for nonrheumatologists (κ = 0.32; 95% CI, 0.25-0.39) and better for rheumatologists (κ = 0.55; 95% CI, 0.39-0.71). Rheumatologists referred 45 patients with SLE as a primary diagnosis, and 33 were given a final diagnosis of SLE (κ = 0.58; 95% CI, 0.37-0.77). Their diagnostic accuracy was 80% and specificity was 60%. For nonrheumatologists, the diagnosis of SLE was confirmed for only 102 of 192 patients (κ = 0.31; 95% CI, 0.21-0.41) (Table 2).
We calculated the sensitivity, specificity, accuracy, and positive and negative predictive values for the referring physicians’ diagnoses (Table 3). Diagnostic sensitivity was high for SLE (88%). High specificity was seen for all diagnoses except SLE (52%), and was explained by clustering of most of the patients in the true negative cell, as 63% of the referrals were diagnosed as SLE and approximately 13% had diagnoses other than autoimmune disorders. The numbers in other disease categories were too small for meaningful analysis.
For patients with a referring diagnosis of SLE, stepwise forward logistic regression was used to evaluate the association between misdiagnosis and the patient’s education and median annual household income and the referring physician’s specialty and year of graduation. Rheumatologists were more likely to make an accurate diagnosis of SLE than nonrheumatologists (OR, 3.9; 95% CI, 1.6-9.4; P<.003). No other variable achieved a significant P value, including education (OR, 1.05; 95% CI, 0.48-2.27; P = .91) and median annual household income (OR, 1.12; 95% CI, 0.68-1.84; P = .66). Univariately, those insured by private policies were more likely to be correctly diagnosed as SLE (OR, 1.85; 95% CI, 1.09-3.12; P = .02). However, after adjustment for the referring physician as rheumatologist or not, the variable did not achieve an OR significantly different from 1.0 at P<.05 (P = .10).
Because some diagnostic criteria are more specific for SLE than others,13 and to exclude the possibility that the positive-ANA patients may have early SLE, we compared the clinical features of patients with SLE (≥4 criteria) vs the 3-criteria positive-ANA group, which consisted of 16% of the total positive-ANA subset (Figure, A and B). The 3-criteria positive-ANA patients had a low prevalence of lupus-specific criteria such as malar rash, photosensitivity, renal disease, and central nervous system involvement (Figure, B). Overall, in the positive-ANA subset, major organ involvement (renal, central nervous system, or hematologic) was rare. In contrast, end-organ damage was apparent in 88 (67%) of 131 patients given a final diagnosis of SLE. The most frequent manifestations in the positive-ANA subset were serological, usually antiphospholipid antibodies (11 of 16 patients). Arthritis, a relatively nonspecific manifestation, was the most common clinical manifestation in this subset.
Certain autoantibodies are highly specific for SLE,14 including anti-dsDNA, anti-Sm, and anti–ribosomal P.12 They may appear years before the onset of SLE,15 thus providing a second approach for evaluating whether the positive-ANA group might have early lupus. Anti-dsDNA antibodies (detected by any technique at any time) were found in 2 of the 70 positive-ANA (6%) and 1 of the 42 other serum samples (2%), whereas anti-Sm antibodies were found in only 1 patient with a positive-ANA serum sample and not at all in the other samples. When anti-dsDNA antibody (Crithidia assay), anti-Sm, and anti–ribosomal P (immunoprecipitation) findings were reassessed at the time of referral, 28% of patients with SLE were seropositive for anti-dsDNA (Crithidia)vs 3% in the positive-ANA and 2% in the other groups. Anti-Sm was detected in 14% of serum samples from patients with SLE and not at all in the other 2 groups. Similarly, anti–ribosomal P was detected in 2% of serum samples from patients with SLE but not in the other groups. Thus, lupus-specific autoantibodies generally were absent in the positive-ANA and other subsets, suggesting that these patients differed serologically as well as clinically from the patients with SLE. Twenty-six percent of the positive-ANA patients had antiphospholipid autoantibodies. Antiphospholipid antibodies also were detected in 23% of the serum samples from patients in the other category.
It was of particular interest to determine whether the misdiagnosis of systemic autoimmune diseases, especially lupus, has health care implications. We studied this by evaluating the prescription of inappropriate medications. Misdiagnoses were made in 203 patients (46% of those referred by physicians in private, solo, or group-based practice and 39% of those referred by university-based physicians). Nearly 67% of those with a misdiagnosis were given a final diagnosis of being positive ANA (no evidence of systemic autoimmune disease) or of having fibromyalgia. Of the 68 patients with a misdiagnosis who were treated with steroids, 39 did not have an autoimmune disease (final diagnosis of positive ANA or fibromyalgia). Their prednisone dosages are shown in Figure, C; 25 of 32 received moderate- to high-dosage prednisone (16-60 mg/d) for their presumed autoimmune condition. The SLE criteria they fulfilled are shown in Figure, D and E. Only 6 met 3 criteria, and major organ involvement was rare.
The objective of this study was to assess the accuracy of diagnosing autoimmune disorders in the community and the effect of misdiagnosis on medical care. Previous studies suggested a reluctance on the part of primary care physicians to propose a tentative diagnosis,6,7,16 an interpretation supported by the present study in which 47 patients were excluded because the referring physician did not provide a tentative diagnosis. Final diagnoses were arrived at by consensus among a group of specialists using accepted diagnostic criteria.17,18 As all patients underwent complete reevaluation, bias introduced by knowing the referring physicians’ diagnoses was minimized. The low rate of agreement between the referring and final diagnosis supports that conclusion. Our results indicate that autoimmune diseases, especially SLE, are overdiagnosed by community physicians. The most frequent final diagnosis for those patients whose referring and final diagnosis differed was positive ANA. It is unlikely that these positive-ANA individuals had early SLE not yet meeting the diagnostic criteria, because they did not exhibit clinical manifestations regarded as highly specific for SLE19 (Figure, B). Overall, 137 patients received an incorrect diagnosis of SLE (positive predictive value, 51%). Most of these patients had a positive ANA finding as the sole criterion for SLE. The serological test results that were negative for disease-specific autoantibodies further supported the notion that most did not have early SLE. Autoimmune disease, which was rare among the positive-ANA subjects, frequently develops in individuals with anti-dsDNA or anti-Sm antibodies.15,20,21 In contrast, a positive ANA finding in the absence of physical signs and symptoms has limited diagnostic utility.22 Depending on age, 3% to 13% of healthy, asymptomatic people are seropositive for ANA.23- 25 Many other conditions are associated with a positive ANA test, including viral or bacterial infections, autoimmune thyroid disease, and medication use.23,26 There is little evidence that lupus eventually develops in these individuals.
Of 476 patients, 203 received a misdiagnosis from the referring physicians, of the 203, 137 ultimately received a diagnosis of positive ANA or fibromyalgia. Of the 137 patients, 39 were receiving prednisone at dosages as high as 60 mg/d. Although disease manifestations might have been suppressed by corticosteroid therapy, we do not believe this was the case for several reasons. First, these individuals did not meet lupus criteria, and only 3% of the positive-ANA patients had major end-organ damage vs 66% of the patients with SLE. Second, they did not produce disease-specific autoantibodies. Finally, during follow-up for a 3-year period after prednisone therapy was discontinued, no clinical or serological manifestations of SLE developed in these patients. It is likely, therefore, that these 39 individuals were treated inappropriately. The risks of long-term corticosteroid use include infection, osteoporosis, diabetes, hypertension, cataracts, and osteonecrosis.27 Thus, in addition to possible emotional and financial (eg, inability to obtain health insurance) consequences, misdiagnosis of SLE may lead to inappropriate use of potentially risky medications.
Although this is the first study to address diagnostic accuracy in a large lupus referral cohort, similar observations have been made in general rheumatology populations. In a Canadian study, there was poor agreement between primary care physicians and the consulting rheumatologist for a variety of common rheumatological diagnoses.6 That patient population, however, differed greatly from ours in the number and type of referrals. The problem of misdiagnosis is not limited to rheumatological conditions, as illustrated by the frequent misdiagnosis of multiple sclerosis28 and chronic obstructive pulmonary disease.29
The referring physician’s specialty was the only factor that significantly affected the rate of misdiagnosis, suggesting that improving physicians’ training in diagnosing rheumatological conditions might be beneficial. Sociodemographic factors (ie, age, education, and income) did not affect diagnostic accuracy, suggesting that the misdiagnosis of autoimmune disease does not reflect lack of access to adequate medical care. This is especially germane in view of the fact that lupus afflicts minorities more frequently than white patients.30
Although referring rheumatologists were more likely to make correct diagnoses, their accuracy was still relatively low. However, our results may have been biased by the fact that rheumatologists tend to refer only those patients who pose diagnostic dilemmas. Another unexpected finding was that 18 (26%) of the patients referred with a diagnosis of SLE and an ultimate diagnosis of positive ANA had antiphospholipid antibodies, which were added to the diagnostic criteria in 1997.10 Various studies suggest a high prevalence (approximately 30%) of antiphospholipid antibodies in SLE.31,32 However, primary antiphospholipid autoantibody syndrome (in the absence of lupus) is a well-known entity.33,34 The positive-ANA patients with antiphospholipid antibodies will need to be followed up longitudinally to see whether autoimmune disease ultimately develops.20
This study has certain limitations. Because our group has been interested in SLE for many years, there was a bias toward that referring diagnosis. Also, our findings might have been strengthened by a blinded review of the medical records. We can estimate the overdiagnosis rate, but not the underdiagnosis rate. The key issue of whether accurate diagnosis makes a difference in the long-term outcome or cost of medical care could not be assessed in detail. Nevertheless, the data suggest that overdiagnosis may lead to unnecessary and potentially dangerous treatment.
It is likely that in some instances the working diagnosis was other than what was reported to us. For instance, Calvo-Alen et al35 described a subset of patients referred with a diagnosis of possible SLE who actually had fibromyalgia-like symptoms and were seropositive for ANA. We tried to assess the impact of such misrepresentation by recording the therapy provided to the patients. We believe that the decision to use potentially dangerous drugs such as corticosteroids provides a measure of the veracity of the referring physician’s working diagnosis. The fact that significant numbers of patients with misdiagnoses were treated with corticosteroids suggests that, in most cases, the working diagnosis was not misrepresented. Nevertheless, there are likely to be some instances where the working diagnosis was other than what was reported to us. Our study design does not permit us to comment on whether primary care physicians or rheumatologists were more likely to misrepresent the diagnosis.
Systemic autoimmune diseases (SLE in particular) are overdiagnosed by community physicians, sometimes leading to inappropriate therapy. There may be emotional and financial consequences as well. Further studies to determine the health, emotional, and financial consequences of erroneous diagnosis of SLE seem warranted. A large number of instances in which a referring diagnosis of SLE rested entirely on the presence of a positive ANA test in the absence of clinical manifestations underscores the potential benefit of continuing medical education. Increased awareness of the clinical presentation of these diseases may help increase diagnostic accuracy. There also may be a role for diagnostic pathways or algorithms to aid in determining who should be referred to a specialist.
Correspondence: Sonali Narain, MD, MPH, Division of Rheumatology & Clinical Immunology, University of Florida, PO Box 100221, Gainesville, FL 32610-0221 (firstname.lastname@example.org).
Accepted for Publication: April 26, 2004.
Financial Disclosure: None.
Funding/Support: This study was supported by research grants AR40391 and M01 R00082 from the US Public Health Service, National Institutes of Health, Bethesda, Md, and by state funds allocated to the University of Florida Center for Autoimmune Diseases.
Acknowledgment: We thank Minna Honkanen-Scott and Krista Behney for technical assistance; Douglas Theriaque, MS (University of Florida General Clinical Research Center), for computer support; Sylvia McLeran for secretarial assistance; Jennifer Myles for providing the Inforum data; and the nursing staff of the University of Florida General Clinical Research Center.