[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure.  Receiver Operating Characteristic Curves Identifying Low Concordance Based on Historical Concordance
Receiver Operating Characteristic Curves Identifying Low Concordance Based on Historical Concordance

Receiver operating characteristic curves based on proportion of episodes of early magnetic resonance imaging scans over 2 years to identify primary care clinicians (A) and primary care sites (B) subsequently determined to be among the 10% least concordant with guidelines for the use of magentic resonance imaging in management of nonspecific low back pain.

Table 1.  Characteristics of Clinicians and Episodes of Uncomplicated Nonspecific Low Back Pain
Characteristics of Clinicians and Episodes of Uncomplicated Nonspecific Low Back Pain
Table 2.  Episodes of Low Back Pain With an Early MRI Scana
Episodes of Low Back Pain With an Early MRI Scana
Table 3.  Multivariable Logistic Regression of Factors Associated With an Early MRI Scana
Multivariable Logistic Regression of Factors Associated With an Early MRI Scana
Table 4.  Early MRI Scans for Low Back Pain Identified by Historical Concordancea
Early MRI Scans for Low Back Pain Identified by Historical Concordancea
1.
Hart  LG, Deyo  RA, Cherkin  DC.  Physician office visits for low back pain: frequency, clinical evaluation, and treatment patterns from a US national survey.   Spine (Phila Pa 1976). 1995;20(1):11-19. doi:10.1097/00007632-199501000-00003 PubMedGoogle ScholarCrossref
2.
Martin  BI, Deyo  RA, Mirza  SK,  et al.  Expenditures and health status among adults with back and neck problems.   JAMA. 2008;299(6):656-664. doi:10.1001/jama.299.6.656 PubMedGoogle ScholarCrossref
3.
Davis  MA, Onega  T, Weeks  WB, Lurie  JD.  Where the United States spends its spine dollars: expenditures on different ambulatory services for the management of back and neck conditions.   Spine (Phila Pa 1976). 2012;37(19):1693-1701. doi:10.1097/BRS.0b013e3182541f45PubMedGoogle ScholarCrossref
4.
Dagenais  S, Galloway  EK, Roffey  DM.  A systematic review of diagnostic imaging use for low back pain in the United States.   Spine J. 2014;14(6):1036-1048. doi:10.1016/j.spinee.2013.10.031 PubMedGoogle ScholarCrossref
5.
Di Iorio  D, Henley  E, Doughty  A.  A survey of primary care physician practice patterns and adherence to acute low back problem guidelines.   Arch Fam Med. 2000;9(10):1015-1021. doi:10.1001/archfami.9.10.1015 PubMedGoogle ScholarCrossref
6.
Ivanova  JI, Birnbaum  HG, Schiller  M, Kantor  E, Johnstone  BM, Swindle  RW.  Real-world practice patterns, health-care utilization, and costs in patients with low back pain: the long road to guideline-concordant care.   Spine J. 2011;11(7):622-632. doi:10.1016/j.spinee.2011.03.017 PubMedGoogle ScholarCrossref
7.
Chou  R, Qaseem  A, Snow  V,  et al; Clinical Efficacy Assessment Subcommittee of the American College of Physicians; American College of Physicians; American Pain Society Low Back Pain Guidelines Panel.  Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society.   Ann Intern Med. 2007;147(7):478-491. doi:10.7326/0003-4819-147-7-200710020-00006 PubMedGoogle ScholarCrossref
8.
Lehnert  BE, Bree  RL.  Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support?   J Am Coll Radiol. 2010;7(3):192-197. doi:10.1016/j.jacr.2009.11.010 PubMedGoogle ScholarCrossref
9.
Picano  E.  Sustainability of medical imaging.   BMJ. 2004;328(7439):578-580. doi:10.1136/bmj.328.7439.578 PubMedGoogle ScholarCrossref
10.
Rao  JK, Kroenke  K, Mihaliak  KA, Eckert  GJ, Weinberger  M.  Can guidelines impact the ordering of magnetic resonance imaging studies by primary care providers for low back pain?   Am J Manag Care. 2002;8(1):27-35.PubMedGoogle Scholar
11.
Emery  DJ, Shojania  KG, Forster  AJ, Mojaverian  N, Feasby  TE.  Overuse of magnetic resonance imaging.   JAMA Intern Med. 2013;173(9):823-825. doi:10.1001/jamainternmed.2013.3804 PubMedGoogle ScholarCrossref
12.
Fritz  JM, Brennan  GP, Hunter  SJ.  Physical therapy or advanced imaging as first management strategy following a new consultation for low back pain in primary care: associations with future health care utilization and charges.   Health Serv Res. 2015;50(6):1927-1940. doi:10.1111/1475-6773.12301 PubMedGoogle ScholarCrossref
13.
Chou  R, Deyo  RA, Jarvik  JG.  Appropriate use of lumbar imaging for evaluation of low back pain.   Radiol Clin North Am. 2012;50(4):569-585. doi:10.1016/j.rcl.2012.04.005 PubMedGoogle ScholarCrossref
14.
Chou  R, Fu  R, Carrino  JA, Deyo  RA.  Imaging strategies for low-back pain: systematic review and meta-analysis.   Lancet. 2009;373(9662):463-472. doi:10.1016/S0140-6736(09)60172-0 PubMedGoogle ScholarCrossref
15.
Lurie  JD, Birkmeyer  NJ, Weinstein  JN.  Rates of advanced spinal imaging and spine surgery.   Spine (Phila Pa 1976). 2003;28(6):616-620. doi:10.1097/01.BRS.0000049927.37696.DC PubMedGoogle Scholar
16.
Webster  BS, Cifuentes  M.  Relationship of early magnetic resonance imaging for work-related acute low back pain with disability and medical utilization outcomes.   J Occup Environ Med. 2010;52(9):900-907. doi:10.1097/JOM.0b013e3181ef7e53 PubMedGoogle ScholarCrossref
17.
Ip  IK, Lacson  R, Hentel  K,  et al.  JOURNAL CLUB: predictors of provider response to clinical decision support: lessons learned from the Medicare Imaging Demonstration.   AJR Am J Roentgenol. 2017;208(2):351-357. doi:10.2214/AJR.16.16373 PubMedGoogle ScholarCrossref
18.
US Department of Health and Human Services.  Medicare Program; Revisions to Payment Policies Under the Physician Fee Schedule and Other Revisions to Part B for CY 2019; Medicare Shared Savings Program Requirements; Quality Payment Program; and Medicaid Promoting Interoperability Program.   Fed Regist. 2018;83(145):35704-36368.Google Scholar
19.
Isaac  T, Weissman  JS, Davis  RB,  et al.  Overrides of medication alerts in ambulatory care.   Arch Intern Med. 2009;169(3):305-311. doi:10.1001/archinternmed.2008.551 PubMedGoogle ScholarCrossref
20.
Ash  JS, Sittig  DF, Dykstra  R, Campbell  E, Guappone  K.  The unintended consequences of computerized provider order entry: findings from a mixed methods exploration.   Int J Med Inform. 2009;78(suppl 1):S69-S76. doi:10.1016/j.ijmedinf.2008.07.015 PubMedGoogle ScholarCrossref
21.
The Lewin Group.  MRI Lumbar Spine for Low Back Pain (OP-8) Comprehensive Reevaluation Report. Yale-New Haven Health Services Corporation/Center for Outcomes Research and Evaluation; 2016.
22.
Gold  R, Esterberg  E, Hollombe  C,  et al.  Low back imaging when not indicated: a descriptive cross-system analysis.   Perm J. 2016;20(2):25-33. doi:10.7812/TPP/15-081 PubMedGoogle Scholar
23.
Sinnott  PL, Siroka  AM, Shane  AC, Trafton  JA, Wagner  TH.  Identifying neck and back pain in administrative data: defining the right cohort.   Spine (Phila Pa 1976). 2012;37(10):860-874. doi:10.1097/BRS.0b013e3182376508 PubMedGoogle Scholar
24.
Tan  G, Jensen  MP, Thornby  JI, Rintala  DH, Anderson  KO.  Categorizing pain in patients seen in a veterans health administration hospital: Pain as the fifth vital sign.   Psychol Serv. 2008;5(3):239. doi:10.1037/1541-1559.5.3.239Google Scholar
25.
DeLong  ER, DeLong  DM, Clarke-Pearson  DL.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.   Biometrics. 1988;44(3):837-845. doi:10.2307/2531595 PubMedGoogle Scholar
26.
Hall  BL, Huffman  KM, Hamilton  BH,  et al.  Profiling individual surgeon performance using information from a high-quality clinical registry: opportunities and limitations.   J Am Coll Surg. 2015;221(5):901-913. doi:10.1016/j.jamcollsurg.2015.07.454 PubMedGoogle Scholar
27.
Hashmi  ZG, Dimick  JB, Efron  DT,  et al.  Reliability adjustment: a necessity for trauma center ranking and benchmarking.   J Trauma Acute Care Surg. 2013;75(1):166-172. doi:10.1097/TA.0b013e318298494f PubMedGoogle Scholar
28.
Hess  CN, Rao  SV, McCoy  LA,  et al.  Identification of hospital outliers in bleeding complications after percutaneous coronary intervention.   Circ Cardiovasc Qual Outcomes. 2015;8(1):15-22. doi:10.1161/CIRCOUTCOMES.113.000749 PubMedGoogle Scholar
29.
Krell  RW, Hozain  A, Kao  LS, Dimick  JB.  Reliability of risk-adjusted outcomes for profiling hospital surgical quality.   JAMA Surg. 2014;149(5):467-474. doi:10.1001/jamasurg.2013.4249 PubMedGoogle Scholar
30.
Manda  SO, Gale  CP, Hall  AS, Gilthorpe  MS.  Statistical profiling of hospital performance using acute coronary syndrome mortality.   Cardiovasc J Afr. 2012;23(10):546-551. doi:10.5830/CVJA-2011-064 PubMedGoogle Scholar
31.
Dimick  JB, Staiger  DO, Birkmeyer  JD.  Ranking hospitals on surgical mortality: the importance of reliability adjustment.   Health Serv Res. 2010;45(6 Pt 1):1614-1629. doi:10.1111/j.1475-6773.2010.01158.x PubMedGoogle Scholar
32.
Glance  LG, Dick  A, Osler  TM, Li  Y, Mukamel  DB.  Impact of changing the statistical methodology on hospital and surgeon ranking: the case of the New York State cardiac surgery report card.   Med Care. 2006;44(4):311-319. doi:10.1097/01.mlr.0000204106.64619.2a PubMedGoogle Scholar
33.
Agency for Healthcare Research and Quality. Quality Indicator Empirical Methods. Accessed June 9, 2020. https://www.qualityindicators.ahrq.gov/Downloads/Resources/Publications/2019/Empirical_Methods_2019.pdf
34.
Morris  C.  Parametric empirical bayes inference: theory and applications.   J Am Stat Assoc. 1983;78(381):47-55. doi:10.1080/01621459.1983.10477920Google Scholar
35.
Snijders  T, Bosker  R.  Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modelilng. 2nd ed. Sage Publications; 2012.
36.
Kouri  BE, Parsons  RG, Alpert  HR.  Physician self-referral for diagnostic imaging: review of the empiric literature.   AJR Am J Roentgenol. 2002;179(4):843-850. doi:10.2214/ajr.179.4.1790843 PubMedGoogle Scholar
37.
Goulet  JL, Brandt  C, Crystal  S,  et al.  Agreement between electronic medical record-based and self-administered pain numeric rating scale: clinical and research implications.   Med Care. 2013;51(3):245-250. doi:10.1097/MLR.0b013e318277f1ad PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 706
    Citations 0
    Original Investigation
    Imaging
    July 13, 2020

    Assessment of Primary Care Clinician Concordance With Guidelines for Use of Magnetic Resonance Imaging in Patients With Nonspecific Low Back Pain in the Veterans Affairs Health System

    Author Affiliations
    • 1Veterans Affairs Health Economics Resource Center, VA Palo Alto Health Care System, Menlo Park, California
    • 2Center for Innovation to Implementation, VA Palo Alto Health Care System, Menlo Park, California
    • 3Department of Radiology, University of Washington, Seattle
    • 4Department of Neurological Surgery, University of Washington, Seattle
    • 5Department of Health Services, University of Washington, Seattle
    • 6Department of Clinical Epidemiology and Medical Informatics, Oregon Health & Science University, Portland
    • 7Department of Medicine, Oregon Health & Science University, Portland
    • 8Quantitative Research Unit, Stanford University Medical School, Stanford, California
    JAMA Netw Open. 2020;3(7):e2010343. doi:10.1001/jamanetworkopen.2020.10343
    Key Points español 中文 (chinese)

    Question  What are rates of concordance with guidelines for the use of magnetic resonance imaging in patients with nonspecific low back pain among primary care clinicians in the US Department of Veterans Affairs (VA)?

    Findings  In this cohort study of 1 285 405 primary care visits of 920 547 patients, an early magnetic resonance imaging scan of the lumbar spine was performed in 2.42% of primary care episodes for uncomplicated low back pain in VA primary care clinics.

    Meaning  Results of this study suggest that the use of magnetic resonance imaging in nonspecific low back pain by VA primary care clinicians is lower than rates that have been reported for US patients with commercial insurance.

    Abstract

    Importance  Magnetic responance imaging (MRI) of the lumbar spine that is not concordant with treatment guidelines for low back pain represents an unnecessary cost for US health plans and may be associated with adverse effects. Use of MRI in the US Department of Veterans Affairs (VA) primary care clinics remains unknown.

    Objective  To assess the use of MRI scans during the first 6 weeks (early MRI scans) of episodes of nonspecific low back pain in VA primary care sites and to determine if historical concordance can identify clinicians and sites that are the least concordant with guidelines.

    Design, Setting, and Participants  Retrospective cohort study of electronic health records from 944 VA primary care sites from the 3 years ending in 2016. Data were analyzed between January 2017 and August 2019. Participants were patients with new episodes of nonspecific low back pain and the primary care clinicians responsible for their care.

    Exposures  MRI scans.

    Main Outcomes and Measures  The proportion of early MRI scans at VA primary care clinics was assessed. Clinician concordance with published guidelines over 2 years was used to select clinicians expected to have low concordance in a third year.

    Results  A total of 1 285 405 new episodes of nonspecific low back pain from 920 547 patients (mean [SD] age, 56.7 [15.8] years; 93.6% men) were attributed to 9098 clinicians (mean [SD] age, 52.1 [10.1] years; 55.7% women). An early MRI scan of the lumbar spine was performed in 31 132 of the episodes (2.42%; 95% CI, 2.40%-2.45%). Historical concordance was better than a random draw in selecting the 10% of clinicians who were subsequently the least concordant with published guidelines. For primary care clinicians, the area under the receiver operating characteristic curve was 0.683 (95% CI, 0.658-0.701). For primary care sites, the area was under this curve was 0.8035 (95% CI, 0.754-0.855). The 10% of clinicians with the least historical concordance were responsible for just 19.2% of the early MRI scans performed in the follow-up year.

    Conclusions and Relevance  VA primary care clinics had low rates of use of early MRI scans. A history of low concordance with imaging guidelines was associated with subsequent low concordance but with limited potential to select clinicians most in need of interventions to implement guidelines.

    Introduction

    Low back pain has been reported to be the second most common reason for physician office visits1 and its associated health care costs have been reported to be increasing.2,3 Magnetic resonance imaging (MRI) is used routinely, with 8% to 21% of patients with low back pain in US health plans receiving a scan.4-6 The Joint Clinical Practice Guideline from the American College of Physicians and the American Pain Society suggests that clinicians should not routinely perform MRI imaging during the first 6 weeks of an episode of nonspecific low back pain without indications that serve as a red flag.7 An estimated 26% to 44% of MRI scans of the lumbar spine are not guideline concordant.8-11 Consequences of inappropriate imaging include the direct costs of imaging and further costs of associated interventions, often with similar or worse patient outcomes.12-16

    The goals of this study were to determine if a program to implement guidelines is needed in the Veterans Affairs (VA) health care system and, if so, assess whether particular clinicians or clinics could reliably be identified as having low concordance. We hypothesized that the use of MRI within 6 weeks of a new episode of low back pain (early MRI scan) is at least as great in US Department of Veterans Affairs (VA) primary care clinics as in other parts of the US health care system. We also hypothesized that historical concordance is better than a random draw in selecting clinicians and clinics for a program to improve concordance with guidelines. A selective approach that focuses on the least concordant clinicians could spare the time of those who are concordant with imaging guidelines. The potential lost time is substantial. The Medicare mandate to use computerized decision support for certain advanced imaging orders17 represents an estimated annual cost of $123 million of clinician time.18 A selective approach also has implications for patient safety. Clinically important alerts may be missed when clinicians receive too many unneeded notifications.17,19,20

    Methods

    We evaluated care for low back pain in the VA primary care system in the 3 years ending June 30, 2016, with a retrospective analysis of a national repository of extracts of electronic health records. The Administrative Panels for the Protection of Human Subjects of Stanford University approved the study with exemption from informed consent because of use of publicly available deidentified data. Analysis began in January 2017 and was completed in August 2019. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies.

    Cohort

    A cohort was constructed of episodes of nonspecific low back pain without any initial indication for MRI. Exclusion criteria included an initial diagnosis of specific back pain, a condition that serves as a red flag, or continuation of care for chronic back pain. Episodes began with a primary care visit for nonspecific pain of the lower back: lumbar sprain, strain, spondylosis, or disk degeneration. Visits characterized exclusively by a diagnosis of unspecified backache or dorsalgia were excluded because these conditions do not necessarily involve the lower back.

    Potential index visits were excluded if there was a specific back pain diagnosis: myelopathy, radiculopathy, sciatica, injury to the coccyx, spinal stenosis, or disk herniation, which are possible indications for MRI. Additional visits were excluded if there was an indication that served as a red flag for MRI: trauma in the prior 45 days; lumbar spine surgery in the past 90 days; cancer, neurologic impairment, infections, or injection drug use in the past year; or autoimmune or inflammatory conditions, neoplastic abnormalities, radiation therapy or congenital malformation in the last 5 years.21 Exclusions were based on all care provided or purchased by VA primary care clinics (eFigure in the Supplement).

    A visit was considered to be the start of a new episode if no health services for thoracic, lumbar, sacral, or sacrococcygeal back pain had been received in the prior 6 months. This interval defined a new episode in previous studies.6,22 Patients with prior care for low back pain could initiate a new episode if at least 6 months had elapsed without care for back pain.

    Diagnostic codes were selected from a report that compared published definitions of back pain with the resulting count of VA back pain cases23 and from the Medicare performance measure for imaging.21 Two of us (J.C.J and R.C.) selected codes to exclude episodes in which MRI might have been indicated at the time of the index visit. International Classification of Diseases, Ninth Revision (ICD-9) codes were applied to care provided before October 1, 2015, and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision (ICD-10) codes were applied after October 1, 2015.

    Episodes were attributed to the primary care clinician at the time of the index visit listed in the VA Primary Care Management Model database. When no primary care clinician was assigned, care was attributed to the clinician seen at that visit. When more than 1 clinician was encountered, care was attributed to the physician clinician. If more than 1 physician saw the patient, the episode was excluded.

    An MRI of the lumbar spine was identified in the Radiology Extract of the VA Managerial Cost Accounting system and in the VA purchased care database (Current Procedural Terminology code 72148 or 72158). Scans performed in the first 6 weeks of an episode were deemed early and not concordant with guidelines.8 This measure of concordance used episodes of nonspecific low back pain as the denominator, distinct from Medicare measure of concordance, which used scans of the lumbar spine as the denominator.23

    Patient characteristics included age, sex (reported as gender), opioid medications and tramadol provided by VA in the previous year, and an indicator of care for low back pain in the previous 24 months. The first numeric pain score recorded on the date of the index visit was categorized on a scale from 0 to 10 (0 indicating no pain, 1-3, mild pain, 4-6, moderate pain, and >7, severe pain), groupings that best represent the interference of pain with daily activities.24

    Clinician characteristics included training (physician, nurse practitioner, physician assistant, or resident physician), age, sex, and the number of episodes attributed to the clinician. Continuous variables were categorized to form 3 approximately equal sized subgroups of clinicians. Primary care provided at a single location was referred to as a primary care site and was characterized as either hospital based or community based.

    Statistical Analysis

    We hypothesized that the proportion of episodes with early MRI at VA was at least as great as in other parts of the US health care system. The 95% CI of the VA proportion was found using the standard error of the binomial distribution.

    The second hypothesis was that historical concordance was better than a random draw in selecting clinicians and clinics for a program of de-implementation. Clinician concordance with guidelines in the first 2 study years (2014-2015) was used to select those expected to be among the 10% least concordant in the third year (2016). A nonparametric method was used to define the 95% CI for the area under the receiver operating characteristic curve.25 The lower bound of the 95% CI was compared with a threshold of 0.5, the area under the curve of a noninformative test. To address potential bias from differences in patient case mix, historical concordance was risk adjusted with the results of multivariable logistic regression with patient characteristics as the independent variables. Shrinkage adjustment, often used to evaluate hospital performance,26-32 addressed potential loss of reliability due to small sample size (fewer observations for a clinician) and lack of variance (reduced variation between clinicians).33 This estimator shrinks less reliable estimates toward the population mean.34 The ROC curves were determined for unadjusted, risk adjusted, and shrinkage and risk-adjusted historical concordance. The areas under these curves were compared using the same nonparametric method.25 The practical value of selecting clinicians expected to be the least concordant based on their historical concordance was determined by finding the proportion of all early MRI scans in the follow-up year that were received by their patients.

    Univariate associations between early MRI and clinician and patient characteristic were examined by finding the proportion of episodes with that characteristic that had an early MRI scan. The 95% CI was estimated from the standard error of the binomial. Because there were multiple independent comparisons, differences in proportions were reported as significant in the text only if P < .001.

    Multivariable logistic regression was used to consider the association of each clinician and patient factor with an early MRI scan while controlling for all other factors. The regression included crossed random effects for clinician and site, a specification that accommodated clinicians practicing at more than 1 site. Within-patient random effects could not be modeled because too few patients had more than 1 episode.

    The intraclass correlation coefficient was determined from a logistic regression with clinician and site random effects and no independent variables.35 Explained variance was partitioned to clinician, site, and patient by analysis of a multivariable random-effects logistic regression with patient level independent variables.35 All analyses were performed using SAS version 9.2 (SAS Institute Inc). All statistical tests were conducted with 2-tailed tests.

    Results

    A total of 1 285 405 episodes of nonspecific low back pain from 920 547 patients in the 3 years ending June 30, 1996, was studied. The mean (SD) patient age at the start of the episode was 56.7 (15.8) years; 93.6% of the cohort were men. Responsibility for these episodes was attributed to 9098 clinicians (mean [SD] clinician age, 52.1 [10.1] years; 55.7% women). Clinician and patient characteristics are found in Table 1.

    The 1 285 405 episodes represented 92.7% an original sample of 1 387 983 new episodes of nonspecific back pain in VA primary care clinics (eFigure 1 in the Supplement). The final cohort excluded 67 626 episodes without a pain score from the index visit (4.9%), 31 587 additional episodes that could not attributed to a clinician (2.3%), 1832 additional episodes that could not be assigned to a site (0.13%), and 1533 episodes that were missing a patient identifier.

    An MRI of the lumbar spine was performed during the first 6 weeks in 31 132 of the 1 285 405 low back pain episodes (2.42%; 95% CI, 2.40%-2.45%; P < .001). The hypothesis that an early MRI was performed in at least 4.2% of episodes (the lower bound of the comparable proportion reported outside VA) was rejected.

    Table 2 and the eTable in the Supplement show the proportion of episodes with an early MRI for each clinician and patient characteristic. Early MRI scans were less common if the clinician was at least 60 years of age rather than younger than 50 years (odds ratio [OR], 0.861; 95% CI, 0.805-0.922), was responsible for at least 172 episodes over 3 years rather than less than 57 episodes (OR, 0.769; 95% CI, 0.719-0.824), if the index visit was at a community clinic rather than a hospital-based clinic (OR, 0.848; 95% CI, 0.813-0.884), or if the episode occurred in 2016 rather than 2014 (OR, 0.948; 95% CI, 0.920-0.976).

    Patients were less likely to receive an early scan if they were male rather than female (OR, 0.902; 95% CI, 0.862-0.943) and older than 70 years rather than younger than 40 years (OR, 0.359; 95% CI, 0.342-0.376). Patients were more likely to receive an early scan if they had more severe pain in their index visit rather than no pain (OR, 4.324; 95% CI, 4.167-4.488), if they had no assigned primary care clinician at their index visit compared with those who saw their assigned primary care physician (OR, 1.181; 95% CI, 1.137-1.227), or if they had no visit for back pain in the previous 24 months compared with those who did (OR, 1.233; 95% CI, 1.201-1.265).

    The results of the multivariable logistic regression appear in Table 3. Most clinician and patient characteristics identified in the univariate comparisons remained statistically significant. For example, performance of MRI scans was less common if the clinician was responsible for at least 172 episodes over 3 years (OR, 0.903; 95% CI, 0.846-0.964), if the index visit was at a community clinic rather than a hospital-based clinic (OR, 0.857; 95% CI, 0.764-0.962), or if the episode occurred in 2016 rather than 2014 (OR, 0.947; 95% CI, 0.917-0.977). Scans were more common in patients with severe pain (OR, 4.197; 95% CI, 4.044-4.356). In contrast with the univariate result, female patients were less likely to get an early scan (OR, 0.936; 95% CI, 0.895-0.983) and there was no significant association between an early scan and attribution of the episode to a physician assistant, having a primary care clinician, or seeing that clinician at the start of the episode. Clinician age and gender were excluded from the multivariable analysis because these variables were missing for more than 20% of observations.

    For clinician and site-level associations with an early MRI scan, the intraclass correlation coefficient was 0.114 for primary care clinicians and 0.089 for primary care sites. Analysis of variance found that 10.4% of variance was explained by patient-level factors, 8.9% by primary care clinician, and 8.7% by site, leaving 71.9% of variance unexplained.

    Historical data were used to select clinicians who were expected to be among the 10% least concordant with guidelines in the follow-up period. These least concordant clinicians accounted for 31.5% of the early MRI scans. An early MRI scan was performed in 13.0% of the least concordant episodes, a mean (SD) of 4.6 (4.0) scans per clinician. The ability of historical concordance to correctly select the least concordant clinicians is graphed in a receiver operating characteristic (ROC) curve (Figure, A). The area under this ROC was 0.683 (95% CI, 0.658-0.701; P < .001), which is significantly greater than 0.5, the area that would result from a random draw. The area under the ROC curve with risk and shrinkage adjustment (C = 0.697; P < .001) was significantly greater than the area under the unadjusted curve. The area under the ROC curve with risk adjustment alone (C = 0.678; P = .18) was significantly less than the area under unadjusted curve.

    Historical data were was also used to select the 10% least concordant primary care sites (Figure, B). An early MRI scan was performed in 7.2% of the episodes of these least concordant sites. They accounted for 15.7% of the scans in the follow-up period. Using historical concordance to select the least concordant sites resulted in the area under ROC of 0.8035 (95% CI, 0.754-0.855; P < .001), significantly better than a random draw. There were no significant differences between areas under ROC curves defined without adjustment, with risk adjustment (C = 0.804; P = .97), or with shrinkage and risk adjustment (C = 0.810; P = .48).

    Table 4 presents the results of different decision thresholds along the ROC curve to select the least concordant clinicians. Using historical data to select clinicians expected to be the 10% least concordant accounted for 19.2% of the episodes with early MRI scans in the follow-up period. Using the historical data to select the 10% less concordant sites accounts for 10.3% (1017 of 9883) of the early MRI scans in the follow-up period.

    Data on clinicians and clinics with episodes only in the follow-up period are not given in the Figure and Table 4. Primary care clinicians without historical data accounted for 14.0% of the clinicians and 9.7% of the early MRI scans in the follow-up year. Primary care sites without historical data accounted for 2.2% of the sites and 2.1% of the early MRI scans in the follow-up year.

    Discussion

    Between 8% and 21% of US patients with low back pain receive an MRI of the lumbar spine.4,5 Findings from these studies are not comparable with our study as they included patients for whom MRI may be appropriate: those with chronic pain, a specific case of back pain, or an indication that served as a red flag. The VA cohort was limited to new episodes of nonspecific low back pain without an indication that served as a red flag. We determined that 2.42% of these patients received an MRI within the first 6 weeks of their episode, the recommended period for a trial of conservative therapy without using MRI.7 The finding was significantly less than the 4.2% found in a comparable study of US patients with commercial insurance.6 That study also included a cohort of newly diagnosed patients with nonspecific low back pain without indications that serve as a red flag. An MRI of the lumbar spine was performed in 8.5% of those patients. Although the proportion of episodes with a scan within 6 weeks was not reported, the median time to a scan was 19 days.6 The true proportion of patients who received a scan within 6 weeks appears to be between 4.2% (the proportion scanned after 19 days) and 8.5% (the total proportion scanned).

    The VA primary care clinicians appear to be more concordant with published practice guidelines for imaging low back pain than other US clinicians. Factors in this concordance may include that VA clinicians are salaried; other US clinicians may have an economic incentive to order scans.36 Several VA sites routinely review orders for advanced imaging. In addition, VA may have a lower capacity to conduct MRI.

    Historical concordance over the previous 2 years was better than random selection in identifying clinicians and sites that were subsequently found to be among the 10% least concordant with imaging guidelines, but the practical utility of this measure was low. Selecting 10% of primary care clinicians based on historical concordance accounted for 19.2% of early MRI scans in the follow-up period. The low number of scans in VA limits the reliability of the historical concordance as a selection criterion, even with risk and shrinkage adjustment.

    Limitations

    This study has limitations. Diagnoses were limited to care provided by or paid for by VA. Diagnostic codes assigned in VA are not subject to review by health care payers and may be less accurate than in other systems. Pain scores in electronic medical records underestimate actual patient pain.37 Reliance on this source may have attenuated patient case mix adjustment. We excluded 7.3% of episodes with missing data. Shrinkage adjustment assumes independence of volume and outcomes, but higher volume clinicians were more concordant with guidelines.

    Conclusions

    In this study, we found use of MRI to manage nonspecific low back pain by VA primary care clinicians to be more concordant with guidelines than the practice described in a study of patients6 with commercial insurance. A history of inappropriate imaging by VA clinicians and clinics was associated with subsequent low concordance, but had limited potential to improve the efficiency of efforts to improve clinician concordance with guidelines. Interventions to improve concordance may be unneeded, given the low utilization of early MRI scans for low back pain in VA.

    Back to top
    Article Information

    Accepted for Publication: April 27, 2020.

    Published: July 13, 2020. doi:10.1001/jamanetworkopen.2020.10343

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Barnett PG et al. JAMA Network Open.

    Corresponding Author: Paul G. Barnett, PhD, Veterans Affairs Palo Alto Health Care System, 795 Willow Rd (Mail Code 152), Menlo Park, CA 94025 (paul.barnett@va.gov).

    Author Contributions: Dr Barnett had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Barnett, Jacobs, Nevedal.

    Acquisition, analysis, or interpretation of data: Barnett, Jacobs, Jarvik, Chou, Boothroyd, Lo.

    Drafting of the manuscript: Barnett, Jacobs, Boothroyd, Lo, Nevedal.

    Critical revision of the manuscript for important intellectual content: Barnett, Jacobs, Jarvik, Chou, Boothroyd, Nevedal.

    Statistical analysis: Barnett, Jacobs, Boothroyd, Lo.

    Obtained funding: Barnett, Nevedal.

    Administrative, technical, or material support: Barnett, Nevedal.

    Supervision: Barnett, Jarvik.

    Conflict of Interest Disclosures: Dr Jacobs reported receiving grants from the US Department of Veterans Affairs (VA) Health Administration during the conduct of the study. Dr Jarvik reported receiving grants from VA subcontract during the conduct of the study; royalties as a book co-editor from Springer Publishing; GE-Association of University Radiologists Radiology Research Academic Fellowship; travel reimbursement for Faculty Board of Review; royalties as a chapter author from Wolters Kluwer/UpToDate. Dr Chou reported receiving grants from the VA during the conduct of the study; and funding from the American College of Physicians to develop guidance on low back pain imaging (prior to 3 years ago); funding from the American Pain Society to conduct systematic reviews on low back pain evaluation and treatment, including imaging (prior to 3 years ago); author on American College of Physicians and American Pain Society guideline on low back pain evaluation and management (>3 years ago). No other disclosures were reported.

    Funding/Support: The work was supported by Merit Review Award I01 HX002016 from the VA Health Services Research & Development Service of the VA Office of Research and Development.

    Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the US Department of Veterans Affairs or the US government.

    References
    1.
    Hart  LG, Deyo  RA, Cherkin  DC.  Physician office visits for low back pain: frequency, clinical evaluation, and treatment patterns from a US national survey.   Spine (Phila Pa 1976). 1995;20(1):11-19. doi:10.1097/00007632-199501000-00003 PubMedGoogle ScholarCrossref
    2.
    Martin  BI, Deyo  RA, Mirza  SK,  et al.  Expenditures and health status among adults with back and neck problems.   JAMA. 2008;299(6):656-664. doi:10.1001/jama.299.6.656 PubMedGoogle ScholarCrossref
    3.
    Davis  MA, Onega  T, Weeks  WB, Lurie  JD.  Where the United States spends its spine dollars: expenditures on different ambulatory services for the management of back and neck conditions.   Spine (Phila Pa 1976). 2012;37(19):1693-1701. doi:10.1097/BRS.0b013e3182541f45PubMedGoogle ScholarCrossref
    4.
    Dagenais  S, Galloway  EK, Roffey  DM.  A systematic review of diagnostic imaging use for low back pain in the United States.   Spine J. 2014;14(6):1036-1048. doi:10.1016/j.spinee.2013.10.031 PubMedGoogle ScholarCrossref
    5.
    Di Iorio  D, Henley  E, Doughty  A.  A survey of primary care physician practice patterns and adherence to acute low back problem guidelines.   Arch Fam Med. 2000;9(10):1015-1021. doi:10.1001/archfami.9.10.1015 PubMedGoogle ScholarCrossref
    6.
    Ivanova  JI, Birnbaum  HG, Schiller  M, Kantor  E, Johnstone  BM, Swindle  RW.  Real-world practice patterns, health-care utilization, and costs in patients with low back pain: the long road to guideline-concordant care.   Spine J. 2011;11(7):622-632. doi:10.1016/j.spinee.2011.03.017 PubMedGoogle ScholarCrossref
    7.
    Chou  R, Qaseem  A, Snow  V,  et al; Clinical Efficacy Assessment Subcommittee of the American College of Physicians; American College of Physicians; American Pain Society Low Back Pain Guidelines Panel.  Diagnosis and treatment of low back pain: a joint clinical practice guideline from the American College of Physicians and the American Pain Society.   Ann Intern Med. 2007;147(7):478-491. doi:10.7326/0003-4819-147-7-200710020-00006 PubMedGoogle ScholarCrossref
    8.
    Lehnert  BE, Bree  RL.  Analysis of appropriateness of outpatient CT and MRI referred from primary care clinics at an academic medical center: how critical is the need for improved decision support?   J Am Coll Radiol. 2010;7(3):192-197. doi:10.1016/j.jacr.2009.11.010 PubMedGoogle ScholarCrossref
    9.
    Picano  E.  Sustainability of medical imaging.   BMJ. 2004;328(7439):578-580. doi:10.1136/bmj.328.7439.578 PubMedGoogle ScholarCrossref
    10.
    Rao  JK, Kroenke  K, Mihaliak  KA, Eckert  GJ, Weinberger  M.  Can guidelines impact the ordering of magnetic resonance imaging studies by primary care providers for low back pain?   Am J Manag Care. 2002;8(1):27-35.PubMedGoogle Scholar
    11.
    Emery  DJ, Shojania  KG, Forster  AJ, Mojaverian  N, Feasby  TE.  Overuse of magnetic resonance imaging.   JAMA Intern Med. 2013;173(9):823-825. doi:10.1001/jamainternmed.2013.3804 PubMedGoogle ScholarCrossref
    12.
    Fritz  JM, Brennan  GP, Hunter  SJ.  Physical therapy or advanced imaging as first management strategy following a new consultation for low back pain in primary care: associations with future health care utilization and charges.   Health Serv Res. 2015;50(6):1927-1940. doi:10.1111/1475-6773.12301 PubMedGoogle ScholarCrossref
    13.
    Chou  R, Deyo  RA, Jarvik  JG.  Appropriate use of lumbar imaging for evaluation of low back pain.   Radiol Clin North Am. 2012;50(4):569-585. doi:10.1016/j.rcl.2012.04.005 PubMedGoogle ScholarCrossref
    14.
    Chou  R, Fu  R, Carrino  JA, Deyo  RA.  Imaging strategies for low-back pain: systematic review and meta-analysis.   Lancet. 2009;373(9662):463-472. doi:10.1016/S0140-6736(09)60172-0 PubMedGoogle ScholarCrossref
    15.
    Lurie  JD, Birkmeyer  NJ, Weinstein  JN.  Rates of advanced spinal imaging and spine surgery.   Spine (Phila Pa 1976). 2003;28(6):616-620. doi:10.1097/01.BRS.0000049927.37696.DC PubMedGoogle Scholar
    16.
    Webster  BS, Cifuentes  M.  Relationship of early magnetic resonance imaging for work-related acute low back pain with disability and medical utilization outcomes.   J Occup Environ Med. 2010;52(9):900-907. doi:10.1097/JOM.0b013e3181ef7e53 PubMedGoogle ScholarCrossref
    17.
    Ip  IK, Lacson  R, Hentel  K,  et al.  JOURNAL CLUB: predictors of provider response to clinical decision support: lessons learned from the Medicare Imaging Demonstration.   AJR Am J Roentgenol. 2017;208(2):351-357. doi:10.2214/AJR.16.16373 PubMedGoogle ScholarCrossref
    18.
    US Department of Health and Human Services.  Medicare Program; Revisions to Payment Policies Under the Physician Fee Schedule and Other Revisions to Part B for CY 2019; Medicare Shared Savings Program Requirements; Quality Payment Program; and Medicaid Promoting Interoperability Program.   Fed Regist. 2018;83(145):35704-36368.Google Scholar
    19.
    Isaac  T, Weissman  JS, Davis  RB,  et al.  Overrides of medication alerts in ambulatory care.   Arch Intern Med. 2009;169(3):305-311. doi:10.1001/archinternmed.2008.551 PubMedGoogle ScholarCrossref
    20.
    Ash  JS, Sittig  DF, Dykstra  R, Campbell  E, Guappone  K.  The unintended consequences of computerized provider order entry: findings from a mixed methods exploration.   Int J Med Inform. 2009;78(suppl 1):S69-S76. doi:10.1016/j.ijmedinf.2008.07.015 PubMedGoogle ScholarCrossref
    21.
    The Lewin Group.  MRI Lumbar Spine for Low Back Pain (OP-8) Comprehensive Reevaluation Report. Yale-New Haven Health Services Corporation/Center for Outcomes Research and Evaluation; 2016.
    22.
    Gold  R, Esterberg  E, Hollombe  C,  et al.  Low back imaging when not indicated: a descriptive cross-system analysis.   Perm J. 2016;20(2):25-33. doi:10.7812/TPP/15-081 PubMedGoogle Scholar
    23.
    Sinnott  PL, Siroka  AM, Shane  AC, Trafton  JA, Wagner  TH.  Identifying neck and back pain in administrative data: defining the right cohort.   Spine (Phila Pa 1976). 2012;37(10):860-874. doi:10.1097/BRS.0b013e3182376508 PubMedGoogle Scholar
    24.
    Tan  G, Jensen  MP, Thornby  JI, Rintala  DH, Anderson  KO.  Categorizing pain in patients seen in a veterans health administration hospital: Pain as the fifth vital sign.   Psychol Serv. 2008;5(3):239. doi:10.1037/1541-1559.5.3.239Google Scholar
    25.
    DeLong  ER, DeLong  DM, Clarke-Pearson  DL.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.   Biometrics. 1988;44(3):837-845. doi:10.2307/2531595 PubMedGoogle Scholar
    26.
    Hall  BL, Huffman  KM, Hamilton  BH,  et al.  Profiling individual surgeon performance using information from a high-quality clinical registry: opportunities and limitations.   J Am Coll Surg. 2015;221(5):901-913. doi:10.1016/j.jamcollsurg.2015.07.454 PubMedGoogle Scholar
    27.
    Hashmi  ZG, Dimick  JB, Efron  DT,  et al.  Reliability adjustment: a necessity for trauma center ranking and benchmarking.   J Trauma Acute Care Surg. 2013;75(1):166-172. doi:10.1097/TA.0b013e318298494f PubMedGoogle Scholar
    28.
    Hess  CN, Rao  SV, McCoy  LA,  et al.  Identification of hospital outliers in bleeding complications after percutaneous coronary intervention.   Circ Cardiovasc Qual Outcomes. 2015;8(1):15-22. doi:10.1161/CIRCOUTCOMES.113.000749 PubMedGoogle Scholar
    29.
    Krell  RW, Hozain  A, Kao  LS, Dimick  JB.  Reliability of risk-adjusted outcomes for profiling hospital surgical quality.   JAMA Surg. 2014;149(5):467-474. doi:10.1001/jamasurg.2013.4249 PubMedGoogle Scholar
    30.
    Manda  SO, Gale  CP, Hall  AS, Gilthorpe  MS.  Statistical profiling of hospital performance using acute coronary syndrome mortality.   Cardiovasc J Afr. 2012;23(10):546-551. doi:10.5830/CVJA-2011-064 PubMedGoogle Scholar
    31.
    Dimick  JB, Staiger  DO, Birkmeyer  JD.  Ranking hospitals on surgical mortality: the importance of reliability adjustment.   Health Serv Res. 2010;45(6 Pt 1):1614-1629. doi:10.1111/j.1475-6773.2010.01158.x PubMedGoogle Scholar
    32.
    Glance  LG, Dick  A, Osler  TM, Li  Y, Mukamel  DB.  Impact of changing the statistical methodology on hospital and surgeon ranking: the case of the New York State cardiac surgery report card.   Med Care. 2006;44(4):311-319. doi:10.1097/01.mlr.0000204106.64619.2a PubMedGoogle Scholar
    33.
    Agency for Healthcare Research and Quality. Quality Indicator Empirical Methods. Accessed June 9, 2020. https://www.qualityindicators.ahrq.gov/Downloads/Resources/Publications/2019/Empirical_Methods_2019.pdf
    34.
    Morris  C.  Parametric empirical bayes inference: theory and applications.   J Am Stat Assoc. 1983;78(381):47-55. doi:10.1080/01621459.1983.10477920Google Scholar
    35.
    Snijders  T, Bosker  R.  Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modelilng. 2nd ed. Sage Publications; 2012.
    36.
    Kouri  BE, Parsons  RG, Alpert  HR.  Physician self-referral for diagnostic imaging: review of the empiric literature.   AJR Am J Roentgenol. 2002;179(4):843-850. doi:10.2214/ajr.179.4.1790843 PubMedGoogle Scholar
    37.
    Goulet  JL, Brandt  C, Crystal  S,  et al.  Agreement between electronic medical record-based and self-administered pain numeric rating scale: clinical and research implications.   Med Care. 2013;51(3):245-250. doi:10.1097/MLR.0b013e318277f1ad PubMedGoogle Scholar
    ×