Association of Clinical Characteristics With Variation in Emergency Physician Preferences for Patients | Emergency Medicine | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Preference Variation for Observable Patient Characteristics Across All Emergency Physicians, as Measured by Individual Physician Fixed Effects
Preference Variation for Observable Patient Characteristics Across All Emergency Physicians, as Measured by Individual Physician Fixed Effects

Physician identifiers were assigned according to physician rank for fixed effect for age. All differences were relative to the reference physician. The F score was F61 = 2.2 (P < .001) for the differences in age, F61 = 4.7 (P < .001) for Emergency Severity Index (ESI) score, and F61 = 1.7 (P < .001) for comorbidity score. Error bars indicate 95% CIs.

aPhysician 35 was the reference physician.

Figure 2.  Mean Reported Emergency Department (ED) Visit Charges and Corrected ED Charges for Individual Physicians
Mean Reported Emergency Department (ED) Visit Charges and Corrected ED Charges for Individual Physicians

All charges were relative to the group median of the mean reported charges. Corrected charges accounted for physician preferences for observable patient characteristics (age, sex, comorbidities, Emergency Severity Index score, and chief concerns). Color gradient from green (lowest) to blue (highest) demonstrates the change in physician ranking order, with each color and data point representing a physician, ordered according to the mean reported charges.

Figure 3.  Preference Variation Over the Course of Emergency Department Shift for Observable Patient Characteristics and Top 20 Chief Concern Groups, as Measured by Shift Order Group Fixed Effects
Preference Variation Over the Course of Emergency Department Shift for Observable Patient Characteristics and Top 20 Chief Concern Groups, as Measured by Shift Order Group Fixed Effects

Patients in each shift were binned according to the order in which they were seen during a shift, up to 5 shift order groups: patients 1 to 5, 6 to 10, 11 to 15, 16 to 20, and 21 and more. The F score was F4 = 292.3 (P < .001) for the differences in age, F4 = 563.4 (P < .001) for Emergency Severity Index score (ESI), and F4 = 125.1 (P < .001) for comorbidity score. Shaded areas indicate 95% CIs.

aPatients 1-5 was the reference group.

Table 1.  Patient, Physician, and Emergency Department Visit Characteristic
Patient, Physician, and Emergency Department Visit Characteristic
Table 2.  F Tests of Significance for Preference Variation Across Physicians During Nonovernight and Overnight Hours
F Tests of Significance for Preference Variation Across Physicians During Nonovernight and Overnight Hours
1.
Wennberg  J, Gittelsohn  A.  Small area variations in health care delivery.  Science. 1973;182(4117):1102-1108. doi:10.1126/science.182.4117.1102PubMedGoogle ScholarCrossref
2.
Tsugawa  Y, Jha  AK, Newhouse  JP, Zaslavsky  AM, Jena  AB.  Variation in physician spending and association with patient outcomes.  JAMA Intern Med. 2017;177(5):675-682. doi:10.1001/jamainternmed.2017.0059PubMedGoogle ScholarCrossref
3.
Corallo  AN, Croxford  R, Goodman  DC, Bryan  EL, Srivastava  D, Stukel  TA.  A systematic review of medical practice variation in OECD countries.  Health Policy. 2014;114(1):5-14. doi:10.1016/j.healthpol.2013.08.002PubMedGoogle ScholarCrossref
4.
Adams  JL, Mehrotra  A, Thomas  JW, McGlynn  EA.  Physician cost profiling–reliability and risk of misclassification.  N Engl J Med. 2010;362(11):1014-1021. doi:10.1056/NEJMsa0906323PubMedGoogle ScholarCrossref
5.
Welch  HG, Miller  ME, Welch  WP.  Physician profiling. An analysis of inpatient practice patterns in Florida and Oregon.  N Engl J Med. 1994;330(9):607-612. doi:10.1056/NEJM199403033300906PubMedGoogle ScholarCrossref
6.
Pines  JM, Isserman  JA, Szyld  D, Dean  AJ, McCusker  CM, Hollander  JE.  The effect of physician risk tolerance and the presence of an observation unit on decision making for ED patients with chest pain.  Am J Emerg Med. 2010;28(7):771-779. doi:10.1016/j.ajem.2009.03.019PubMedGoogle ScholarCrossref
7.
Abaluck  J, Agha  L, Kabrhel  C, Raja  A, Venkatesh  A.  The determinants of productivity in medical testing: intensity and allocation of care.  Am Econ Rev. 2016;106(12):3730-3764. doi:10.1257/aer.20140260PubMedGoogle ScholarCrossref
8.
Sirovich  B, Gallagher  PM, Wennberg  DE, Fisher  ES.  Discretionary decision making by primary care physicians and the cost of U.S. health care.  Health Aff (Millwood). 2008;27(3):813-823. doi:10.1377/hlthaff.27.3.813PubMedGoogle ScholarCrossref
9.
Mehrotra  A, Reid  RO, Adams  JL, Friedberg  MW, McGlynn  EA, Hussey  PS.  Physicians with the least experience have higher cost profiles than do physicians with the most experience.  Health Aff (Millwood). 2012;31(11):2453-2463. doi:10.1377/hlthaff.2011.0252PubMedGoogle ScholarCrossref
10.
Gowrisankaran  G, Joiner  KA, Léger  P-T.  Physician practice style and healthcare costs: evidence from emergency departments. National Bureau of Economic Research working paper 24155. https://ideas.repec.org/p/nbr/nberwo/24155.html. Published 2017. Accessed January 18, 2019. .
11.
Van Parys  J.  Variation in physician practice styles within and across emergency departments.  PLoS One. 2016;11(8):e0159882. doi:10.1371/journal.pone.0159882PubMedGoogle Scholar
12.
Greenwood  BN, Carnahan  S, Huang  L.  Patient-physician gender concordance and increased mortality among female heart attack patients.  Proc Natl Acad Sci U S A. 2018;115(34):8569-8574. doi:10.1073/pnas.1800097115PubMedGoogle ScholarCrossref
13.
Barnett  ML, Olenski  AR, Jena  AB.  Opioid-prescribing patterns of emergency physicians and risk of long-term use.  N Engl J Med. 2017;376(7):663-673. doi:10.1056/NEJMsa1610524PubMedGoogle ScholarCrossref
14.
von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.  Ann Intern Med. 2007;147(8):573-577. doi:10.7326/0003-4819-147-8-200710160-00010PubMedGoogle ScholarCrossref
15.
US Centers for Medicare & Medicaid Services. NPPES NPI registry. National Plan and Provider Enumeration System. https://npiregistry.cms.hhs.gov/. Accessed May 6, 2019.
16.
Wuerz  RC, Milne  LW, Eitel  DR, Travers  D, Gilboy  N.  Reliability and validity of a new five-level triage instrument.  Acad Emerg Med. 2000;7(3):236-242. doi:10.1111/j.1553-2712.2000.tb01066.xPubMedGoogle ScholarCrossref
17.
Gagne  JJ, Glynn  RJ, Avorn  J, Levin  R, Schneeweiss  S.  A combined comorbidity score predicted mortality in elderly patients better than existing scores.  J Clin Epidemiol. 2011;64(7):749-759. doi:10.1016/j.jclinepi.2010.10.004PubMedGoogle ScholarCrossref
18.
Barnett  V, Lewis  T.  Outliers in Statistical Data. 3rd ed. New York, NY: Wiley; 1994.
19.
McCarthy  ML, Zeger  SL, Ding  R, Aronsky  D, Hoot  NR, Kelen  GD.  The challenge of predicting demand for emergency department services.  Acad Emerg Med. 2008;15(4):337-346. doi:10.1111/j.1553-2712.2008.00083.xPubMedGoogle ScholarCrossref
20.
Downing  A, Wilson  R.  Temporal and demographic variations in attendance at accident and emergency departments.  Emerg Med J. 2002;19(6):531-535. doi:10.1136/emj.19.6.531PubMedGoogle ScholarCrossref
21.
Jones  SS, Thomas  A, Evans  RS, Welch  SJ, Haug  PJ, Snow  GL.  Forecasting daily patient volumes in the emergency department.  Acad Emerg Med. 2008;15(2):159-170. doi:10.1111/j.1553-2712.2007.00032.xPubMedGoogle ScholarCrossref
22.
Bhattacharyya  T, Millham  FH.  Relationship between weather and seasonal factors and trauma admission volume at a level I trauma center.  J Trauma. 2001;51(1):118-122. doi:10.1097/00005373-200107000-00019PubMedGoogle ScholarCrossref
23.
Rising  WR, O’Daniel  JAB, Roberts  CS.  Correlating weather and trauma admissions at a level I trauma center.  J Trauma. 2006;60(5):1096-1100. doi:10.1097/01.ta.0000197435.82141.27PubMedGoogle ScholarCrossref
24.
Tang  N, Stein  J, Hsia  RY, Maselli  JH, Gonzales  R.  Trends and characteristics of US emergency department visits, 1997-2007.  JAMA. 2010;304(6):664-670. doi:10.1001/jama.2010.1112PubMedGoogle ScholarCrossref
25.
Pallin  DJ, Allen  MB, Espinola  JA, Camargo  CA  Jr, Bohan  JS.  Population aging and emergency departments: visits will not increase, lengths-of-stay and hospitalizations will.  Health Aff (Millwood). 2013;32(7):1306-1312. doi:10.1377/hlthaff.2012.0951PubMedGoogle ScholarCrossref
26.
Hong  CS, Atlas  SJ, Chang  Y,  et al.  Relationship between patient panel characteristics and primary care physician clinical performance rankings.  JAMA. 2010;304(10):1107-1113. doi:10.1001/jama.2010.1287PubMedGoogle ScholarCrossref
27.
Greenfield  S, Kaplan  SH, Kahn  R, Ninomiya  J, Griffith  JL.  Profiling care provided by different groups of physicians: effects of patient case-mix (bias) and physician-level clustering on quality assessment results.  Ann Intern Med. 2002;136(2):111-121. doi:10.7326/0003-4819-136-2-200201150-00008PubMedGoogle ScholarCrossref
28.
Van Parys  J, Skinner  J.  Physician practice style variation-implications for policy.  JAMA Intern Med. 2016;176(10):1549-1550. doi:10.1001/jamainternmed.2016.4433PubMedGoogle ScholarCrossref
29.
Mafi  JN, Wee  CC, Davis  RB, Landon  BE.  Association of primary care practice location and ownership with the provision of low-value care in the United States.  JAMA Intern Med. 2017;177(6):838-845. doi:10.1001/jamainternmed.2017.0410PubMedGoogle ScholarCrossref
30.
Cutler  D, Skinner  J, Stern  AD, Wennberg  D.  Physician beliefs and patient preferences: a new look at regional variation in health care spending. National Bureau of Economic Research working paper 19320. Revised February 2018. https://www.nber.org/papers/w19320. Accessed March 3, 2019.
31.
Schuur  JD, Hsia  RY, Burstin  H, Schull  MJ, Pines  JM.  Quality measurement in the emergency department: past and future.  Health Aff (Millwood). 2013;32(12):2129-2138. doi:10.1377/hlthaff.2013.0730PubMedGoogle ScholarCrossref
32.
Song  Y, Skinner  J, Bynum  J, Sutherland  J, Wennberg  JE, Fisher  ES.  Regional variations in diagnostic practices.  N Engl J Med. 2010;363(1):45-53. doi:10.1056/NEJMsa0910881PubMedGoogle ScholarCrossref
33.
Burke  LG, Wild  RC, Orav  EJ, Hsia  RY.  Are trends in billing for high-intensity emergency care explained by changes in services provided in the emergency department? An observational study among US Medicare beneficiaries.  BMJ Open. 2018;8(1):e019357. doi:10.1136/bmjopen-2017-019357PubMedGoogle Scholar
34.
Werner  RM, Asch  DA.  The unintended consequences of publicly reporting quality information.  JAMA. 2005;293(10):1239-1244. doi:10.1001/jama.293.10.1239PubMedGoogle ScholarCrossref
35.
Poses  RM, McClish  DK, Smith  WR,  et al.  Results of report cards for patients with congestive heart failure depend on the method used to adjust for severity.  Ann Intern Med. 2000;133(1):10-20. doi:10.7326/0003-4819-133-1-200007040-00003PubMedGoogle ScholarCrossref
36.
Schwartz  AL, Landon  BE, Elshaug  AG, Chernew  ME, McWilliams  JM.  Measuring low-value care in Medicare.  JAMA Intern Med. 2014;174(7):1067-1076. doi:10.1001/jamainternmed.2014.1541PubMedGoogle ScholarCrossref
37.
Frances  CD, Go  AS, Dauterman  KW,  et al.  Outcome following acute myocardial infarction: are differences among physician specialties the result of quality of care or case mix?  Arch Intern Med. 1999;159(13):1429-1436. doi:10.1001/archinte.159.13.1429PubMedGoogle ScholarCrossref
38.
Danziger  S, Levav  J, Avnaim-Pesso  L.  Extraneous factors in judicial decisions.  Proc Natl Acad Sci U S A. 2011;108(17):6889-6892. doi:10.1073/pnas.1018033108PubMedGoogle ScholarCrossref
39.
Linder  JA, Doctor  JN, Friedberg  MW,  et al.  Time of day and the decision to prescribe antibiotics.  JAMA Intern Med. 2014;174(12):2029-2031. doi:10.1001/jamainternmed.2014.5225PubMedGoogle ScholarCrossref
40.
Chan  MY, Cohen  H, Spiegel  BMR.  Fewer polyps detected by colonoscopy as the day progresses at a Veteran’s Administration teaching hospital.  Clin Gastroenterol Hepatol. 2009;7(11):1217-1223. doi:10.1016/j.cgh.2009.07.013PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    Health Policy
    January 22, 2020

    Association of Clinical Characteristics With Variation in Emergency Physician Preferences for Patients

    Author Affiliations
    • 1Department of Emergency Medicine, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts
    • 2Berkeley School of Public Health, University of California, Berkeley
    JAMA Netw Open. 2020;3(1):e1919607. doi:10.1001/jamanetworkopen.2019.19607
    Key Points español 中文 (chinese)

    Question  Do physicians have measurable preferences for certain kinds of patients, and is preference variation associated with overall practice variation among physicians?

    Findings  In this cross-sectional study of 294 915 emergency department visits, attending physicians’ preference varied significantly in how they selected the patients they saw. Accounting for these individual preferences resulted in substantial reordering of physician performance ranking by care intensity.

    Meaning  This study suggests that overall practice variation among physicians likely reflects preference variation, which is currently not considered in efforts to reduce practice variation.

    Abstract

    Importance  Much of the wide variation in health care has been associated with practice variation among physicians. Physicians choosing to see patients with more (or fewer) care needs could also produce variations in care observed across physicians.

    Objective  To quantify emergency physician preferences by measuring nonrandom variations in patients they choose to see.

    Design, Setting, and Participants  This cross-sectional study used a large, detailed clinical data set from an electronic health record system of a single academic hospital. The data set included all emergency department (ED) encounters of adult patients from January 1, 2010, to May 31, 2015, as well as ED visits information. Data were analyzed from September 1, 2018, to March 31, 2019.

    Exposure  Patient assignment to a particular emergency physician.

    Main Outcomes and Measures  Variation in patient characteristics (age, sex, acuity [Emergency Severity Index score], and comorbidities) seen by emergency physicians before patient selection, adjusted for temporal factors (seasonal, weekly, and hourly variation in patient mix).

    Results  This study analyzed 294 915 visits to the ED seen by 62 attending physicians. Of the 294 915 patients seen, the mean (SD) age was 48.6 (19.8) years and 176 690 patients (59.9%) were women. Many patient characteristics, such as age (F = 2.2; P < .001), comorbidities (F = 1.7; P < .001), and acuity (F = 4.7; P < .001), varied statistically significantly. Compared with the lowest-quintile physicians for each respective characteristic, the highest-quintile physicians saw patients who were older (mean age, 47.9 [95% CI, 47.8-48.1] vs 49.7 [95% CI, 49.5-49.9] years, respectively; difference, +1.8 years; 95% CI, 1.5-2.0 years) and sicker (mean comorbidity score: 0.4 [95% CI, 0.3-0.5] vs 1.8 [95% CI, 1.7-1.8], respectively; difference, +1.3; 95% CI, 1.2-1.4). These differences were absent or highly attenuated during overnight shifts, when only 1 physician was on duty and there was limited room for patient selection. Compared with earlier in the shift, the same physician later in the shift saw patients who were younger (mean age, 49.7 [95% CI, 49.4-49.7] vs 44.6 [95 % CI, 44.3-44.9] years, respectively; difference, –5.1 years; 95% CI, 4.8-5.5) and less sick (mean comorbidity score: 0.7 [95% CI, 0.7-0.8] vs 1.1 [95% CI, 1.1-1.1], respectively; difference, –0.4; 95% CI, 0.4-0.4). Accounting for preference variation resulted in substantial reordering of physician ranking by care intensity, as measured by ED charges, with 48 of 62 physicians (77%) being reclassified into a different quintile and 9 of 12 physicians (75%) in the highest care intensity quintile moving into a lower quintile. A regression model demonstrated that 22% of reported ED charges were associated with physician preference.

    Conclusions and Relevance  This study found preference variation across physicians and within physicians during the course of a shift. These findings suggest that current efforts to reduce practice variation may not affect the variation associated with physician preferences, which reflect underlying differences in patient needs and not physician practice.

    Introduction

    Substantial variation in practice patterns across geographic areas, hospitals, and physicians (and, consequently, in health care costs) has been widely established.1-3 Considerably less evidence exists, however, on the mechanisms of this variation; specifically, why do some physicians deliver higher-intensity, higher-cost care? A common explanation for this question is that, given a similar patient (eg, one with chest pain), physicians have intrinsic preferences for higher or lower levels of care intensity (eg, whether or not to order a stress test). In the aggregate, these preferences, which are potentially driven by local practice culture, training institution, or risk tolerance, are a factor in the wide variation in practice across hospitals and regions.4-9

    In this cross-sectional study, we explored an alternative mechanism for how variation may arise: physicians’ intrinsic preferences for different kinds of patients, who may require different levels of care intensity (eg, patients with chest pain vs toe pain). Sorting particular kinds of patients according to particular kinds of physicians is another explanation for the wide variation in care and costs across physicians (eg, patients with chest pain need different care from those with toe pain), but it has different policy implications than approaches to reduce preferences for care intensity.

    To date, studies of practice variation have relied on claims data to compare regions, hospitals, or physicians. Because claims data contain limited information on physician behaviors or preferences besides care intensity and costs, they can say little about the mechanisms by which variation arises. In addition, physician preferences for patient characteristics are challenging to study in many clinical settings because physician-patient matching occurs infrequently (eg, first primary care visit), and it can be difficult to analytically isolate the role of physician preference from the role of patient preference, appointment availability, geographic barriers, insurance coverage, and other factors. The existence of preference variation may also pose methodologic problems for many practice variation studies. If patients who are sicker (or less sick) are preferentially seen by different clinicians, this practice would challenge the assumption that patients are randomly assigned to clinicians (eg, in emergency departments [EDs]).10-13

    In this study, by contrast, we examined a setting in which physician-patient matching was frequent and easy to identify. We used a comprehensive electronic health record (EHR) data set with detailed information on physician behaviors for more than 300 000 patient encounters at a large ED of an academic hospital, including what physicians observed about patients before signing up to see them, the exact times and shift orders in which patients were seen, and detailed financial charges. In this ED, multiple physicians practiced simultaneously and independently, and physicians had the discretion to select patients. This unique setting allowed us to explore both the variation across physicians and the variation within physicians over time. Although it is important to acknowledge the main limitation of this study, that these data came from the EHR system of only 1 academic hospital, the available level of detail was, to our knowledge, not available in most data sets of this size and provides a new window into the mechanisms associated with variation.

    Methods

    This cross-sectional study was approved by the institutional review board of Partners HealthCare, which also granted a waiver of informed consent given that this secondary analysis of existing data presented no more than minimal risk to participants. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.14

    Study Population and Setting

    All adult visits to a large, urban ED of an academic hospital between January 1, 2010, and May 31, 2015, were identified. All data from adult patients and ED visits, including demographic information, previous diagnoses, attending physician of record, orders and prescriptions, and dates and times of ED arrivals and orders, were obtained from the hospital EHR system for analysis in July 2018. In this mixed-acuity ED, patients were not assigned to different clinical areas by their chief concern or acuity, and physicians were randomly assigned to these care areas in staggered, overlapping shifts.

    No financial incentives were offered to physicians on the basis of patient selection or performance metrics. Physician sex and number of years after residency were obtained from the National Provider Identifier database, and physicians were categorized as having 0 to 4, 5 to 9, or 10 or more years of postresidency practice; all physicians had emergency medicine training.15

    A total of 12 136 visits (3.9%) with incomplete time stamp, ED encounter, or attending physician information were excluded. We also excluded 15 031 visits (4.8%) seen by physicians in the bottom 25th percentile of visits per year during the study period (the median annual visits for excluded physicians were 192 visits per year).

    Observable Patient Characteristics Before Selection

    The patient characteristics that can be observed in the EHR system by physicians before patient selection were age, sex, Emergency Severity Index (ESI) score, chief concerns, and comorbidities. Age, sex, ESI score, and chief concerns were recorded in triage by a nurse and were available on the ED clinical interface used by physicians to assign themselves to a patient (eFigure 1 in the Supplement). The ESI is an ED triage algorithm that stratifies patients into 5 levels from 1 (most urgent) to 5 (least urgent) on the basis of acuity and resource needs.16 Chief concern is the patient’s reason for the ED visit and captured as free text.

    To facilitate analysis, we classified all chief concerns from ED visits using regular expression matching into 154 categories, and the highest frequency chief concerns were explored. For example, chest pain and chest discomfort were both classified into the chief concern group of chest pain. Given that comorbidities are readily accessible in the EHR system and are often reviewed by physicians before patient selection, we calculated a Gagne comorbidity score for each encounter and included it as a potentially observable characteristic.17 We did not examine patient characteristics that were not easily available (ie, displayed on the EHR interface) before patient selection, such as health insurance status and type or primary language spoken.

    ED Charges, In-Shift Order, and Time Control Variables

    We obtained ED charges from ED administrative data. Extreme values were winsorized at the 0.05th and 99.95th percentile values (ie, values below the 0.05th percentile were replaced with the 0.05th percentile values and above the 99.95th were replaced with the 99.95th).18

    To determine the order in which a patient was seen on a physician shift, we identified the attending physician of record and the time of first order for each visit. To account for physicians often managing multiple patients simultaneously and patients possibly not being seen in the exact order in which orders were placed, we binned patients in each given shift into up to 5 shift order groups: patients 1 to 5, 6 to 10, 11 to 15, 16 to 20, and 21 and more. For the 6453 visits (2.2%) without any orders during the ED encounter, we estimated the time of first contact from visits with similar lengths of stay (ie, by regressing the minutes to first order on length of stay for those with first order and imputing the minutes to first contact for those without an order).

    Time and date of ED arrival were obtained for each visit. The month, day of the week, and hour of visit were separately recorded to account for the known temporal and demographic periodicity in ED shifts.19-21 For example, ED visit volumes are highest on Mondays and lowest between midnight and early morning, whereas presentations for influenza peak in the fall and winter and traumas peak in the summer.22,23 These time variables also account for the nonrandom assignment of some physicians to particular shifts, given that physicians who work only weekday daytime shifts see different types of patients than those who work predominantly on the weekends (we did not account for preference variation through the channel of shift choice). The year of visit was used to account for temporal trends in ED visits.24,25 Visits with time of arrival between midnight and 7 am, when all patients must be seen by the only physician on duty, were classified as overnight visits.

    Statistical Analysis

    To examine the variation in observable characteristics of patients seen by different physicians, we estimated individual physician fixed effects for each patient characteristic by using multivariable regression (eg, difference in age of patients assigned to one physician over another) adjusted for a set of ED arrival time control variables. These time control variables accounted for the known temporal variation in ED shifts and nonrandom assignment of physician shifts. Primary outcomes were the variation in observable patient characteristics across different ED physicians and over the course of a shift (ie, across shift order groups). For clinical contextualization, we also stratified physicians into quintile groups by each patient characteristic and compared them across quintiles. Odds ratios (ORs) comparing physician quintiles were then obtained from the regression models of specific patient characteristics (eg, aged >50 years; ESI score of 1 or 2) on quintile groups, adjusted for time controls.

    As a robustness check, we performed identical analyses for patients who presented during the overnight hours (between midnight and 7 am), when all patients must be seen by the only physician on duty in the ED. We hypothesized that, because having 1 physician overnight would leave little room for physicians to exercise choice, it would eliminate or attenuate the variation seen during the nonovernight hours (between 7 am and midnight).

    To examine the variation in the observable characteristics of patients seen over the course of a shift, we similarly regressed patient characteristics on the order in which a patient was seen during a shift (ie, shift order groups 1 through 5), again adjusted for ED arrival time control variables. The ORs were calculated from regression coefficients to identify the association between in-shift order groups and patient characteristics. Data were analyzed from September 1, 2018, to March 31, 2019.

    To illustrate the importance of preference variation, we quantified the extent to which preference variation accounted for the differences in the amount of care physicians delivered as measured by ED charges. First, we estimated the association between ED charges and observable patient characteristics using a regression model of ED charges on age, sex, comorbidities, ESI score, chief concerns, and time control variables. Second, we corrected the observable patient characteristics by adding the corresponding fixed effect (eg, patients seen by a physician with an age effect of +1 year will have their age adjusted down by 1 year) to simulate a scenario without preferences. Third, using multivariable regression, we estimated the charges for each encounter using the corrected patient characteristic (eg, the charges if the patient were 1 year younger), and we tabulated these corrected charges by physician. The proportion of charges attributable to preferences was determined using a regression model of the difference between reported and corrected charges on reported charges.

    F tests were used to test for the significance of physician and fixed effects of shift order group. Two-sided P = .05 was considered statistically significant. Bonferroni correction was performed for multiple comparisons. All analyses were performed in R statistical software, version 3.4.3 (R Foundation). All SEs were corrected at the patient level to account for patients with multiple visits.

    Results
    Patient and ED Visit Characteristics

    This study analyzed 294 915 visits to the ED seen by 62 attending physicians, representing 94.0% of all visits to the study ED during the study period after excluding encounters with incomplete patient or physician data and encounters managed by physicians with a low number of clinical encounters per year. Of the 294 915 patients seen, the mean (SD) age was 48.6 (19.8) years, and 176 690 patients (59.9%) were women. A mean of 19.0 (95% CI, 19.0-19.1) patients were seen during each shift, with 40 925 patients (13.9%) presenting during overnight hours when only 1 physician was on duty in the ED (Table 1). Many patient characteristics, such as age (F = 2.2; P < .001), comorbidities (F = 1.7; P < .001), and acuity (F = 4.7; P < .001), varied statistically significantly.

    Preference Variation Across Physicians

    We quantified preference variation across physicians in observable patient characteristics before patient selection. Figure 1 demonstrates that considerable variation exists in the fixed effects of patient characteristics among different physicians. Physicians in the highest quintile for patient age saw patients who were 1.8 years older (95% CI, 1.5-2.0 years) compared with physicians in the lowest quintile for age (mean age, 49.7 [95% CI, 49.5-49.9] vs 47.9 [95% CI, 47.8-48.1] years). The OR of physicians in the highest quintile for age seeing patients older than 50 years compared with physicians in the lowest quintile was 1.2 (95% CI, 1.1-1.3). The difference in mean ESI scores for the highest and lowest quintile physicians was 3.1% (2.8 vs 2.9; difference, –0.1; 95% CI, 0.1-0.1). The OR of physicians in the highest quintile for ESI score seeing patients with an ESI score of 1 or 2 compared with physicians in the lowest quintile was 1.3 (95% CI, 1.2-1.3). Physicians in the highest quintile saw patients with a 74% higher mean comorbidity score compared with physicians in the lowest quintile (1.8 [95% CI, 1.7-1.8] vs 0.4 [95% CI, 0.3-0.5]; difference, +1.3; 95% CI, 1.2-1.4). A fixed-effects model showed no statistically significant difference in the proportion of female patients seen across different physicians (F1,61 = 1.2; P = .15) (eFigure 2A in the Supplement). Of the top 20 chief concern groups by frequency, significant differences were found in 8 concern groups across all physicians (distal extremity concern [F = 2.02; P < .001]; chest pain [F = 1.33; P = .04]; motor vehicle–related injury [F = 1.34; P = .04]; bleeding [F = 1.34; P = .04]; ear, nose, or throat concern [F = 1.46; P = .01]; dizziness or lightheadedness [F = 1.41; P = .02]; skin concern [F = 1.40; P = .02]; and vaginal bleeding [F = 1.36; P = .03]) (eTable 1 in the Supplement).

    In addition, we found associations between physician preference for age, comorbidities, and ESI score. Physicians who saw older patients also saw patients with more comorbidities (comorbidities score: 1.7 vs 0.8) and higher acuity (lower ESI score: 2.8 vs 2.9) (eFigure 3 in the Supplement). No preference variation was observed across physicians according to physician sex or years of postresidency practice (eTable 2 and eTable 3 in the Supplement).

    Preference Variation During Overnight Presentations

    We hypothesized that preference variation across physicians would be attenuated during overnight hours when all patients in the ED had to be seen by the only physician on duty and there was less opportunity for patient selection. Table 2 shows a comparison of the results of F tests of significance for nonovernight shifts (7 am to midnight) and the overnight hours (midnight to 7 am). During nonovernight hours, statistically significant variations in age (F1,61 = 2.1; P < .001), ESI score (F1,61 = 4.5; P < .001), and comorbidities (F1,61 = 1.5; P = .004) were found. In contrast, with reduced discretion for patient selection during the overnight hours, no statistically significant variations were found in observable patient characteristics across physicians.

    Physician Rankings by Care Intensity

    We quantified the changes in physician rankings by care intensity, as measured by ED charges, when accounting for preference variation. Figure 2 compares the mean reported ED charges for each physician with their mean corrected ED charges, which accounted for preference. Physicians whose corrected charges were lower than their reported charges represented those with greater preference for patients with higher needs (ie, older age, higher comorbidity, and higher acuity). Accounting for physician preference resulted in considerable reordering of physician performance ranking, with 48 of 62 of physicians (77%) being reclassified into different quintiles, and in a mean absolute percentile change of 29.3 (95% CI, 24.3-34.3). Fifty-one physicians (82%) had a greater than 10th percentile absolute change in rankings, and 33 (53%) had a greater than 25th percentile absolute change in rankings. Nine of 12 physicians (75%) in the highest quintile moved into a lower quintile after accounting for preference, and 6 of 12 physicians (50%) in the lowest quintile moved into a higher quintile after accounting for preference. A regression model demonstrated that 22% of the reported ED charges were associated with physician preference.

    Preference Variation Over the Course of a Shift

    Another unexplored possibility was that physician preference may change over the course of a shift. Figure 3A compares observable patient characteristics across the 5 shift order groups. Compared with the start of the shift, at the end of the shift, physicians tended to see patients who were younger, were female, had lower comorbidity score, and had a higher ESI score (ie, lower acuity). Patients seen at the end of the shift (group 5) were a mean of 5.1 years younger (95% CI, 4.8-5.5) than those seen at the start of the shift (group 1) (mean age, 44.6 years [95% CI, 44.3-44.9] vs 49.7 [95% CI, 49.4-49.7] years, respectively). The OR of patients older than 50 years being seen at the end of the shift compared with the start of the shift was 0.7 (95% CI, 0.6-0.7). The mean combined comorbidity score was 34% lower for patients at the end of the shift compared with the start of the shift (mean score, 0.7 [95% CI, 0.7-0.8] vs 1.1 [95% CI, 1.1-1.1]; difference, –0.4; 95% CI, 0.4-0.4). Patients at the end, compared with the start, of the shift had 7% lower acuity (mean ESI score, 2.8 [95% CI, 2.8-2.8] vs 3.0 [95% CI, 3.0-3.0]; difference, 0.2; 95% CI, 0.2-0.2). The OR of patients with an ESI score of 1 or 2 being seen at the end of the shift compared with the start of the shift was 0.5 (95% CI, 0.5-0.6). A fixed-effects model showed that physicians at the end of the shift saw 7% more female patients than at the end of shift (F1,61 = 17.4; P < .001) (eFigure 2B in the Supplement).

    Figure 3B compares the variation in the chief concern groups over the course of the shift. At the start of the shift, physicians preferred patients with more undifferentiated chief concerns, such as syncope and weakness or failure to thrive. At the end of the shift, physicians preferred patients with more differentiated chief concerns, such as skin concerns and lacerations.

    Discussion

    Using a large clinical data set from an EHR system, we observed significant variation in patient selection preference both across physicians and within physicians over the course of the shift. First, some physicians appeared to prefer patients with higher comorbidities, older age, or less defined chief concerns, whereas others preferred the opposite. Second, preferences appeared to change over the course of a shift, with younger patients, healthier patients, and patients with more defined concerns being preferred later in the shift. This variation across physicians was eliminated when analysis was restricted to overnight hours when all patients must be seen by the only physician on duty. Although physicians may not always have the option to select patients, these findings suggest that, when given the opportunity to choose, physicians likely exercise their choice based on individual preference. In addition, accounting for these individual preferences resulted in substantial reordering of physician ranking by care intensity; 75% of physicians in the highest charge quintile moved to a lower charge quintile given their preference for sicker patients. Twenty-two percent of reported ED charges were associated with physician preference.

    These results could have considerable implications for how we understand and address variation in physician practice, given that varying levels of acuity or medical complexity require different clinical management.26-28 What has been labeled as practice variation to date (eg, different physicians manage the same patient differently) may instead reflect preference variation (ie, different physicians choose to see different types of patients, thereby engendering heterogeneity in practice).13,29,30 Physicians who prefer sicker, more complex patients would likely order more diagnostic studies compared with physicians who prefer healthier patients, and thus variation in physician practice may be associated with the differences in the types of patients the physicians prefer. These findings suggest that the commonly held assumption that patient assignments are quasi-randomized should be carefully considered when studying physician variation.

    Accounting for preferences resulted in substantial reordering of physician ranking by care intensity, further illustrating the importance of recognizing the role of physician choice in evaluating performance metrics. Metrics associated with ED charges are increasingly used to compare emergency physician productivity and to determine compensation, but most metrics do not account for the types of patients seen.31-37 Case-mix characteristics are increasingly considered at the systems level, but findings from the present investigation support the need for rigorous risk adjustment when profiling physicians, even at the local level. As more programs target variation in practice across physicians, a more nuanced approach of understanding the factors associated with variation in performance measures could not only facilitate quality improvement but also mitigate physician burnout by better aligning metrics with the realities of clinical practice.

    This study also found variation in preference over the course of an ED shift. Physicians chose to see patients who were younger, were healthier, had lower acuity measures, and had more defined chief concerns at the end of the shift, which demonstrated physicians’ ability to preferentially select patients. These physicians may choose younger, healthier patients at the end of shift for a variety of reasons, such as simplifying the handoff to incoming clinicians or leaving a shift on time. This proclivity, however, may also signal cognitive fatigue at the end of a shift. Given the existing literature on the association of cognitive and decision fatigue with clinical practice, further studies into the implications of time of day for patient care and outcomes may be warranted.38-40

    Limitations

    This study has some limitations. It could be performed only at a single center because of the required granularity of data. The findings may not be generalizable and may apply only to EDs in which multiple physicians work simultaneously and independently and in which physicians have the discretion to select patients. The study ED has only 1 single-coverage shift per day, which limits the sample size available for investigation if preference variation decreases with reduced discretion for patient selection. We investigated the association between attending physicians and patient characteristics and not the behaviors of resident trainees or physician assistants or clinician preferences for working with certain colleagues. However, physician assistant shifts are randomly distributed across multiple care areas at several EDs, and resident trainees rotate through multiple hospital departments in monthly blocks; none of these schedules are correlated with attending physician shifts. In addition, given the large number of physician assistants and residents working in the study ED (approximately 50 physician assistants, 50 emergency medicine residents, and >150 off-service trainees), repeated pairings over multiple shifts occur infrequently. Areas of future research include the preference variation in other clinical settings and the association of preference with other physician performance metrics.

    Conclusions

    Preference variation in observable patient characteristics before patient selection was found both across physicians and within physicians over the course of an ED shift. Accounting for preference variation resulted in substantial reordering of physician performance rankings. Variation in physician preference for different types of patients should be considered when evaluating physician practice variation.

    Back to top
    Article Information

    Accepted for Publication: November 18, 2019.

    Published: January 22, 2020. doi:10.1001/jamanetworkopen.2019.19607

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Chang CY et al. JAMA Network Open.

    Corresponding Author: Cindy Y. Chang, MD, MPH, Department of Emergency Medicine, Brigham and Women’s Hospital, 75 Francis St, Boston, MA 02115 (cchang@bwh.harvard.edu).

    Author Contributions: Drs Chang and Obermeyer had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Both authors.

    Acquisition, analysis, or interpretation of data: Both authors.

    Drafting of the manuscript: Both authors.

    Critical revision of the manuscript for important intellectual content: Both authors.

    Statistical analysis: Both authors.

    Obtained funding: Obermeyer.

    Administrative, technical, or material support: Both authors.

    Supervision: Obermeyer.

    Conflict of Interest Disclosures: None reported.

    Funding/Support: This study was funded by grant DP5 OD012161 from the Office of the Director of the National Institutes of Health (Dr Obermeyer).

    Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    References
    1.
    Wennberg  J, Gittelsohn  A.  Small area variations in health care delivery.  Science. 1973;182(4117):1102-1108. doi:10.1126/science.182.4117.1102PubMedGoogle ScholarCrossref
    2.
    Tsugawa  Y, Jha  AK, Newhouse  JP, Zaslavsky  AM, Jena  AB.  Variation in physician spending and association with patient outcomes.  JAMA Intern Med. 2017;177(5):675-682. doi:10.1001/jamainternmed.2017.0059PubMedGoogle ScholarCrossref
    3.
    Corallo  AN, Croxford  R, Goodman  DC, Bryan  EL, Srivastava  D, Stukel  TA.  A systematic review of medical practice variation in OECD countries.  Health Policy. 2014;114(1):5-14. doi:10.1016/j.healthpol.2013.08.002PubMedGoogle ScholarCrossref
    4.
    Adams  JL, Mehrotra  A, Thomas  JW, McGlynn  EA.  Physician cost profiling–reliability and risk of misclassification.  N Engl J Med. 2010;362(11):1014-1021. doi:10.1056/NEJMsa0906323PubMedGoogle ScholarCrossref
    5.
    Welch  HG, Miller  ME, Welch  WP.  Physician profiling. An analysis of inpatient practice patterns in Florida and Oregon.  N Engl J Med. 1994;330(9):607-612. doi:10.1056/NEJM199403033300906PubMedGoogle ScholarCrossref
    6.
    Pines  JM, Isserman  JA, Szyld  D, Dean  AJ, McCusker  CM, Hollander  JE.  The effect of physician risk tolerance and the presence of an observation unit on decision making for ED patients with chest pain.  Am J Emerg Med. 2010;28(7):771-779. doi:10.1016/j.ajem.2009.03.019PubMedGoogle ScholarCrossref
    7.
    Abaluck  J, Agha  L, Kabrhel  C, Raja  A, Venkatesh  A.  The determinants of productivity in medical testing: intensity and allocation of care.  Am Econ Rev. 2016;106(12):3730-3764. doi:10.1257/aer.20140260PubMedGoogle ScholarCrossref
    8.
    Sirovich  B, Gallagher  PM, Wennberg  DE, Fisher  ES.  Discretionary decision making by primary care physicians and the cost of U.S. health care.  Health Aff (Millwood). 2008;27(3):813-823. doi:10.1377/hlthaff.27.3.813PubMedGoogle ScholarCrossref
    9.
    Mehrotra  A, Reid  RO, Adams  JL, Friedberg  MW, McGlynn  EA, Hussey  PS.  Physicians with the least experience have higher cost profiles than do physicians with the most experience.  Health Aff (Millwood). 2012;31(11):2453-2463. doi:10.1377/hlthaff.2011.0252PubMedGoogle ScholarCrossref
    10.
    Gowrisankaran  G, Joiner  KA, Léger  P-T.  Physician practice style and healthcare costs: evidence from emergency departments. National Bureau of Economic Research working paper 24155. https://ideas.repec.org/p/nbr/nberwo/24155.html. Published 2017. Accessed January 18, 2019. .
    11.
    Van Parys  J.  Variation in physician practice styles within and across emergency departments.  PLoS One. 2016;11(8):e0159882. doi:10.1371/journal.pone.0159882PubMedGoogle Scholar
    12.
    Greenwood  BN, Carnahan  S, Huang  L.  Patient-physician gender concordance and increased mortality among female heart attack patients.  Proc Natl Acad Sci U S A. 2018;115(34):8569-8574. doi:10.1073/pnas.1800097115PubMedGoogle ScholarCrossref
    13.
    Barnett  ML, Olenski  AR, Jena  AB.  Opioid-prescribing patterns of emergency physicians and risk of long-term use.  N Engl J Med. 2017;376(7):663-673. doi:10.1056/NEJMsa1610524PubMedGoogle ScholarCrossref
    14.
    von Elm  E, Altman  DG, Egger  M, Pocock  SJ, Gøtzsche  PC, Vandenbroucke  JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.  Ann Intern Med. 2007;147(8):573-577. doi:10.7326/0003-4819-147-8-200710160-00010PubMedGoogle ScholarCrossref
    15.
    US Centers for Medicare & Medicaid Services. NPPES NPI registry. National Plan and Provider Enumeration System. https://npiregistry.cms.hhs.gov/. Accessed May 6, 2019.
    16.
    Wuerz  RC, Milne  LW, Eitel  DR, Travers  D, Gilboy  N.  Reliability and validity of a new five-level triage instrument.  Acad Emerg Med. 2000;7(3):236-242. doi:10.1111/j.1553-2712.2000.tb01066.xPubMedGoogle ScholarCrossref
    17.
    Gagne  JJ, Glynn  RJ, Avorn  J, Levin  R, Schneeweiss  S.  A combined comorbidity score predicted mortality in elderly patients better than existing scores.  J Clin Epidemiol. 2011;64(7):749-759. doi:10.1016/j.jclinepi.2010.10.004PubMedGoogle ScholarCrossref
    18.
    Barnett  V, Lewis  T.  Outliers in Statistical Data. 3rd ed. New York, NY: Wiley; 1994.
    19.
    McCarthy  ML, Zeger  SL, Ding  R, Aronsky  D, Hoot  NR, Kelen  GD.  The challenge of predicting demand for emergency department services.  Acad Emerg Med. 2008;15(4):337-346. doi:10.1111/j.1553-2712.2008.00083.xPubMedGoogle ScholarCrossref
    20.
    Downing  A, Wilson  R.  Temporal and demographic variations in attendance at accident and emergency departments.  Emerg Med J. 2002;19(6):531-535. doi:10.1136/emj.19.6.531PubMedGoogle ScholarCrossref
    21.
    Jones  SS, Thomas  A, Evans  RS, Welch  SJ, Haug  PJ, Snow  GL.  Forecasting daily patient volumes in the emergency department.  Acad Emerg Med. 2008;15(2):159-170. doi:10.1111/j.1553-2712.2007.00032.xPubMedGoogle ScholarCrossref
    22.
    Bhattacharyya  T, Millham  FH.  Relationship between weather and seasonal factors and trauma admission volume at a level I trauma center.  J Trauma. 2001;51(1):118-122. doi:10.1097/00005373-200107000-00019PubMedGoogle ScholarCrossref
    23.
    Rising  WR, O’Daniel  JAB, Roberts  CS.  Correlating weather and trauma admissions at a level I trauma center.  J Trauma. 2006;60(5):1096-1100. doi:10.1097/01.ta.0000197435.82141.27PubMedGoogle ScholarCrossref
    24.
    Tang  N, Stein  J, Hsia  RY, Maselli  JH, Gonzales  R.  Trends and characteristics of US emergency department visits, 1997-2007.  JAMA. 2010;304(6):664-670. doi:10.1001/jama.2010.1112PubMedGoogle ScholarCrossref
    25.
    Pallin  DJ, Allen  MB, Espinola  JA, Camargo  CA  Jr, Bohan  JS.  Population aging and emergency departments: visits will not increase, lengths-of-stay and hospitalizations will.  Health Aff (Millwood). 2013;32(7):1306-1312. doi:10.1377/hlthaff.2012.0951PubMedGoogle ScholarCrossref
    26.
    Hong  CS, Atlas  SJ, Chang  Y,  et al.  Relationship between patient panel characteristics and primary care physician clinical performance rankings.  JAMA. 2010;304(10):1107-1113. doi:10.1001/jama.2010.1287PubMedGoogle ScholarCrossref
    27.
    Greenfield  S, Kaplan  SH, Kahn  R, Ninomiya  J, Griffith  JL.  Profiling care provided by different groups of physicians: effects of patient case-mix (bias) and physician-level clustering on quality assessment results.  Ann Intern Med. 2002;136(2):111-121. doi:10.7326/0003-4819-136-2-200201150-00008PubMedGoogle ScholarCrossref
    28.
    Van Parys  J, Skinner  J.  Physician practice style variation-implications for policy.  JAMA Intern Med. 2016;176(10):1549-1550. doi:10.1001/jamainternmed.2016.4433PubMedGoogle ScholarCrossref
    29.
    Mafi  JN, Wee  CC, Davis  RB, Landon  BE.  Association of primary care practice location and ownership with the provision of low-value care in the United States.  JAMA Intern Med. 2017;177(6):838-845. doi:10.1001/jamainternmed.2017.0410PubMedGoogle ScholarCrossref
    30.
    Cutler  D, Skinner  J, Stern  AD, Wennberg  D.  Physician beliefs and patient preferences: a new look at regional variation in health care spending. National Bureau of Economic Research working paper 19320. Revised February 2018. https://www.nber.org/papers/w19320. Accessed March 3, 2019.
    31.
    Schuur  JD, Hsia  RY, Burstin  H, Schull  MJ, Pines  JM.  Quality measurement in the emergency department: past and future.  Health Aff (Millwood). 2013;32(12):2129-2138. doi:10.1377/hlthaff.2013.0730PubMedGoogle ScholarCrossref
    32.
    Song  Y, Skinner  J, Bynum  J, Sutherland  J, Wennberg  JE, Fisher  ES.  Regional variations in diagnostic practices.  N Engl J Med. 2010;363(1):45-53. doi:10.1056/NEJMsa0910881PubMedGoogle ScholarCrossref
    33.
    Burke  LG, Wild  RC, Orav  EJ, Hsia  RY.  Are trends in billing for high-intensity emergency care explained by changes in services provided in the emergency department? An observational study among US Medicare beneficiaries.  BMJ Open. 2018;8(1):e019357. doi:10.1136/bmjopen-2017-019357PubMedGoogle Scholar
    34.
    Werner  RM, Asch  DA.  The unintended consequences of publicly reporting quality information.  JAMA. 2005;293(10):1239-1244. doi:10.1001/jama.293.10.1239PubMedGoogle ScholarCrossref
    35.
    Poses  RM, McClish  DK, Smith  WR,  et al.  Results of report cards for patients with congestive heart failure depend on the method used to adjust for severity.  Ann Intern Med. 2000;133(1):10-20. doi:10.7326/0003-4819-133-1-200007040-00003PubMedGoogle ScholarCrossref
    36.
    Schwartz  AL, Landon  BE, Elshaug  AG, Chernew  ME, McWilliams  JM.  Measuring low-value care in Medicare.  JAMA Intern Med. 2014;174(7):1067-1076. doi:10.1001/jamainternmed.2014.1541PubMedGoogle ScholarCrossref
    37.
    Frances  CD, Go  AS, Dauterman  KW,  et al.  Outcome following acute myocardial infarction: are differences among physician specialties the result of quality of care or case mix?  Arch Intern Med. 1999;159(13):1429-1436. doi:10.1001/archinte.159.13.1429PubMedGoogle ScholarCrossref
    38.
    Danziger  S, Levav  J, Avnaim-Pesso  L.  Extraneous factors in judicial decisions.  Proc Natl Acad Sci U S A. 2011;108(17):6889-6892. doi:10.1073/pnas.1018033108PubMedGoogle ScholarCrossref
    39.
    Linder  JA, Doctor  JN, Friedberg  MW,  et al.  Time of day and the decision to prescribe antibiotics.  JAMA Intern Med. 2014;174(12):2029-2031. doi:10.1001/jamainternmed.2014.5225PubMedGoogle ScholarCrossref
    40.
    Chan  MY, Cohen  H, Spiegel  BMR.  Fewer polyps detected by colonoscopy as the day progresses at a Veteran’s Administration teaching hospital.  Clin Gastroenterol Hepatol. 2009;7(11):1217-1223. doi:10.1016/j.cgh.2009.07.013PubMedGoogle ScholarCrossref
    ×