Patient and Visit Characteristics Associated With Use of Direct Scheduling in Primary Care Practices | Electronic Health Records | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Distribution of Directly Scheduled Visits per Patient Among Direct Scheduling Adopters
Distribution of Directly Scheduled Visits per Patient Among Direct Scheduling Adopters

The y-axis uses a log scale.

Figure 2.  Time of Day Visit Scheduled for Directly vs Usually Scheduled Visits
Time of Day Visit Scheduled for Directly vs Usually Scheduled Visits
Table 1.  Characteristics of Direct Scheduling Adopters and Nonadopters
Characteristics of Direct Scheduling Adopters and Nonadopters
Table 2.  Distribution of Most Common Primary Diagnoses Among Directly Scheduled and Usually Scheduled Visits
Distribution of Most Common Primary Diagnoses Among Directly Scheduled and Usually Scheduled Visits
1.
Office of the National Coordinator for Health IT. Health care professional EHR health IT developers. Updated July 2017. Accessed January 24, 2018. https://dashboard.healthit.gov/quickstats/pages/FIG-Vendors-of-EHRs-to-Participating-Professionals.php
2.
McNeill  SM.  Lower your overhead with a patient portal.   Fam Pract Manag. 2016;23(2):21-25. Accessed January 24, 2018. https://www.aafp.org/fpm/2016/0300/p21.html.PubMedGoogle Scholar
4.
NHS Digital. NHS e-Referral Service. Accessed January 29, 2018. https://digital.nhs.uk/services/e-referral-service/
5.
Zhao  P, Yoo  I, Lavoie  J, Lavoie  BJ, Simoes  E.  Web-based medical appointment systems: A systematic review.   J Med Internet Res. 2017;19(4):e134. doi:10.2196/jmir.6747 PubMedGoogle Scholar
6.
Pradhan  A. Now, book your appointment with an AIIMS doctor online. Times of India. February 5, 2016. Accessed October 25, 2019. https://timesofindia.indiatimes.com/city/bhubaneswar/Now-book-your-appointment-with-an-AIIMS-doctor-online/articleshow/50860853.cms
7.
Nitkin  K; Johns Hopkins Medicine. Epic tools give patients power to schedule. Accessed January 24, 2018. https://www.hopkinsmedicine.org/news/articles/epic-tools-give-patients-power-to-schedule
8.
Anthony  DL, Campos-Castillo  C, Lim  PS.  Who isn’t using patient portals and why: evidence and implications from a national sample of US adults.   Health Aff (Millwood). 2018;37(12):1948-1954. doi:10.1377/hlthaff.2018.05117 PubMedGoogle ScholarCrossref
10.
Habibi  MRM, Mohammadabadi  F, Tabesh  H, Vakili-Arki  H, Abu-Hanna  A, Eslami  S.  Effect of an online appointment scheduling system on evaluation metrics of outpatient scheduling system: a before-after multicenter study.   J Med Syst. 2019;43(8):281. doi:10.1007/s10916-019-1383-5 PubMedGoogle ScholarCrossref
11.
Yamin  CK, Emani  S, Williams  DH,  et al.  The digital divide in adoption and use of a personal health record.   Arch Intern Med. 2011;171(6):568-574. doi:10.1001/archinternmed.2011.34 PubMedGoogle ScholarCrossref
12.
Kontos  E, Blake  KD, Chou  W-YS, Prestin  A.  Predictors of eHealth usage: insights on the digital divide from the Health Information National Trends Survey 2012.   J Med Internet Res. 2014;16(7):e172. doi:10.2196/jmir.3117 PubMedGoogle Scholar
13.
Parmar  V, Large  A, Madden  C, Das  V.  The online outpatient booking system ‘Choose and Book’ improves attendance rates at an audiology clinic: a comparative audit.   Inform Prim Care. 2009;17(3):183-186. doi:10.14236/jhi.v17i3.733 PubMedGoogle Scholar
14.
Siddiqui  Z, Rashid  R.  Cancellations and patient access to physicians: ZocDoc and the evolution of e-medicine.   Dermatol Online J. 2013;19(4):14.PubMedGoogle Scholar
15.
Zhang  X, Yu  P, Yan  J, Ton A M Spil  I.  Using diffusion of innovation theory to understand the factors impacting patient acceptance and use of consumer e-health innovations: a case study in a primary care clinic.   BMC Health Serv Res. 2015;15:71. doi:10.1186/s12913-015-0726-2PubMedGoogle ScholarCrossref
16.
Friedman  JP.  Internet patient scheduling in real-life practice.   J Med Pract Manage. 2004;20(1):13-15.PubMedGoogle Scholar
17.
Brandenburg  L, Gabow  P, Steele  G, Toussaint  J, Tyson  BJ. Innovation and best practices in health care scheduling. Accessed January 24, 2018. https://nam.edu/wp-content/uploads/2015/06/SchedulingBestPractices.pdf
18.
Kumar  P, Soni  V, Sutaria  S. The access imperative. Accessed January 24, 2018. https://healthcare.mckinsey.com/wp-content/uploads/2020/02/McK-white-paper_Patient-Access_Nov-2014.pdf
19.
Cao  W, Wan  Y, Tu  H,  et al.  A web-based appointment system to reduce waiting for outpatients: a retrospective study.   BMC Health Serv Res. 2011;11:318. doi:10.1186/1472-6963-11-318 PubMedGoogle ScholarCrossref
20.
Jones  R, Menon-Johansson  A, Waters  AM, Sullivan  AK.  eTriage—a novel, web-based triage and booking service: enabling timely access to sexual health clinics.   Int J STD AIDS. 2010;21(1):30-33. doi:10.1258/ijsa.2008.008466 PubMedGoogle ScholarCrossref
21.
Friedberg  MW, Hussey  PS, Schneider  EC.  Primary care: a critical review of the evidence on quality and costs of health care.   Health Aff (Millwood). 2010;29(5):766-772. doi:10.1377/hlthaff.2010.0025 PubMedGoogle ScholarCrossref
22.
Starfield  B, Shi  L, Macinko  J.  Contribution of primary care to health systems and health.   Milbank Q. 2005;83(3):457-502. doi:10.1111/j.1468-0009.2005.00409.x PubMedGoogle ScholarCrossref
23.
Basu  S, Berkowitz  SA, Phillips  RL, Bitton  A, Landon  BE, Phillips  RS.  Association of primary care physician supply with population mortality in the United States, 2005-2015.   JAMA Intern Med. 2019;179(4):506-514. doi:10.1001/jamainternmed.2018.7624 PubMedGoogle ScholarCrossref
25.
Brown  EJ, Polsky  D, Barbu  CM, Seymour  JW, Grande  D.  Racial disparities in geographic access to primary care in Philadelphia.   Health Aff (Millwood). 2016;35(8):1374-1381. doi:10.1377/hlthaff.2015.1612 PubMedGoogle ScholarCrossref
26.
Shi  L, Green  LH, Kazakova  S.  Primary care experience and racial disparities in self-reported health status.   J Am Board Fam Pract. 2004;17(6):443-452. doi:10.3122/jabfm.17.6.443 PubMedGoogle ScholarCrossref
27.
Lin  SC, Lyles  CR, Sarkar  U, Adler-Milstein  J.  Are patients electronically accessing their medical records: evidence from national hospital data.   Health Aff (Millwood). 2019;38(11):1850-1857. doi:10.1377/hlthaff.2018.05437 PubMedGoogle ScholarCrossref
28.
Boulware  LE, Barnes  GJ, Wilson  RF,  et al.  Value of the periodic health evaluation.   Evid Rep Technol Assess (Full Rep). 2006;(136):1-134.PubMedGoogle Scholar
29.
Althouse  AD.  Adjust for multiple comparisons: it’s not that simple.   Ann Thorac Surg. 2016;101(5):1644-1645. doi:10.1016/j.athoracsur.2015.11.024 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    2 Comments for this article
    EXPAND ALL
    High % of white patients vs community average
    Henry Wei, MD | Henry Wei MD
    US Census Data show that the local population is 73% White for the Greater Boston Combined Statistical Area (CSA).

    Table 1 shows that the entire population studied is about 80% White.

    How might this higher proportion of White patients in this medical practice population (vs the community proportion) impact the results of the adopter vs non-adopter comparisons by White vs non-White patients?

    Are there any potential hidden variables that would lead fewer non-White people to use this medical practice in general, agnostic to scheduling offerings?
    CONFLICT OF INTEREST: None Reported
    Appointment Scheduling - Implications for Wider Adoption
    Cimarron Buser, MBA | TASBIA-The Appointment Scheduling & Booking Industry Association
    This article addresses the issue that "Direct Scheduling", i.e., the ability of consumers (patients) to schedule a date and time via a web browser or mobile app, is growing in popularity. Especially with Covid-19, personal visits to retail establishments, banks, personal care (e.g., hair cuts), and many other previous "walk in" activities are increasingly being made with self-service scheduling.

    To that end, the research is important not only for the conclusions made for primary care appointments, but for appointments that members of the community need to make for many other activities.

    I'd like to address the overall conclusion
    that direct scheduling users "were more likely to be young, White, and commercially insured, and to the extent these differences persist as direct scheduling is used more widely, this service may widen socioeconomic disparities in primary care access."

    First, the statement that early adopters were "more likely to be young" aligns with findings from other business verticals. Use of technology, and especially mobile apps, is widespread, but especially so with a younger demographic. It would be interesting to see what the split between mobile and non-mobile appointments was using your data. One area that is difficult to measure is the ease-of-use of the user interface, and how this can impact the rate of use.

    Second, the statement that early adopters were "more likely commercially insured" begs the question as to why this is the case. One possibility is whether individuals who do not have insurance are less likely to use direct scheduling due to concerns about payment (or other reasons). If this is the case, they may want to speak directly to someone at the office rather than "take a chance" on making an appointment with an unknown financial outcome.

    Finally, is it possible that the reason that early adopters were "more likely to be White" may be directly correlated to the overall racial disparities in health insurance coverage?

    See "Racial and Ethnic Disparities in Health Insurance Coverage: Dynamics of Gaining and Losing Coverage over the Life-Course" (Popul Res Policy Rev. 2017 Apr; 36(2): 181–201), https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5370590/

    This correlation could potentially be a "hidden variable" mentioned by Dr. Henry Wei in the comment posted on August 29 ("High % of white patients vs community average") and could be worth exploring.
    CONFLICT OF INTEREST: None Reported
    READ MORE
    Original Investigation
    Health Policy
    August 27, 2020

    Patient and Visit Characteristics Associated With Use of Direct Scheduling in Primary Care Practices

    Author Affiliations
    • 1Harvard Medical School, Boston, Massachusetts
    • 2Division of General Internal Medicine, Brigham and Women’s Hospital, Boston, Massachusetts
    • 3Department of Biostatistics, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
    • 4Division of General Internal Medicine, Massachusetts General Hospital, Boston
    • 5Mass General Brigham, Boston, Massachusetts
    JAMA Netw Open. 2020;3(8):e209637. doi:10.1001/jamanetworkopen.2020.9637
    Key Points

    Question  What characteristics are associated with the online portal–based scheduling of medical visits?

    Findings  In this cross-sectional study of 62 080 patients and 134 225 completed visits at 17 primary care practices within a large academic medical center, early adopters of direct scheduling were more often young, White, and commercially insured. Compared with visits scheduled by speaking with clinic staff in person or by telephone, directly scheduled visits were more likely to be with one’s own primary care physician.

    Meaning  These findings suggest that direct scheduling may contribute to primary care continuity and that if greater adoption by younger, White, commercially insured patients persists, this service may widen socioeconomic disparities in primary care access.

    Abstract

    Importance  Medical practices increasingly allow patients to schedule their own visits through online patient portals, yet little is known about who adopts direct scheduling or how this service is used.

    Objective  To determine patient and visit characteristics associated with direct scheduling, visit patterns, and potential implications for access and continuity in the primary care setting.

    Design, Setting, and Participants  This cross-sectional study used electronic health record (EHR) data from 17 adult primary care practices in a large academic medical center in the Boston, Massachusetts, area. Participants included patients 18 years or older who were attributed in the EHR to an active primary care physician at 1 of the included primary care practices, were enrolled in the patient portal, and had at least 1 visit to 1 of these practices between March 1, 2018, and March 1, 2019, the period of analysis. Data were analyzed from October 25, 2019, to April 14, 2020.

    Main Outcomes and Measures  Adoption of direct scheduling, defined as at least 1 use during the study period. Usual scheduling was defined as scheduling with clinic staff by telephone or in person.

    Results  We examined 134 225 completed visits by 62 080 patients (mean [SD] age, 51.1 [16.4] years, 37 793 [60.9%] women) attributed to 140 primary care physicians at 17 primary care practices. A total of 5020 patients (8.1% [95% CI, 7.9%-8.3%]) adopted direct scheduling, with an age range of 18 to 95 years. Compared with nonadopters in the same practices, adopters were younger (adjusted odds ratio [AOR] per additional year, 0.98 [95% CI, 0.98-0.99]) and were more likely to be White (AOR, 1.09 [95% CI, 1.01-1.17]) and commercially insured (AOR vs uninsured, 1.40 [95% CI, 1.11-1.76]) and to have more comorbidities (AOR per additional comorbidity, 1.07 [95% CI, 1.04-1.11]). Compared with usually scheduled visits, directly scheduled visits were more likely to be for general medical examinations (1979 visits [36.7%] vs 26 519 visits [21.9%]; P < .001) and with one’s own primary care physician (5267 visits [95.2%] vs 94 634 visits [73.5%]; P < .001).

    Conclusions and Relevance  These findings suggest that direct scheduling was associated with greater primary care continuity. Early adopters were more likely to be young, White, and commercially insured, and to the extent these differences persist as direct scheduling is used more widely, this service may widen socioeconomic disparities in primary care access.

    Introduction

    Medical practices increasingly allow patients to schedule their own office visits through online patient portals, otherwise known as direct scheduling. This option is offered by the largest electronic health record (EHR) vendors and via external health care applications in the United States and internationally,1-8 which is consistent with trends toward self-service in most other industries. In a 2013 survey,9 77% of US respondents said it was important to them that their medical provider offered online appointment booking; however, only 43% of respondents said that direct scheduling was offered in some form at the time.

    Direct scheduling is intended to improve patient convenience while reducing administrative burden for practices.5,10 This offering may have additional benefits, especially in the primary care setting, such as promoting continuity with one’s usual primary care physician (PCP). Conversely, direct scheduling might worsen disparities in access to care via the so-called digital divide.8,11,12 Small studies of individual clinics and hospitals in the US, United Kingdom, Australia, and China provide early evidence that this approach may be associated with lower no-show rates13,14 and the ability to schedule visits outside of usual business hours.15,16 Yet there is little rigorous research on who uses direct scheduling, how it is used, or any unintended consequences.17,18

    By January 2018, a large academic medical center in the Boston, Massachusetts, area had introduced direct scheduling across 17 adult primary care practices, providing an opportunity to study early adoption and consequences associated with this approach. We used EHR data to identify patient characteristics associated with direct scheduling adoption, how direct scheduling was used, and potential implications for primary care access and continuity.

    Methods

    The Partners HealthCare institutional review board approved this study and waived the use of informed consent per 45 CFR 46.116(f). The study follows the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

    Prestudy Exploratory Survey

    To inform our analytical plan, in 2017 and 2018 we purposively sampled 30 clinical and administrative leaders at practices that offered or were planning to offer direct scheduling. We filled in additional details through personal communication with the operational team (eAppendix 1 and eTable 1 in the Supplement).

    Implementation Details

    Practices staggered the introduction of direct scheduling between October 2016 and January 2018 (eTable 2 in the Supplement). The direct scheduling service was available through the Epic Systems Corporation patient portal desktop or mobile application and allowed any portal user to schedule a problem-based, follow-up, or annual physical examination appointment in the subsequent 2 to 90 days with any physician or nurse practitioner whom they had previously seen. Users indicated their preferred dates and times, then chose from the time slots made available by the practice or individual clinician. Patients at these practices also had the option of usual scheduling, which involved speaking to a staff member at the front desk by telephone or in person and, in some scenarios, being transferred to a nurse.

    Prior to implementation, most practices educated clinicians and staff members about direct scheduling, encouraged patient portal enrollment, and adjusted scheduling templates and front desk workflows to accommodate the new offering. Most practices advertised direct scheduling by sending secure message blasts through the patient portal and displaying flyers or cards in the clinic reception area. Most practices reviewed direct scheduling bookings at least daily to confirm safety and appropriateness.

    In the survey, some practice leaders reported concerns that with direct scheduling, patients might schedule unnecessary visits or too many visits at a time. Most practice leaders anticipated benefits, including lower no-show rates, decreased front-desk workload, and improved patient convenience (eTable 1 in the Supplement).

    Data Source

    We used the health system’s Enterprise Data Warehouse to extract data from March 1, 2017, to March 1, 2019. The Enterprise Data Warehouse includes patient demographic information collected during registration and detailed clinical encounter and billing data from the EHR.

    Patient and Visit Cohorts

    We identified a cohort of adult patients aged 18 years or older who were attributed (defined by the EHR PCP field) to an active PCP (a physician with specialty in internal or family medicine providing at least 50 visits during calendar year 2018) at 1 of 17 included primary care practices, were enrolled in the patient portal as of March 1, 2019 (defined by at least 1 portal log-in within the previous 2 years), and had at least 1 completed visit to 1 of these practices between March 1, 2018, and March 1, 2019. We examined all completed visits by these patients to the included practices between March 1, 2018, and March 1, 2019 (the study period), excluding any visits that had been scheduled before a given practice offered direct scheduling (eAppendix 2 in the Supplement).

    Primary Outcome

    We defined adopters as patients who scheduled at least 1 primary care visit through direct scheduling during our study period. Nonadopters were defined as those who had at least 1 appointment during the study period but did not use direct scheduling to schedule an appointment.

    Patient Characteristics

    For all patients, we captured data on age, sex, race/ethnicity, comorbidity count (range, 0-6; including hypertension, diabetes, chronic kidney disease, depression, asthma, or obesity), and number of within–health system visits (not limited to primary care) and number of within–health system hospitalizations in the year prior to the study period (ie, March 1, 2017, to February 28, 2018). We cross-walked patients’ zip codes to American Community Survey data to measure area-level education (categorized based on percentage of residents with high school education or above) and income (categorized as <200%, 200% to <400%, or ≥400% of the 2017 federal poverty level for a family of 4). Among adopters, we assessed each patient’s number of directly scheduled visits during the study period.

    Practice Characteristics

    We determined the number of months each practice had offered direct scheduling prior to the start of the study period (range, 2-17 months). Next, we measured the percentage of patients in each practice who had adopted direct scheduling during the study period.

    Visit Characteristics

    We assessed visit characteristics, including whether the visit was usually or directly scheduled, time of day the visit was scheduled, and time elapsed between visit scheduling and occurrence. We ranked the top 30 primary diagnosis codes associated with directly and usually scheduled visits, grouped any codes corresponding to identical diagnoses, reranked the diagnoses, and presented the top 15 diagnoses in each group. We further assessed visit billing level (among those with Evaluation and Management Current Procedural Terminology billing codes 99211-99215) and the clinician providing the visit (dichotomized as patient’s attributed PCP vs other).

    Statistical Analysis

    We performed descriptive patient-, practice-, and visit-level analyses, including adoption rates overall and by primary care practice. For our primary analysis, we compared direct scheduling adopters vs nonadopters in univariate analyses using the χ2 test for categorical data and the t test or Wilcoxon test for continuous variables. To identify factors associated with adoption, we built a patient-level multivariable logistic regression model that included patient characteristics and practice fixed effects.

    In secondary analyses, we used linear regression to assess the association of the length of time practices had been offering direct scheduling with their patients’ adoption rates. We used the Wilcoxon rank-sum test to compare time from visit scheduling to occurrence and the t test and χ2 test to compare other characteristics of directly and usually scheduled visits overall and among direct scheduling adopters specifically. We used Poisson regression to analyze the number of visits that adopters had with their own PCPs compared with nonadopters. The model included an offset for each patient’s total number of visits so that it effectively compared the percentage of visits for which patients were seen by their own PCP.

    We also created a hierarchical linear probability model to determine the relative magnitude of patient-, physician-, and practice-level components of variance in whether a given visit was usually or directly scheduled. Reported P values were 2-sided and considered significant at P < .05. All analyses were performed with Stata statistical software version 14.2 (StataCorp). Data were analyzed from October 25, 2019, to April 14, 2020.

    Results

    Our study sample included 17 primary care practices, 140 PCPs, 62 080 patients, and 134 225 visits. Patients had a mean (SD) age of 51.1 (16.4) years and included 37 793 (60.9%) women.

    Patient-Level Analysis

    Among the 62 080 patients included, 5020 (8.1% [95% CI, 7.9%-8.3%]) used direct scheduling during the study period (age range, 18-95 years) (Table 1). In univariate analyses, adopters, compared with nonadopters, were younger (mean [SE] age, 45.4 [0.2] years vs 51.6 [0.1] years; P < .001) and enrolled in commercial insurance (4289 adopters [85.4%] vs 41 264 nonadopters [72.3%]; P < .001). Additionally, adopters had fewer comorbidities (mean (SE), 1.03 [0.02] comorbidities vs 1.18 [<0.01] comorbidities; P < .001); they resided in areas with lower income (<200% FPL: 217 adopters [4.3%] vs 2018 nonadopters [3.5%]; 200% to <400% FPL: 2836 adopters [56.5%] vs 32 972 nonadopters [57.8%]; ≥400% FPL: 1965 adopters [39.2%] vs 22 014 nonadopters [38.6%]; P < .001) and higher education levels (lowest: 1475 adopters [29.4%] vs 18 691 nonadopters [32.8%]; medium: 1791 adopters [35.7%] vs 19 216 nonadopters [33.7%]; highest: 1754 adopters [34.9%] vs 19 129 nonadopters [33.5%]; P < .001). Adopters also had fewer office visits in the prior year (median [interquartile range], 3 [1-7] visits vs 4 [2-8] visits]; P < .001) and were less likely to have been hospitalized in the prior year (215 adopters [4.3%] vs 3302 nonadopters [5.8%]; P < .001). In the multivariable model with practice fixed effects, using direct scheduling was associated with younger age (adjusted odds ratio [AOR] per additional year, 0.98 [95% CI, 0.98-0.99]), White race (AOR, 1.09 [95% CI, 1.01-1.17]), commercial insurance (AOR vs uninsured, 1.40 [95% CI, 1.11-1.76]), and more comorbidities (AOR per additional comorbidity, 1.08 [95% CI, 1.04-1.11]). Furthermore, having more prior visits was associated with adoption (AOR, 1.00 [95% CI, 1.00-1.01]). Most adopters directly scheduled a total of 1 (4602 patients [91.7%]) or 2 (361 patients [7.2%]) visits during the study period (Figure 1). Adopters had a 11.1% (95% CI 8.8%-13.5%]) higher rate of visits with their own PCPs than nonadopters.

    Practice Analysis

    Adoption rates differed widely across practices (range, 1.3%-18.9%). Practices that had been offering direct scheduling for a longer time had higher practice-wide adoption rates during the study period (0.93% [95% CI, 0.44%-1.41%] increase in practice adoption for every additional month of direct scheduling being offered) (eFigure 1 in the Supplement).

    Visit Analysis

    During the study period, 5531 of 134 225 visits (4.1% [95% CI, 4.0%-4.2%]) were directly scheduled. Of directly scheduled visits, 4331 visits (78.3%) were scheduled during usual business hours (ie, 8 am-5 pm) (Figure 2). Directly scheduled visits had longer wait times from scheduling to occurrence (median [interquartile range], 20 [7-51] days vs 14 [1-66] days; P < .001). At the visit level, directly scheduled visits were more likely than usually scheduled visits to carry a primary diagnosis of general medical examination (1979 visits [36.7%] vs 26 519 visits [21.9%], P < .001) (Table 2). Among problem-based visits for established patients, directly scheduled visits were less likely than usually scheduled visits to be billed at the highest complexity level (Evaluation and Management code 99215) (80 visits [3.1%] vs 4083 visits [5.1%]; P < .001) (eFigure 2 in the Supplement). Directly scheduled visits were more likely than usually scheduled visits to be with the patient’s own PCP (5267 visits [95.2%] vs 94 634 visits [73.5%]; P < .001).

    Among adopters specifically, 5531 of 11 434 visits (48.4% [95% CI, 47.5%-49.3%]) were directly scheduled. When we compared directly scheduled to usually scheduled visits among adopters, directly scheduled visits were still more likely to carry a primary diagnosis of general medical examination (1979 visits [36.7%] vs 792 visits [14.2%]; P < .001) and more likely to be with the patient’s own PCP (4103 visits [95.2%] vs 5267 visits [69.5%]; P < .001). In the hierarchical model, patient factors accounted for 22.0% of the variance in use of direct vs usual scheduling, more than practice (2.5%) or PCP (1.0%) factors, while 74.6% of the variance remained unexplained.

    Discussion

    This cross-sectional study across 17 large primary care practices offering direct scheduling found that early adopters of the patient portal feature were more likely to be younger and White and to have commercial insurance, more comorbidities, and higher prior utilization compared with nonadopters in the same practices. Prior work has reported mixed results on sex and age.13,14,19,20 Most directly scheduled visits were scheduled with a patient’s own PCP and during usual business hours. Compared with nonadopters, direct scheduling adopters had higher rates of visits with their own PCP.

    Our results suggest that direct scheduling may contribute to continuity and access, core aspects of high-functioning primary care that are associated with better health outcomes and lower costs.21-23 Directly scheduled visits were more likely than usually scheduled visits to take place with a patient’s own PCP, even when comparing these visit types among direct scheduling adopters, likely in part due to the design feature requiring patients to schedule their appointment with a clinician they had seen before. We also found evidence that patients might find direct scheduling more convenient than usual scheduling: most directly scheduled visits were scheduled during usual business hours when patients could have called the office yet chose to schedule online. This finding substantiates concerns that usual scheduling processes can be prohibitively cumbersome—by one estimate, it took a mean of 8.1 minutes to schedule a doctor’s appointment by telephone, with calls transferred 63% of the time.24

    At the same time, our findings raise the possibility that direct scheduling might contribute to disparities in primary care access. As direct scheduling is more widely adopted, its disproportionate use among younger, White, commercially insured patients may crowd out visit access for older patients and patients in racial/ethnic minority groups with potentially greater need for, or historically decreased access to, primary care.25,26 These findings are particularly worrisome given that we restricted our sample to patients who had enrolled in the patient portal. Prior studies show that members of racial/ethnic minority groups and other historically underserved populations are less likely to enroll in patient portals in the first place.8,11 Even among patients with portal access, those who are socioeconomically disadvantaged are less likely to use the technology.8,27

    Furthermore, while practice leaders’ concerns about patients directly scheduling too many visits were largely unfounded (99% of adopters scheduled just 1 or 2 visits in a year), our results leave open the possibility of more unnecessary visits. Directly scheduled visits were less likely than usually scheduled visits to be billed at the highest complexity level. They were also more likely to represent general medical examinations (partly a consequence of the design), which are of uncertain value among younger, healthy adults.28 More reassuringly, direct scheduling adopters had slightly more comorbidities and therefore, perhaps greater need for the service, consistent with prior work showing higher rates of comorbidities among portal adopters in general.11

    In addition to continuity and access considerations, direct scheduling theoretically offers a third benefit of practice efficiency. While we did not directly examine office staff workload or use of clinician schedules in this study, more recent (ie, July-September 2019) internal operational data of all visits from these 17 practices show that directly scheduled visits had higher cancellation rates than usually scheduled visits (4-57 percentage points higher than for usually scheduled visits across practices), consistent with the possibility that patients may work the schedule by canceling and rescheduling as new options arise or their personal schedules change. For all but 1 of 17 practices, these operational data show directly scheduled visits had the same or lower no-show rates (0-8 percentage points fewer no-show visits) (Mariela Arnal Istillarte, MSIE [Massachusetts General Hospital] email, December 2, 2019).

    Limitations

    This study has several limitations. It examines a single academic medical center, albeit a large center with 17 primary care sites covering more than 65 000 patients. We required study participants to be enrolled in the patient portal, since this is a prerequisite to using the direct scheduling application. Of note, internal, operational data show that 50% of primary care patients across the larger health system were active portal users as of December 2019 and that these users were more often women, English-speaking, and White compared with nonusers (Sarah Wilkie, MS [Mass General Brigham], email, January 6, 2020; eTable 3 in the Supplement). There was heterogeneity among practices, but we accounted for this using practice fixed effects. Additionally, any interpretation of statistical significance should be cautious of false-positive results due to multiple testing.29

    Conclusions

    The findings of this cross-sectional study suggest that that direct scheduling may promote continuity and convenience but also has the potential to widen disparities in primary care access. Since widespread adoption of direct scheduling is likely inevitable, in keeping with other service industries, it will be important to evaluate any associations with health outcomes over time.

    Back to top
    Article Information

    Accepted for Publication: April 27, 2020.

    Published: August 27, 2020. doi:10.1001/jamanetworkopen.2020.9637

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Ganguli I et al. JAMA Network Open.

    Corresponding Author: Ishani Ganguli, MD, MPH, Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, 1620 Tremont St, 3rd Floor, Boston, MA 02120 (iganguli@bwh.harvard.edu).

    Author Contributions: Dr Ganguli had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Ganguli, Metlay, Sequist.

    Acquisition, analysis, or interpretation of data: Ganguli, Orav, Lupo.

    Drafting of the manuscript: Ganguli, Lupo.

    Critical revision of the manuscript for important intellectual content: Ganguli, Orav, Metlay, Sequist.

    Statistical analysis: Ganguli, Orav, Lupo.

    Administrative, technical, or material support: Lupo.

    Supervision: Metlay, Sequist.

    Conflict of Interest Disclosures: Dr Ganguli reported receiving personal fees from Haven and Blue Cross Blue Shield MA and grants from International Business Machines outside the submitted work. No other disclosures were reported.

    Additional Contributions: Craig J. Cochran, MPA; Sandra Santos, BS; S. Dominic Aranha, MEM; Virginia Manzella, MS, MBA; and Mariela Arnal Istillarte, MSIE (Massachusetts General Hospital), and Jill M. Fiore, BS; Harrison T. Hubbell, BS; and Sarah Wilkie, MS (Mass General Brigham), helped with extracting data and providing operational details. They were not compensated for their work.

    References
    1.
    Office of the National Coordinator for Health IT. Health care professional EHR health IT developers. Updated July 2017. Accessed January 24, 2018. https://dashboard.healthit.gov/quickstats/pages/FIG-Vendors-of-EHRs-to-Participating-Professionals.php
    2.
    McNeill  SM.  Lower your overhead with a patient portal.   Fam Pract Manag. 2016;23(2):21-25. Accessed January 24, 2018. https://www.aafp.org/fpm/2016/0300/p21.html.PubMedGoogle Scholar
    4.
    NHS Digital. NHS e-Referral Service. Accessed January 29, 2018. https://digital.nhs.uk/services/e-referral-service/
    5.
    Zhao  P, Yoo  I, Lavoie  J, Lavoie  BJ, Simoes  E.  Web-based medical appointment systems: A systematic review.   J Med Internet Res. 2017;19(4):e134. doi:10.2196/jmir.6747 PubMedGoogle Scholar
    6.
    Pradhan  A. Now, book your appointment with an AIIMS doctor online. Times of India. February 5, 2016. Accessed October 25, 2019. https://timesofindia.indiatimes.com/city/bhubaneswar/Now-book-your-appointment-with-an-AIIMS-doctor-online/articleshow/50860853.cms
    7.
    Nitkin  K; Johns Hopkins Medicine. Epic tools give patients power to schedule. Accessed January 24, 2018. https://www.hopkinsmedicine.org/news/articles/epic-tools-give-patients-power-to-schedule
    8.
    Anthony  DL, Campos-Castillo  C, Lim  PS.  Who isn’t using patient portals and why: evidence and implications from a national sample of US adults.   Health Aff (Millwood). 2018;37(12):1948-1954. doi:10.1377/hlthaff.2018.05117 PubMedGoogle ScholarCrossref
    10.
    Habibi  MRM, Mohammadabadi  F, Tabesh  H, Vakili-Arki  H, Abu-Hanna  A, Eslami  S.  Effect of an online appointment scheduling system on evaluation metrics of outpatient scheduling system: a before-after multicenter study.   J Med Syst. 2019;43(8):281. doi:10.1007/s10916-019-1383-5 PubMedGoogle ScholarCrossref
    11.
    Yamin  CK, Emani  S, Williams  DH,  et al.  The digital divide in adoption and use of a personal health record.   Arch Intern Med. 2011;171(6):568-574. doi:10.1001/archinternmed.2011.34 PubMedGoogle ScholarCrossref
    12.
    Kontos  E, Blake  KD, Chou  W-YS, Prestin  A.  Predictors of eHealth usage: insights on the digital divide from the Health Information National Trends Survey 2012.   J Med Internet Res. 2014;16(7):e172. doi:10.2196/jmir.3117 PubMedGoogle Scholar
    13.
    Parmar  V, Large  A, Madden  C, Das  V.  The online outpatient booking system ‘Choose and Book’ improves attendance rates at an audiology clinic: a comparative audit.   Inform Prim Care. 2009;17(3):183-186. doi:10.14236/jhi.v17i3.733 PubMedGoogle Scholar
    14.
    Siddiqui  Z, Rashid  R.  Cancellations and patient access to physicians: ZocDoc and the evolution of e-medicine.   Dermatol Online J. 2013;19(4):14.PubMedGoogle Scholar
    15.
    Zhang  X, Yu  P, Yan  J, Ton A M Spil  I.  Using diffusion of innovation theory to understand the factors impacting patient acceptance and use of consumer e-health innovations: a case study in a primary care clinic.   BMC Health Serv Res. 2015;15:71. doi:10.1186/s12913-015-0726-2PubMedGoogle ScholarCrossref
    16.
    Friedman  JP.  Internet patient scheduling in real-life practice.   J Med Pract Manage. 2004;20(1):13-15.PubMedGoogle Scholar
    17.
    Brandenburg  L, Gabow  P, Steele  G, Toussaint  J, Tyson  BJ. Innovation and best practices in health care scheduling. Accessed January 24, 2018. https://nam.edu/wp-content/uploads/2015/06/SchedulingBestPractices.pdf
    18.
    Kumar  P, Soni  V, Sutaria  S. The access imperative. Accessed January 24, 2018. https://healthcare.mckinsey.com/wp-content/uploads/2020/02/McK-white-paper_Patient-Access_Nov-2014.pdf
    19.
    Cao  W, Wan  Y, Tu  H,  et al.  A web-based appointment system to reduce waiting for outpatients: a retrospective study.   BMC Health Serv Res. 2011;11:318. doi:10.1186/1472-6963-11-318 PubMedGoogle ScholarCrossref
    20.
    Jones  R, Menon-Johansson  A, Waters  AM, Sullivan  AK.  eTriage—a novel, web-based triage and booking service: enabling timely access to sexual health clinics.   Int J STD AIDS. 2010;21(1):30-33. doi:10.1258/ijsa.2008.008466 PubMedGoogle ScholarCrossref
    21.
    Friedberg  MW, Hussey  PS, Schneider  EC.  Primary care: a critical review of the evidence on quality and costs of health care.   Health Aff (Millwood). 2010;29(5):766-772. doi:10.1377/hlthaff.2010.0025 PubMedGoogle ScholarCrossref
    22.
    Starfield  B, Shi  L, Macinko  J.  Contribution of primary care to health systems and health.   Milbank Q. 2005;83(3):457-502. doi:10.1111/j.1468-0009.2005.00409.x PubMedGoogle ScholarCrossref
    23.
    Basu  S, Berkowitz  SA, Phillips  RL, Bitton  A, Landon  BE, Phillips  RS.  Association of primary care physician supply with population mortality in the United States, 2005-2015.   JAMA Intern Med. 2019;179(4):506-514. doi:10.1001/jamainternmed.2018.7624 PubMedGoogle ScholarCrossref
    25.
    Brown  EJ, Polsky  D, Barbu  CM, Seymour  JW, Grande  D.  Racial disparities in geographic access to primary care in Philadelphia.   Health Aff (Millwood). 2016;35(8):1374-1381. doi:10.1377/hlthaff.2015.1612 PubMedGoogle ScholarCrossref
    26.
    Shi  L, Green  LH, Kazakova  S.  Primary care experience and racial disparities in self-reported health status.   J Am Board Fam Pract. 2004;17(6):443-452. doi:10.3122/jabfm.17.6.443 PubMedGoogle ScholarCrossref
    27.
    Lin  SC, Lyles  CR, Sarkar  U, Adler-Milstein  J.  Are patients electronically accessing their medical records: evidence from national hospital data.   Health Aff (Millwood). 2019;38(11):1850-1857. doi:10.1377/hlthaff.2018.05437 PubMedGoogle ScholarCrossref
    28.
    Boulware  LE, Barnes  GJ, Wilson  RF,  et al.  Value of the periodic health evaluation.   Evid Rep Technol Assess (Full Rep). 2006;(136):1-134.PubMedGoogle Scholar
    29.
    Althouse  AD.  Adjust for multiple comparisons: it’s not that simple.   Ann Thorac Surg. 2016;101(5):1644-1645. doi:10.1016/j.athoracsur.2015.11.024 PubMedGoogle ScholarCrossref
    ×