Customize your JAMA Network experience by selecting one or more topics from the list below.
Smith PC, Araya-Guerra R, Bublitz C, et al. Missing Clinical Information During Primary Care Visits. JAMA. 2005;293(5):565–571. doi:10.1001/jama.293.5.565
Author Affiliations: Department of Family Medicine,
University of Colorado Health Sciences Center, Denver.
Context The coordinating function of primary care is information-intensive and
may be impeded by missing clinical information. However, missing clinical
information has not been explicitly investigated in the primary care setting.
Objective To describe primary care clinicians’ reports of missing clinical
Design, Setting, and Participants Cross-sectional survey conducted in 32 primary care clinics within State
Networks of Colorado Ambulatory Practices and Partners (SNOCAP), a consortium
of practice-based research networks participating in the Applied Strategies
for Improving Patient Safety medical error reporting study. Two hundred fifty-three
clinicians were surveyed about 1614 patient visits between May and December
2003. For every visit during 1 half-day session, each clinician completed
a questionnaire about patient and visit characteristics and stated whether
important clinical information had been missing. Clinician characteristics
were also recorded.
Main Outcome Measures Reports of missing clinical information frequency, type, and presumed
location; perceived likelihood of adverse effects, delays in care, and additional
services; and time spent looking for missing information. Multivariate analysis
was conducted to assess the relationship of missing information to patient,
visit, or clinician characteristics, adjusting for potential confounders and
effects of clustering.
Results Clinicians reported missing clinical information in 13.6% of visits;
missing information included laboratory results (6.1% of all visits), letters/dictation
(5.4%), radiology results (3.8%), history and physical examination (3.7%),
and medications (3.2%). Missing clinical information was frequently reported
to be located outside their clinical system but within the United States (52.3%),
to be at least somewhat likely to adversely affect patients (44%), and to
potentially result in delayed care or additional services (59.5%). Significant
time was reportedly spent unsuccessfully searching for missing clinical information
(5-10 minutes, 25.6%; >10 minutes, 10.4%). After adjustment, reported missing
clinical information was more likely when patients were recent immigrants
(odds ratio [OR], 1.78; 95% confidence interval [CI], 1.06-2.99), new patients
(OR, 2.39; 95% CI, 1.70-3.35), or had multiple medical problems compared with
no problems (1 problem: OR, 1.09; 95% CI, 0.69-1.73; 2-5 problems: OR, 1.87;
95% CI, 1.21-2.89; >5 problems: OR, 2.78; 95% CI, 1.61-4.80). Missing clinical
information was less likely in rural practices (OR, 0.52; 95% CI, 0.29-0.92)
and when individual clinicians reported having full electronic records (OR,
0.40; 95% CI, 0.17-0.94).
Conclusions Primary care clinicians report that missing clinical information is
common, multifaceted, likely to consume time and other resources, and may
adversely affect patients. Additional research on missing information is needed
to focus on validating clinicians’ perceptions and on conducting prospective
studies of its causes and sequelae.
Effectively managing clinical information (patient information such
as demographics, medical history, medications, test results, and family structure)1 is an essential part of all medical care; it is particularly
crucial for primary care to be able to fulfill what the Institute of Medicine
and others consider to be its defining task of coordinating comprehensive
care across the health care system.2-7 Unfortunately,
multiple barriers complicate the collecting, synthesizing, recording, and
sharing of clinical information, including privacy regulations, decentralized
medical systems, inadequate interprofessional communication, the transfer
of patients’ care within and across care settings, and the rapid turnover
of patients’ insurance plans.8-14 Accordingly,
physicians may not have clinical information available when it is important
for a patient’s care.
Missing clinical information has been implicated in injurious adverse
et al9 reported that missing clinical information
was associated with 15.6% of all reported errors in primary care, most of
which were perceived by clinicians as likely to be harmful, and was implicated
in every major category of medical error. In the only research studying missing
clinical information directly,22 Canadian emergency
department physicians reported that 15.3% of visits had important information
missing at the time of the encounter that was very likely to result in patient
harm. Such harm could include otherwise avoidable drug interactions or duplications,
missed or delayed diagnoses, missed immunizations, unnecessary testing and
procedures, and the downstream effects of such events.23
Despite its potential impact on the essential coordination function
of primary care, missing clinical information has not yet been explicitly
investigated in this setting. To begin to describe this phenomenon, we surveyed
primary care clinicians about clinical information reported as missing during
patient care visits.
This study was conducted within the State Networks of Colorado Ambulatory
Practices and Partners (SNOCAP), a consortium of Colorado practices and practice-based
research networks. These include practices from the Colorado Research Network
(CaReNet) and the High Plains Research Network (HPRN). Although CaReNet focuses
on the care of underserved patients,24 it has
a diverse membership including academic, private, and community practices
and encompasses both private and publicly funded entities. HPRN settings are
in rural and frontier communities across northeastern Colorado.25 All
38 SNOCAP practices participating in the Applied Strategies for Improving
Patient Safety error reporting project26 were
invited to participate. Six practices with only 1 clinician were excluded
to protect anonymity, and first-year residents were excluded because they
were unlikely to be familiar with practice information systems. Clinicians
in CaReNet were surveyed between May and August 2003 and those in HPRN between
August and December 2003.
A 2-part cross-sectional survey of primary care clinicians was created
using a modified Delphi technique.27 For each
visit, an anonymous study questionnaire asked the clinician about patient
variables, including age and sex; whether the patient had moved to the United
States within the last 5 years; and the number of active medical problems.
The respondent was also asked whether this was the patient’s first visit
to the practice, if he or she was the patient’s usual primary clinician,
and “Do any communication barriers exist with this patient?” (a
broad question intended to include such barriers as language discrepancy,
severe dementia, and developmental delay). The clinician was asked to indicate
patient race (all that apply: white, black, Asian, Native American, do not
know) and ethnicity (Hispanic, non-Hispanic, do not know) to determine if
these variables were associated with missing clinical information.
The respondent was then asked, “Was any existing information,
important for the care of this patient, unavailable at the time of the visit?”
The questionnaire explained that this referred only to information known to
exist. The term “important for the care of this patient” was not
further defined but was intended to capture essential but not necessarily
urgent information. To study the entire scope of missing information, we included
information that might not always be reasonably expected to be available at
the visit. For example, we asked whether missing clinical information was
located outside the practice (eg, in the hospital or in another state) or
inside the practice (eg, a misplaced chart or malfunctioning electronic systems).
Because we wanted to assess information missing at the time that most medical
decisions are made, clinicians completed the questionnaire at the end of each
visit. Thus, clinical information initially missing but found prior to the
end of the visit was not classified as missing, whereas information found
after the visit had ended was still classified as missing.
If clinical information was reported missing, clinicians answered additional
questions pertaining to that information. They chose among nonmutually exclusive,
fixed-response options that also had an “other” option accompanied
by space for free text. These questions included (1) the type of information
reported as missing; (2) whether they thought the missing information likely
resided within or outside their clinical system (defined as their practice
and any associated hospital, university, or community health system) or within
or outside the United States; (3) whether, as a consequence of the information
being missing, they thought the patient was likely to have a delay in care
or require additional medical services; and (4) whether the clinician or a
staff member had attempted to find the information, and if not, why not. If
clinicians searched for the missing information but didn’t find it during
the visit, they were asked to estimate the time spent looking (<1, 1-4,
5-10, or >10 minutes). Finally, they recorded on a 5-point Likert scale their
estimate of “How likely is this missing information to adversely affect
the patient’s well being?”, with anchors ranging from “not
at all likely” to “very likely.” The questionnaire instructions
asked only that this be considered in the context of the patient’s medical
care but did not define “adversely affect.” These estimates of
adverse effects were not confirmed or otherwise characterized.
A second clinician questionnaire asked for the clinicians’ own
demographic information and specialty, whether they were physicians or midlevel
clinicians (nurse practitioner or physician assistant), and whether they were
residents. The questionnaire also asked the respondents to choose the single
best description of their practice’s information system: paper charts,
partial or hybrid electronic medical records (EMRs), or full (EMRs). Finally,
clinicians reported whether or not they had electronic access in their office
to patient data from their primary hospital. We did not assess the extent
to which each respondent used any existing electronic systems.
The survey was reviewed by experts in medical error and communication
to maximize face and content validity and was pilot tested by experienced
clinicians.28 The study questionnaire was limited
to 1 page to maximize response rate; average completion time was less than
1 minute. No patient or clinician identifying information was included on
the questionnaires, and the study was approved as exempt by the Colorado Multiple
Institutional Review Board and all necessary local institutional review boards.
Each participating clinician completed the study questionnaire at the
end of every consecutive patient visit during 1 half-day clinic session. Each
clinician also completed 1 anonymous clinician questionnaire. Recent preexisting
network surveys provided data on practice size, estimated by the number of
full-time equivalent clinicians at each practice. Network data were used to
determine which of these were residency practices to assess whether their
unique structure influenced missing clinical information independent of the
behavior of resident vs nonresident physicians. Because residents are frequently
away from the clinics, they were considered 0.3 full-time equivalents. The
month of data collection for each practice was recorded.
Missing clinical information rates, frequency distributions, and means
(SDs) were calculated for all variables of interest. The intraclass correlation
coefficient was computed to assess potential clustering effects. The intraclass
correlation coefficient for patients within physicians was 0.076, indicating
the need to use methods appropriate for clustered data. To determine whether
missing clinical information was associated with patient demographics, visit
characteristics, and practice or clinician factors, generalized linear mixed
models (multilevel models) were used with missing clinical information (yes/no)
as the outcome (logit link) to extend the traditional logistic regression
model to accommodate the hierarchical structure of the data (SAS Proc MIXED
with GLIMMIX macro).29 Variance components
at each level were examined to determine whether random effects should be
retained (clinician, practice). After accounting for clinician-level variability,
variability at the practice level was not significant (P>.20). Thus a 2-level model was used (patient, clinician). Sensitivity
analyses were performed by strata when cell frequencies were adequate.
Significance from the generalized linear mixed models was determined
using the F statistic, a joint significance test of global differences among
any categories. Statistical significance was defined as P<.05 (2-tailed test). To study characteristics associated with
reported missing clinical information, power calculations indicated that a
sample of 340 events of missing information per group was necessary to detect
a 10% absolute difference in rates of missing clinical information in a 2-group
comparison with 80% power, assuming an intraclass correlation coefficient
of 0.08 (variance inflation factor, 1.48) and a missing information rate of
13% in 1 group. All analyses were performed using SAS 8.2 (SAS Institute Inc,
A total of 253 clinicians in 32 practices returned study questionnaires
for 1614 visits. Eight of these practices were rural HPRN sites and 24 were
urban/suburban CaReNet practices. Six invited practices, representing 34 clinicians,
did not participate; reasons included extreme weather, an influenza outbreak,
and being too busy with practice or concomitant surveys. Participating and
nonparticipating practices did not differ significantly in size (P = .26), rurality (P = .33),
or whether they were residency practices (P = .64).
Although the number of clinicians within the networks is constantly changing,
we estimated that the 253 participants represent 71% of all network clinician
full-time equivalents. Of these 253 clinicians, 7 did not complete the clinician
survey, leaving 51 patient visits without clinician information. As a result,
clinician information was available for 1563 patient visits (96.8%).
The results of univariate analyses of patient, visit, clinician, and
practice characteristics are shown in Table 1 and Table 2. Diverse age
groups and both sexes were well-represented. Clinicians characterized most
patients as white (74.6%) but one third of patients as Hispanic. Half of all
patients had at least 2 active medical problems, while relatively few were
characterized as first-time patients (13.0%) or recent immigrants (5.1%).
Most respondents were family physicians. Most practices were nonrural and
reported electronic access to inpatient data.
Clinical information considered important was reported to be missing
at the time of the visit in 220 (13.6%) of 1614 visits, and many visits had
more than 1 type of information missing (Table
3). Clinicians reported that the types of information missing included
(as a percentage of total visits) laboratory results (6.1%), letters/dictation
(5.4%), radiology results (3.8%), history and physical examination (3.7%),
and medications (3.2%). In 97 (44.0%) of these visits, clinicians reported
that missing information was at least somewhat likely to adversely affect
the patient (Table 3). Clinicians believed
the missing information was outside their clinical system in 57.3% of visits
with missing information. They also reported that someone attempted to find
the missing information in 125 (56.8%) of these visits. For 45 (36.0%) of
these 125 visits, clinicians reported spending at least 5 minutes looking
for missing clinical information. They also reported that during 36 (28.8%)
of the 125 visits, staff spent at least 5 minutes looking for missing information.
Clinicians believed that missing information would likely result in either
delayed care or at least 1 duplicative medical service in 59.5% of visits
with missing information (Table 3).
Associations between missing clinical information and patient, visit,
clinician, and practice characteristics, separately and in combination, were
tested using multilevel models adjusted for clustering of patients within
physicians (Table 4). Increased reporting
of missing clinical information was significantly associated with first visit
(odds ratio [OR], 2.39; 95% confidence interval [CI], 1.70-3.35), rural clinician
(OR, 0.52; 95% CI, 0.29-0.92), immigration within 5 years (OR, 1.78; 95% CI,
1.06-2.99), and number of active medical problems (no problems vs 1 problem:
OR, 1.09; 95% CI, 0.69-1.73; 2-5 problems: OR, 1.87; 95% CI, 1.21-2.89; >5
problems: OR, 2.78; 95% CI, 1.61-4.80). Clinical information was equally likely
to be reported missing regardless of electronic access to information at one’s
primary hospital, the size of the practice, the month of data collection,
whether physicians were residents, or whether the setting was a residency
practice. Family physicians had rates of visits with missing information similar
to those of other physicians (13.2% vs 14.4%; P = .61).
While physicians had a smaller percentage of missing clinical information
than did midlevel clinicians (13.4% vs 26.5%), the small numbers of visits
for which midlevel clinicians reported missing information (n = 9)
precluded further analysis.
Within a given practice, there was only 81% agreement on average among
clinicians on how to classify the practice’s charting system. Accordingly,
we assessed practices’ charting systems using both individual clinician
report and clinician concurrence, determined by taking the response most often
reported by the clinicians within each practice. Only 17 clinicians indicated
that their offices had full EMRs. When compared with respondents who reported
having hybrid EMRs or paper records, clinicians who reported having full EMRs
were significantly less likely to report missing clinical information (Table 4), while reporting a partial EMR did not
confer a difference (OR, 0.88; 95% CI, 0.60-1.28). However, when using the
practice-level variable of clinician concurrence rather than individual report,
no benefit was seen for practices determined to have full EMRs (OR, 0.60;
95% CI, 0.25-1.40).
We studied primary care clinicians’ reports about missing clinical
information during patient visits and their beliefs about its potential consequences.
In nearly 1 in 7 visits, they reported that clinical information important
for the patient’s care was missing. Although laboratory reports and
dictations or letters predominated, clinicians reported that the missing information
originated from a variety of sources and often included more than 1 type.
In 44% of the visits with missing information, clinicians believed the patient
would be at least somewhat likely to be adversely affected. If validated by
future research, these results could have serious implications for the 220
million primary care visits that occur in the United States each year.30
Poon et al31 found that 83% of surveyed
physicians had reviewed at least 1 test result in the previous 2 months that
they would have wanted to know about earlier, despite having fairly advanced
electronic information systems. It is not surprising that in our study clinicians
and staff spent significant amounts of time looking for missing information,
especially when they believe it often leads to delayed care, duplicative services,
or potential adverse effects for their patients. We did not validate these
time estimates, and based on other research32 clinicians
may have overestimated the amount of time spent unsuccessfully looking for
missing information. However, by excluding any time spent during the visit
that resulted in finding the information (so that it was not classified as
missing), or time spent looking for missing information after the visit was
over, we may have underestimated the total lost time related to searching.
This may represent less time available for direct patient care, a further
reduction in a resource that is already under threat from other competing
We found relatively few predictors of missing clinical information.
Clinicians were more likely to report missing clinical information during
visits in which the patient had recently moved to the United States, was new
to a practice, or had multiple medical problems. These factors have been implicated
in missing information–related medical errors and adverse events in
other settings.10,12,15,33 Rural
clinicians were less likely to report missing information than urban or suburban
clinicians, perhaps because of simpler and more self-contained systems of
care, with fewer clinicians and facilities compared with urban areas. It is
possible that the influence of broader systemic factors on missing clinical
information that could not be discerned in this study may overwhelm such patient,
clinician, or practice factors.
Clinicians reporting a full EMR in their practice were significantly
less likely to report missing clinical information, but this did not eliminate
the problem. Missing information was believed more likely to be outside the
clinical system than within it and therefore may be beyond the reach of an
EMR. The lack of impact of partial EMRs and electronic access to hospital
data on adverse events has been found in other settings.11,18 We
found no difference in reports of missing information when we used the concurrence
among clinicians within a practice to determine the EMR variable. This difference
from individual report may indicate that familiarity with or actual use of
an EMR is a better predictor of effective information management than the
mere presence of an EMR.
This study has several important limitations. The data are cross-sectional
and based on clinician report, including patient race and ethnicity, which
may be less accurate than patient self-identification. Several network practices
reported being too busy to participate. Although this number was small, had
they participated the rate of reported missing clinical information may have
been slightly higher. There was no independent verification that questionnaires
were completed on every consecutive patient in each clinic session. The definition
of information that was “important for the care of this patient”
was open to broad interpretation by the respondent. Such information may be
both important and urgent (eg, an allergy to a newly prescribed medication)
or important but not urgent (eg, a written advance directive for a patient
with dementia, or urinary microalbumin results for a patient with diabetes).
To explore the widest possible scope of the problem of missing clinical
information, there was no requirement that having the information available
during the visit was reasonable. Expecting prior medical records at a first
visit may not yet be realistic in many practices, and primary efforts to remedy
the problem may best focus on limiting missing information for existing patients.
However, these findings suggest that robust, long-term solutions may need
to include transfers of care across care settings, even across international
borders.34 One model for a solution is the
Continuity of Care Record, a data standard that enables diverse information
systems to share a minimal clinical data set whose components closely mirror
the types of missing information reported in this study,35 that
has the potential to be disseminated via portable memory devices or secure
e-mail or Web servers, and that can be printed and given directly to patients
or new clinicians.
Because clinicians were not given a specific definition of an adverse
effect from missing clinical information, their responses may have considered
outcomes ranging from minor inconvenience to financial hardship to actual
physical injury. We did not validate or characterize these estimates of potential
adverse effects. Although other studies have demonstrated that errors related
to missing clinical information are common and can adversely affect patients,8,9,11,15,20,36-38 future
research should focus on the actual impact of missing information on patients,
clinicians, practices, and systems of care.
Although we did not validate the accuracy of clinician report of missing
clinical information, a recent direct-observation study indicated that primary
care physicians’ reports of events during patient visits are highly
accurate.32 We did not confirm whether information
reported as missing actually existed and, if it did, whether it was truly
inaccessible to the clinician or was functionally missing (ie, actually available
but not found when needed). Clinicians may have reported nonexistent information
(such as a laboratory test ordered but never actually performed) as missing.
Conversely, they may have reported information as missing that was actually
at their fingertips but that they did not or could not access (such as results
buried inside a thick paper record). We did not determine how well the practices’
electronic systems were functioning or used during the study, which may have
transiently altered the rate of missing information. However, busy clinicians
making medical decisions during clinic visits need information systems that
are both effective and efficient. Because most medical decisions are made
during patient visits, clinicians may not distinguish between actually missing
and functionally missing information.
Although this is a state-level survey, our sample included diverse clinicians
and patients from a variety of practices in multiple geographic, economic,
and demographic settings. Although the racial and ethnic composition of our
sample was different from national norms, we found no differential rates of
missing clinical information based on race or ethnicity. We therefore believe
that these results should be generalizable.
This is the first direct study of missing clinical information in primary
care, in contrast to retrospective detection of missing information as the
etiology of a medical error or adverse event. It demonstrates reports of a
high frequency of missing important clinical information, with a wide array
of potential impact on patient care. Additional research on missing clinical
information should focus on validating clinicians’ perceptions and conducting
prospective studies of its actual causes and sequelae.
Corresponding Author: Peter C. Smith, MD,
Department of Family Medicine, University of Colorado Health Sciences Center
at Fitzsimons, PO Box 6508, Mail Stop F496, 12474 E 19th Ave, Bldg 402, Aurora,
CO 80045-0508 (email@example.com).
Author Contributions: Dr Smith had full access
to all of the data in the study and takes responsibility for the integrity
of the data and the accuracy of the data analyses.
Study concept and design: Smith, Araya-Guerra,
Parnes, Westfall, Pace.
Acquisition of data: Smith, Araya-Guerra, Dickinson,
Van Vorst, Westfall, Pace.
Analysis and interpretation of data: Smith,
Araya-Guerra, Bublitz, Parnes, Dickinson, Westfall, Pace.
Drafting of the manuscript; critical revision of the
manuscript for important intellectual content: Smith, Araya-Guerra,
Bublitz, Parnes, Dickinson, Van Vorst, Westfall, Pace.
Statistical analysis: Araya-Guerra, Bublitz,
Parnes, Dickinson, Van Vorst.
Obtained funding: Smith, Pace.
Administrative, technical, or material support:
Araya-Guerra, Westfall, Pace.
Study supervision: Smith, Araya-Guerra, Dickinson,
Financial Dislosure: None reported.
Funding/Support: This study was funded by the
American Academy of Family Physicians Foundation and the Joint AAFP/F-AAFP
Grant Awards Council (MIA Care: the Missing Information in Ambulatory Care
Study, grant G0307RS) (Dr Smith) and in part by the Agency for Healthcare
Research and Quality (Applied Strategies for Improving Patient Safety, grant
U18 HS11878) (Dr Pace).
Role of the Sponsors: None of the funding sources
had any role in the design and conduct of the study; the collection, preparation,
or interpretation of the data; or the preparation or approval of the manuscript.
Previous Presentations: An earlier draft of
this article was presented as a distinguished paper at the North American
Primary Care Research Group annual meeting; October 10-13, 2004; Orlando,
Acknowledgment: We thank Tillman Farley, MD,
and Marc Ringel, MD, for their early conception and support of this study
and Linda Niebauer, Elizabeth Staton, and especially Sherry Holcomb for their
invaluable assistance. We thank all the participating SNOCAP practices for
their dedicated participation: AF Williams Family Medicine, Colorado Springs
Osteopathic Foundation and Family Medicine Center, Comprehensive Family Care
Center, PC, Denver Health Medical Plan Clinic, Exempla St Joseph Family Practice,
Generations Health Care, High Street Internal Medicine, Internal Medicine–AOP,
Kids Care Clinic, La Casa-Quigg Newton Health Center, Lowry Family Health,
Mariposa Family Health, Park Hill Family Health, People’s Clinic, Plains
Medical Center Limon and Strasburg Clinics, Rose Family Medicine, Saint Mary’s
Family Practice, Salud Family Health Centers (Brighton Family Health, Commerce
City, Ft Lupton, Ft Morgan, and Sterling), Southern Colorado Family Practice,
St Anthony's Family Medicine Center West, Swedish Family Medicine, University
Family Medicine (Park Meadows and Westminster), Westside Family Health Center
Pediatric and Teen Clinic, Wray Clinic, and Yuma Clinic.