Ye F, Moon DH, Carpenter WR, Reeve BB, Usinger DS, Green RL, Spearman K, Sheets NC, Pearlstein KA, Lucero AR, Waddle MR, Godley PA, Chen RC. Comparison of Patient Report and Medical Records of ComorbiditiesResults From a Population-Based Cohort of Patients With Prostate Cancer . JAMA Oncol. Published online February 16, 2017. doi:10.1001/jamaoncol.2016.6744
Do medical records and patient reports on the presence or absence of comorbid conditions agree in patients with newly diagnosed prostate cancer?
In this population-based and diverse cohort of 881 patients, comorbidities were prevalent, and patient reports and medical records for most medical conditions agreed in more than 90% of patients.
Patient reporting provides information similar to medical record abstraction and may be a less costly method for assessing comorbid conditions for observational comparative effectiveness research.
The comorbid conditions of patients with cancer affect treatment decisions, which in turn affect survival and health-related quality-of-life outcomes. Comparative effectiveness research studies must account for these conditions via medical record abstraction or patient report.
To examine the agreement between medical records and patient reports in assessing comorbidities.
Design, Setting, and Participants
Patient-reported information and medical records were prospectively collected as part of the North Carolina Prostate Cancer Comparative Effectiveness & Survivorship Study, a population-based cohort of 881 patients with newly diagnosed localized prostate cancer enrolled in the North Carolina Central Cancer Registry from January 1, 2011, through June 30, 2013. The presence or absence of 20 medical conditions was compared based on patient report vs abstraction of medical records.
Main Outcomes and Measures
Agreement between patient reports and medical records for each condition was assessed using the κ statistic. Subgroup analyses examined differences in κ statistics based on age, race, marital status, educational level, and income. Logistic regression models for each condition examined factors associated with higher agreement.
A total of 881 patients participated in the study (median age, 65 years; age range, 41-80 years; 633 white [71.9%]). In 16 of 20 conditions, there was agreement between patient reports and medical records for more than 90% of patients; agreement was lowest for hyperlipidemia (68%; κ = 0.36) and arthritis (66%; κ = 0.14). On multivariable analysis, older age (≥70 years old) was significantly associated with lower agreement for myocardial infarction (odds ratio [OR], 0.31; 95% CI, 0.12-0.80), cerebrovascular disease (OR, 0.10; 95% CI, 0.01-0.78), coronary artery disease (OR, 0.37; 95% CI, 0.20-0.67), arrhythmia (OR, 0.44; 95% CI, 0.25-0.79), and kidney disease (OR, 0.18; 95% CI, 0.06-0.52). Race and educational level were not significantly associated with κ in 18 of 19 modeled conditions.
Conclusions and Relevance
Overall, patient reporting provides information similar to medical record abstraction without significant differences by patient race or educational level. Use of patient reports, which are less costly than medical record audits, is a reasonable approach for observational comparative effectiveness research.
Prostate cancer treatment decision making and patient outcomes are greatly affected by patient baseline comorbid conditions.1,2 Because the median age of diagnosis of prostate cancer in the United States is 67 years,3 many patients have other medical conditions concomitantly, such as diabetes, hypertension, cardiovascular disease, and cerebrovascular disease.4,5 Radical prostatectomy is a more likely treatment given to younger patients with fewer comorbidities, whereas radiotherapy and conservative management (hormone therapy or no treatment) are more likely given to older patients and those with more comorbidities.3,6- 8 Another study9 suggests that patients with fewer comorbidities are more likely to travel long distances to receive treatment at large-volume academic centers.
Comparative effectiveness of patient outcomes among different localized prostate cancer treatment options is one of the highest-priority research areas according to the Institute of Medicine.10 Because a patient’s comorbid conditions heavily influence treatment selection and directly affect survival and health-related quality-of-life outcomes,1,2,11 observational comparative effectiveness research studies must account for these conditions. A central methodologic issue is whether to collect comorbidity data using medical record abstraction or patient report to maximize data quality while minimizing cost of data collection. Medical record collection and abstraction depend on the scrupulousness of the documenting health care professional, require an abstractor with sufficient medical training, and are more costly to perform.12 On the other hand, patient report relies on each patient accurately knowing his or her medical history, which may be dependent in part on his or her health literacy. For observational comparative effectiveness research, it is not clear that a current criterion standard exists, and each source of information (patient vs medical records) has the potential to overreport and underreport comorbid medical conditions.
The purpose of this study was to compare patient reports and medical records in assessing comorbidity in a population-based cohort of patients with newly diagnosed prostate cancer. We quantify the level of agreement between these 2 sources on common comorbid conditions and assess factors associated with agreement. Given that socioeconomic status is associated with health literacy,13,14 we hypothesized that patients who were nonwhite and those with lower educational levels would have lower agreement between patient reports and medical records.
The North Carolina Prostate Cancer Comparative Effectiveness & Survivorship Study (NC ProCESS) is a prospective, population-based cohort of patients with newly diagnosed localized prostate cancer enrolled throughout North Carolina in collaboration with the Rapid Case Ascertainment system of the North Carolina Central Cancer Registry. The Rapid Case Ascertainment system proactively identified patients with newly diagnosed prostate cancer from all 100 North Carolina counties from January 1, 2011, through June 30, 2013. Names of patients, pathologic and diagnostic information, and physician names and addresses were sent weekly to the Rapid Case Ascertainment staff by tumor registrars at local hospitals. The patients were then approached by the NC ProCESS staff for study participation; 1419 (57.2%) of 2480 eligible patients agreed to enroll. All patients were enrolled before treatment and followed up prospectively to collect data from medical records and patient-reported outcomes. For each patient, primary care physician and urologist records, as well as radiation oncologist records if consulted, were obtained. Individual-level sociodemographic information (race, marital status, educational level, income) was collected by patient report. This study was approved by the University of North Carolina Institutional Review Board, including consent to obtain records. Written informed consent was obtained from all patients, and data were deidentified for analysis.
This study included 881 patients enrolled in the NC ProCESS. Data were collected on the following 20 comorbid conditions: myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, chronic obstructive pulmonary disease, peptic ulcer disease, liver disease, diabetes, kidney disease, other cancers, human immunodeficiency virus or AIDS, coronary artery disease, arrhythmia, clotting disorders, hypertension, hyperlipidemia, inflammatory bowel disease, asthma, anemia and other blood disorders, and arthritis. These conditions are included in the most commonly used comorbidity indexes in cancer research, including the Charlson Comorbidity Index,15,16 Adult Comorbidity Evaluation Index,17 Index of Co-Existent Diseases,16 and Kaplan-Feinstein Comorbidity Index.18,19 The conditions were assessed by patient report via telephone survey (“Have you ever been told by a doctor or other health professional that you have [comorbid condition]?”) and by medical record abstraction at the time of study enrollment, which was always before treatment. Telephone surveys were conducted by the Carolina Survey Research Laboratory at the University of North Carolina, and staff followed the above script to elicit comorbidity information. In addition, medical records were abstracted for the presence of these comorbidities.
We describe the presence of each condition based on patient report, medical record abstraction, both, or neither; κ statistics were used to quantify the level of agreement between patient report and medical records. The κ statistic is a way of reporting agreement between 2 information sources by providing a quantitative measurement of the interobserver agreement magnitude, accounting for chance agreement.20 This is standardized to fit a scale of −1 to 1; 0 represents the amount of expected agreement by chance alone, whereas 1 is perfect agreement.20 Data from patients who did not respond to the telephone survey (11 of 881 patients [1.2%]) or reported that they were unsure whether they had a condition (affecting ≥1 comorbid conditions in 15 of 881 patients [1.7%]) were excluded from analysis. Landis and Koch thresholds were used to classify agreement levels as poor or slight (<0.20), fair (≥0.20 to <0.40), moderate (≥0.40 to <0.60), substantial (≥0.60 to <0.80), or almost perfect (≥0.80).21 We performed subgroup analyses to determine whether the κ statistic varied by age, race, educational level, income, or cancer aggressiveness (defined by prostate cancer risk group).22 We then performed logistic regression to assess covariates associated with agreement between patient report and medical records, with separate models for each comorbidity. The subgroup and multivariable analyses inform our understanding of whether certain subgroups of patients with prostate cancer have higher or lower agreement in terms of the presence of comorbid conditions from the 2 data sources. All statistical analyses were performed using SAS statistical software, version 9.4 (SAS Institute Inc).
A total of 881 patients were analyzed (median age, 65 years; age range, 41-80 years). This cohort was sociodemographically diverse, with 633 white participants (71.9%) and 248 nonwhite participants (28.1%), 281 (31.9%) with high school education or less, and 322 (36.5%) with household income of $40 000 or less. Baseline characteristics are given in Table 1.
Table 2 gives the frequency of patient and physician reporting of each comorbid condition, level of agreement, and κ statistics. We found agreement between patient reports and medical records in more than 90% of patients for all conditions, except coronary artery disease (87%), arrhythmia (86%), hypertension (85%), hyperlipidemia (68%), and arthritis (66%). When assessed using the κ statistic, agreement was substantial or almost perfect for myocardial infarction (κ = 0.62), cerebrovascular disease (κ = 0.72), diabetes (κ = 0.90), human immunodeficiency virus or AIDS (κ = 1.00), and hypertension (κ = 0.68), whereas agreement was lower for the other conditions. When there was disagreement, both scenarios (medical records indicating condition not reported by patients and patient indicating presence of condition not indicated in medical records) were observed. For cerebrovascular disease, chronic obstructive pulmonary disease, diabetes, coronary artery disease, and hypertension, medical records were more likely to report the condition than patients, whereas for myocardial infarction, congestive heart failure, peptic ulcer disease, kidney disease, other cancers, arrhythmia, clotting disorders, hyperlipidemia, asthma, anemia and other blood disorders, and arthritis, patients were more likely to indicate the condition.
Subgroup analysis for the κ statistic based on age, race, marital status, educational level, income, and prostate cancer risk groups for each comorbidity is given in Table 3. The κ statistics differed by patient race in 7 conditions: white patients had higher κ statistics in chronic obstructive pulmonary disease, liver disease, cancers other than prostate, and coronary artery disease, whereas nonwhite patients had higher κ statistics in congestive heart failure, clotting disorders, and inflammatory bowel disease. The κ statistics also differed by educational level in 4 conditions: patients with a high school education or less had higher κ statistics in kidney disease, clotting disorders, and anemia and other blood disorders, whereas those with more than a high school education had a higher κ statistic in inflammatory bowel disease.
In multivariable logistic regression (Table 4), older age was associated with lower overall agreement in multiple conditions: older than 70 years compared with younger than 60 years was associated with lower overall agreement for myocardial infarction (odds ratio [OR], 0.31; 95% CI, 0.12-0.80), cerebrovascular disease (OR, 0.10; 95% CI, 0.01-0.78), kidney disease (OR, 0.18; 95% CI, 0.06-0.52), coronary artery disease (OR, 0.37; 95% CI, 0.20-0.67), and arrhythmia (OR, 0.44; 95% CI, 0.25-0.79); age between 60 and 69 years compared with younger than 60 years was associated with lower overall agreement for cerebrovascular disease (OR, 0.11; 95% CI, 0.01-0.85), kidney disease (OR, 0.33; 95% CI, 0.12-0.91), and coronary artery disease (OR, 0.55; 95% CI, 0.31-0.96). Nonwhite race was associated with lower overall agreement for kidney disease (OR, 0.21; 95% CI, 0.10-0.43). A high school education or less compared with more education was associated with higher overall agreement for anemia and other blood disorders (OR, 2.57; 95% CI, 1.24-5.33).
In this study of 881 patients with prostate cancer from a diverse, population-based cohort, we examined the agreement between patient report and medical record documentation for 20 comorbid conditions commonly included in comorbidity indexes used in cancer research. In 16 of these conditions, we found agreement between patient report and medical record documentation in more than 90% of patients, whereas agreement was notably lower for hyperlipidemia and arthritis. On multivariable analysis, older age was significantly associated with lower overall agreement for multiple cardiovascular conditions and kidney disease. Our hypothesis that nonwhite race and lower educational level would be associated with lower patient vs physician report agreement was mostly incorrect.
Accurate assessment of comorbidities is important because prostate cancer treatment decision making is directly affected by a patient’s baseline comorbidity status.1,2 In the United States, younger and healthier patients commonly undergo prostatectomy, whereas older patients and those with more comorbid conditions receive radiation or conservative management.19 Comparative effectiveness research for localized prostate cancer treatment options is a high-priority research area10; given clinical patient selection into different treatment groups, observational studies must accurately account for a patient’s comorbidities to reach valid conclusions. Because a patient’s comorbid conditions directly affect survival2 and quality of life,11 a better understanding of how to assess comorbidity information addresses one of the most important methodologic issues in comparative effectiveness research.
Our study is novel because, to our knowledge, no prior studies have directly compared patient and physician report of comorbidities specifically in patients with prostate cancer. Both sources of information are used in different prostate cancer studies to account for comorbidities,23- 27 yet we do not know whether patient reports and medical records provide the same information. A report23 from the Prostate Cancer Outcomes Study, which in a population-based cohort of patients enrolled in 1994 collected patient-reported comorbidity information serially, found that patient reports of comorbid conditions were consistent over time; however, the study did not provide detailed data on how patient reports compared with medical records. Another study published by Katz et al12 in 1996 of 170 hospitalized patients compared patient reports and hospital records. For conditions studied that overlap with our study, the κ statistics were similar to our findings for myocardial infarction, congestive heart failure, peripheral vascular disease, cerebrovascular disease, chronic obstructive pulmonary disease, and peptic ulcer disease, whereas they differed by more than 0.20 for diabetes, kidney disease, and other cancers.12 We also found several other studies28- 30 that examined concordance between medical records and patient reports. However, these studies did not specifically include patients with cancer, were conducted in settings that were not real world (inpatient, Veterans Affairs ambulatory care, or patients with specific health plans), and did not examine whether agreement differed by patient subgroups.
Given the lack of prior data, results from this study provide important information on a central methodologic issue in observational comparative effectiveness research: to accurately account for potential differences in patient characteristics across comparison groups. Assessing comorbid medical conditions using patient report is a less expensive option than medical record abstraction. Our results suggest that patient report provides information similar to medical records for most comorbidities studied, and there is no consistent pattern of higher or lower agreement by race or educational level, but there was a lower κ statistic in older patients for several, mostly cardiovascular, conditions, which are especially likely to affect older individuals. One potential explanation is that older patients are more likely to have more comorbidities,5 and as the number of comorbidities increases, precision between medical record and patient report decreases. In addition, older patients may have lower health literacy, which may have contributed to the lower agreement. However, additional research is needed to further examine measurement of cardiovascular comorbidities in older patients with cancer. Because the NC ProCESS obtains longer-term follow-up in these patients, we will be able to assess whether patient report, medical records, or both best predict survival and quality-of-life outcomes.
There is an established literature on patient-reported outcomes that has consistently found discordance between patient report vs physician report in cancer treatment–related adverse effects and health-related quality of life.31- 33 Patients tend to report these symptoms earlier and more frequently than do physicians,32 and this research has led to a common acceptance of patient-reported outcomes as a valid measure in clinical trials. Thematically, these prior studies31- 33 provide a framework for interpreting results from this study: indeed, there are modest discrepancies between patient and physician reporting of comorbid conditions. In this study, we obtained and abstracted medical records from the primary care physician and cancer specialists because the specialists need to accurately collect comorbidity information for treatment decision making and the primary care physician would be expected to know and document all medical conditions in their patients. Doing so, we found agreement between patient report and medical record documentation in more than 90% of patients. These results support the recent standard set defined by an international group of experts regarding data that should be collected for all patients with localized prostate cancer, including comorbidities using patient report instead of medical record abstraction.34
This study contains multiple methodologic strengths. The NC ProCESS cohort is large and population based, and the rich diversity of enrolled patients allowed for subgroup analyses to examine agreement by age, race, educational level, and other factors. The cohort is modern, with all patients enrolled from 2011 to 2013. Enrollment of patients from the community setting also provides information that is reflective and generalizable to real-world patients and medical care. In addition, patient-reported comorbidity information and medical records were obtained before prostate cancer treatment, which avoids potential confounding from conditions developed after (or potentially because of) treatment. On the other hand, in our literature search, we found no existing data on what constitutes a meaningful difference in κ statistic. In Table 3 (subgroup analysis), we highlighted differences of 0.20 or greater to help the reader more easily identify subgroups with sizable differences in agreement,21 and the actual values are provided to allow readers to make their own conclusions.
One limitation of the study is that we did not examine whether different sources of medical records (academic vs community, primary care physician vs specialist) differed in information accuracy. The issue of accuracy in comorbidity reporting is difficult to assess; when medical records and patient reports differ, it is unclear who is right. Conditions listed in the medical record may be incorrect if they are assumptions based on a patient’s medication list or errors that continue to propagate over time. Another limitation of the study is that we were unable to compare calculated comorbidity scores (eg, Charlson Comorbidity Index) between medical records and patient report, which are commonly used in comparative effectiveness studies, because of a lack of certain specific details required for score calculation. However, because there was overall high agreement in most comorbid conditions examined, we hypothesize that calculated comorbidity scores will be similar between medical records and patient report, which will need to be confirmed in future studies.
In a population-based cohort of patients with newly diagnosed prostate cancer, agreement between patient and physician reporting occurred in more than 90% of patients for 16 of 20 assessed comorbid conditions. This is the first large-scale study, to our knowledge, to specifically examine information source in comorbidity reporting, a central issue in observational comparative effectiveness research. Overall, patient reporting provides information similar to medical record abstraction, without significant differences by patient race or educational level. However, assessment of cardiovascular conditions in older patients requires further study.
Corresponding Author: Ronald C. Chen, MD, MPH, Department of Radiation Oncology, University of North Carolina at Chapel Hill, Campus Box 7512, 101 Manning Dr, Chapel Hill, NC 27599 (email@example.com).
Accepted for Publication: November 16, 2016.
Published Online: February 16, 2017. doi:10.1001/jamaoncol.2016.6744
Author Contributions: Dr Chen had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Ye, Godley, Chen.Acquisition, analysis, or interpretation of data: Ye, Moon, Carpenter, Reeve, Usinger, Green, Spearman, Sheets, Pearlstein, Lucero, Waddle, Chen.Drafting of the manuscript: Ye, Carpenter, Usinger, Green, Spearman, Sheets, Pearlstein, Lucero, Chen.Critical revision of the manuscript for important intellectual content: Ye, Moon, Carpenter, Reeve, Usinger, Waddle, Godley, Chen.Statistical analysis: Ye, Green.Obtained funding: Chen.Administrative, technical, or material support: Carpenter, Usinger, Spearman, Sheets, Lucero, Waddle, Godley.
Study supervision: Chen.
Conflict of Interest Disclosures: None reported.
Funding/Support: This research was funded by contract HHSA29020050040I from the Agency for Healthcare Research and Quality, US Department of Health and Human Services, as part of the DEcIDE (Developing Evidence to Inform Decisions about Effectiveness) program.
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.