Ming ME, Levy RM, Hoffstad OJ, Filip J, Gimotty PA, Margolis DJ. Validity of Patient Self-reported History of Skin Cancer. Arch Dermatol. 2004;140(6):730-735. doi:10.1001/archderm.140.6.730
Michael E.BigbyMDRosamariaCoronaDSc, MDDamianoAbeniMD, MPHAlexandra B.KimballMD, MPHMoysesSzkloMD, MPH, DrPHHywelWilliamsMSc, PhD, FRCP
To determine the validity of patient self-report of skin cancer history.
A cohort of patients was randomly selected from the case group in a prior case-control study involving skin cancer, and a second cohort was randomly selected from the controls of that study. Patient self-reported history (as determined by responses to a survey) was compared with the gold standard of chart documentation of a pathology report or a procedure note from Mohs micrographic surgery demonstrating skin cancer.
University-based outpatient dermatology clinic.
Three hundred patients were selected.
Main Outcome Measures
Patients were considered to have correctly classified their skin cancer history if their self-reported history was consistent with chart documentation.
We obtained chart information for 258 patients.Of those patients, 183 (70.9%) had chart documentation of nonmelanoma skin cancer, and 16 (6.2%) had chart documentation of a melanoma. Using chart documentation as the gold standard, we found that patients correctly identified their basal cell carcinoma status in 84.3% of cases; their squamous cell carcinoma status in 81.5% of cases; their overall nonmelanoma skin cancer status in 91.8% of cases; their melanoma status in 94.8% of cases; and their overall skin cancer status in 92.6% of cases. Patients' self-reported history of skin cancer of any type had a positive predictive value of 95.1% and a negative predictive value of 85.9%.
Self-reported history of skin cancer had a high degree of sensitivity and specificity and a high positive and negative predictive value within the study population. Obtaining medical information by patient report appears to be a useful tool for determining medical history of skin cancer.
An accurate understanding of a patient's medical history is important for good medical care,1 particularly in cases in which the patient's likelihood of having disease may be affected by his medical history. Patients with a previous diagnosis of skin cancer have a greater likelihood of developing an additional malignancy than the general population has for developing a malignancy.2- 4 In a sense, asking patients about their skin cancer history is akin to administering a screening test with the hope that the results accurately reflect the patient's true medical status. The patient response may influence medical care decisions such as whether to perform a biopsy on a clinically borderline lesion because patients with a history of skin cancer have a different a priori probability of presenting with skin cancer than patients without such a history. Patients with a history of skin cancer also need to have a clear understanding of the importance of avoiding risk factors for future development of skin cancer such as UV light exposure and may also have evidence of a genetic predisposition for skin malignancy if there is a strong family history of disease.
In most cases, information regarding a patient's medical history is obtained by patient self-report at the time of the clinical visit. Patient self-report is a common tool that is used in clinical practice and epidemiologic research to gather medical information.5,6 While self-reporting is an efficient way to obtain this type of information, the validity of the information found through this method may vary. Some studies, including those examining self-report of chronic medical illnesses such as hypertension and diabetes6,7 and others examining self-report of fractures8 and history of systemic cancer,9,10 have shown that self-report is an accurate method of obtaining medical information, showing self-reported medical history to match the gold standard of chart documentation in 80% to 90% of cases. However, others have found that patient-reported medical history has relatively low validity when obtaining information on previous hospitalizations, particularly readmissions,11 child immunization history from parents,12 and oral contraceptive use.13 Additionally, other investigators have found numerous personal and disease-related characteristics, such as sex, annual number of sick days, and prevalence of the disease in the general population, to influence patient self-reporting of chronic disease.5
To the best of our knowledge, no study has investigated the validity of self-reported medical history of skin cancer in any patient population. A better understanding of whether self-reported history is a valid method of determining skin cancer status would help us to understand whether this common means of obtaining medical history is appropriate and useful. We sought to improve our understanding of methods of obtaining patient information by determining the accuracy of patient self-reported history of skin cancer in a group of patients seen in a university-based dermatology clinic.
Our subjects were part of a larger survey study examining the relationship between atopic dermatitis and skin cancer.14,15 The larger study used a random sample of patients who had been seen in the Department of Dermatology at the University of Pennsylvania between January 1998 and June 2001 and who had been assigned 1 of the following ICD-9 codes (International Classification of Diseases, Ninth Revision) at that visit: 173.x (other [nonmelanoma] malignant neoplasm of skin); 078.10 (viral warts); 695.3 (rosacea); 110.x (dermatophytosis); and 702.1 (seborrheic keratosis). Each of the subjects was mailed a questionnaire using a modification of the Dillman Tailored Design Method.16 (The Dillman Tailored Design Method is a method for maximizing response rate to surveys and includes multiple mailings and specific questionnaire and mailing design.) Among other questions, the survey asked if the subject had ever been diagnosed with basal cell carcinoma, squamous cell carcinoma, or malignant melanoma, and if so, the year of diagnosis. Patients were also asked if they had had a skin cancer of a type not mentioned above and if so, the year of diagnosis.
In the larger study, 3203 (69.8%) of the 4591 patients who were asked to participate completed the questionnaire.14 The demographic characteristics for participants and nonparticipants were generally similar, although African Americans (57.7% response rate), those not between age 60 and 79 years (66.8% response rate), and those who lived within the Philadelphia city limits (68.5% response rate) had slightly lower response rates than those who did not have these characteristics.15 For participants who completed the questionnaire, we attempted to locate the dermatology records for a random sample of 200 subjects with an ICD-9 code assignment of 173.x (other [nonmelanoma] malignant neoplasm of skin) and 100 subjects who had been assigned 1 of the other ICD-9 codes. The random sample was obtained by first serially assigning a number to each eligible subject, then using a random number generator to identify subjects for the present study. Patients were considered to have chart documentation of a particular type of skin cancer if the medical record contained either a pathology report with a final diagnosis of that particular type of skin cancer or documentation from Mohs micrographic surgery showing that particular type of skin cancer within the Mohs surgical specimen.
We considered patients with chart documentation of a particular type of skin cancer to have correctly identified their cancer history if they said that they had had that type of skin cancer, and we considered them to have incorrectly identified their cancer history if they denied having had that type of skin cancer. We considered patients without chart documentation of a particular type of skin cancer to have correctly identified their cancer history if they denied having had that type of skin cancer, and we considered them to have incorrectly identified their cancer history if they said that they had had that type of skin cancer during a particular year and they had been a member of our clinic for that entire year but there was no chart documentation of their ever having had a skin cancer. We considered patients to be members of our clinic during the time period between their first and last visit to our clinic. Similar rules were used for determining whether patients correctly identified a history of having ever had any type of skin cancer. The number of skin cancers of a specific type was not used in analysis.
Some patients were not members of our clinic during the entire year in which they claimed to have had a skin cancer, so it was not possible to determine the accuracy of their responses, since they might have had a biopsy performed elsewhere that revealed skin cancer. For the primary analysis, these patients were all considered to have the "worst-case" scenario: they were all classified as having incorrectly identified their cancer history. An additional analysis was also performed in which such patients were all considered to have correctly identified their cancer history ("best-case" scenario).
For each type of skin cancer, the following parameters were determined: sensitivity (the probability that the patient will report a positive history given that the patient truly had that type of skin cancer); specificity (the probability that the patient will report a negative history given that the patient truly did not have that type of skin cancer); correct classification rate (the probability that the patient's self-reported history will accurately reflect the true skin cancer status); positive predictive value (the probability that a person truly had a skin cancer given that she or he reports such a history); and negative predictive value (the probability that a person truly did not have a skin cancer given that he or she reports a negative history).17
After the initial characterizations were performed, we determined whether individual exposure variables were associated with correct classification of skin cancer status by performing logistic regression, for which the outcome of interest was the dichotomous variable of whether the patient correctly classified his or her own skin cancer status. We initially hypothesized that older patients and male patients might have a lower rate of correct classification. Statistical analysis was performed using Stata 6.0 statistical software (Stata Corporation, College Station, Tex).
After a thorough search, we obtained information from the medical charts of 258 (86%) of the 300 eligible subjects; the remainder of the charts were not locatable. Basic demographic information was similar for the group with locatable charts and the group without locatable charts (Table 1).
Of the 258 subjects for whom we could obtain chart information, 147 (57.2%) had chart documentation of having had a basal cell carcinoma; 81 (36.4%) had chart documentation of having had a squamous cell carcinoma; and 16 (6.2%) had chart documentation of having had a melanoma. None of the patients had chart evidence of any other type of cutaneous carcinoma. A total of 183 (70.9%) had chart documentation of at least 1 nonmelanoma skin cancer, and 187 (72.6%) had chart documentation of at least 1 skin cancer of any type.
For the primary analysis and the best-case scenario, the percentage of patients, along with 95% confidence intervals (CIs), who correctly identified their cancer history for each type of skin cancer is listed in Table 2. Data for patients who did not answer the survey question for a particular type of skin cancer were excluded from Table 2 and from further analysis. Using chart evidence of skin cancer as the gold standard, we found that for the primary analysis, patients correctly identified their basal cell carcinoma status in 84.3% of cases; their squamous cell carcinoma status in 81.5% of cases; their overall nonmelanoma skin cancer status in 91.8% of cases; their melanoma status in 94.8% of cases; and their overall skin cancer status in 92.6% of cases. In the best-case scenario, those numbers rose slightly to 89.4%, 86.4%, 93.0%, 97.1%, and 94.5%, respectively.
The positive and negative predictive values for having skin cancer in patients who claimed a specific skin cancer status are listed in Table 3. In the primary analysis, patients' responses to whether they had previously had a basal cell carcinoma had a positive predictive value of 81.4% and a negative predictive value of 89.8%. For squamous cell carcinoma, the values were 70.8% and 86.0%, respectively; for nonmelanoma skin cancer of any type, 94.4% and 85.5%, respectively; for melanoma, 57.1% and 99.6%, respectively; and for any type of skin cancer, 95.1% and 85.9%, respectively. The positive predictive values in the best-case scenario were 89.2% for basal cell carcinoma; 87.5% for squamous cell carcinoma; 96.1% for nonmelanoma skin cancer; 78.6% for melanoma; and 97.8% for any type of skin cancer.
We also used logistic regression to identify patient characteristics associated with correct self-classification of skin cancer status. We found that men were more likely to accurately state their overall skin cancer status than women (univariate odds ratio [OR], 3.15), but the 95% CI was wide (95% CI, 1.14-8.68). The likelihood of accurately reporting overall skin cancer status was similar for those aged 65 years or younger and those older than 65 years (univariate OR, 1.12; 95% CI, 0.44-2.86). For individual types of skin cancer, those aged 65 years or younger were more likely to accurately state their squamous cell carcinoma status than their older counterparts (OR, 2.44; 95% CI, 1.24-4.83), but no difference was seen by age for basal cell carcinoma or melanoma, or by sex for any individual type of skin cancer. Multivariate analysis did not show an interaction between sex and age.
For self-reported history of skin cancer, we found that the sensitivity, specificity, and correct classification rate for nonmelanoma skin cancers as a whole, for melanoma, and for having had a skin cancer of any type were all over 85% in our study population. However, the specificity for history of basal cell carcinoma was 71.8%, and the sensitivity for history of squamous cell carcinoma was 68.0%. While some patients were apparently unsure what type of nonmelanoma skin cancer they had (many apparently believing they had a basal cell carcinoma when in fact they had a squamous cell carcinoma), patients were able to report whether they had had some type of nonmelanoma skin cancer with a high degree of accuracy. From a clinical perspective, knowing that a patient had a nonmelanoma skin cancer without knowing the exact type is often sufficient, and this inaccuracy in self-report may not be clinically meaningful.
The high degree of sensitivity and specificity for history of skin cancer found in the present study compares favorably with rates found in previous studies examining self-report of chronic disease. Although some studies examining self-reported history found sensitivity and specificity values that were in the same range as ours,9,10 other studies found much lower rates.11,13 Thus, the time-honored method of obtaining medical information by patient report seems to be effective for assessing history of skin cancer.
While sensitivity and specificity help us understand whether our screening test (asking patients for their medical history) is a potentially effective way to predict whether the patient actually had skin cancer, it is also important to know whether the patient is likely to have disease if the screening test is positive or is unlikely to have disease if the screening test is negative. We determine the probability of disease in these 2 circumstances by evaluating the positive and negative predictive value. In the present study, predictive values were generally high for nonmelanoma skin cancer as a whole as well as for history of skin cancer as a whole. As with sensitivity and specificity, the predictive values for basal cell carcinoma and squamous cell carcinoma individually were lower than for nonmelanoma skin cancers as a whole. Positive predictive value for melanoma was also relatively low at 57.1% in the primary analysis. However, 50% (6/12) of the patients who claimed to have a melanoma but did not have chart documentation of this cancer reported that the melanoma was diagnosed outside of the time of their dermatologic care at our university. This proportion was higher than that for basal cell carcinoma or squamous cell carcinoma, and the positive predictive value for melanoma markedly improved in the best-case scenario to 78.6%.
Although sensitivity and specificity do not depend on the prevalence of disease in the population, predictive values may vary with prevalence. In general, positive predictive values will increase as the prevalence of disease increases, while negative predictive values will decrease. The prevalence of skin cancer in our study was 72.6%, a rate higher than in most populations, because 1 of the criteria for entering the study was to have been given a diagnosis of skin cancer. If we exclude those patients, we find that the prevalence of chart-documented skin cancer in the remaining patients (who were seen by their dermatologist for a reason other than skin cancer) was 21.7% in our population. Using the figure of 21.7% as a rough estimate of the prevalence of a positive history of skin cancer in a general dermatology population, we found that the positive and negative predictive values for history of any skin cancer were 0.67 and 0.98, respectively. Thus, in a dermatology practice similar to ours, a patient who is not being seen for skin cancer but reports that she or he has a history of skin cancer would be giving a valid history about two thirds of the time. If that same patient denies a history of skin cancer, he or she will be correct 98% of the time. Our results can be used when considering likelihood ratios for the difference in pretest and posttest probabilities of a positive result of a history of skin cancer. In a patient with a high pretest probability of skin cancer (such as a patient from our study population), the positive predictive value is much higher than in a patient with a lower pretest probability of skin cancer.
Although male sex was associated with an increased likelihood of correctly reporting overall skin cancer status in our study, the confidence interval was wide, which reflects the small number of subjects who incorrectly classified their overall skin cancer status (19/256, or 7.4%). This study was insufficiently powered to make an accurate assessment of whether accurate skin cancer self-reporting is related to sex, and this question awaits further studies. There are similar concerns regarding the association found between age and reported squamous cell carcinoma status. In any event, the generally high rates of accurate reporting of skin cancer status decrease the importance of identifying patient populations with different rates of accurate reporting. Although male patient self-reports were more accurate, 88.8% of female patients correctly reported their overall skin cancer status.
As with any study, there are limitations in our analysis. We only examined dermatology patients at a single academic center, and it is not possible to know whether our findings can be generalized to patients in other academic centers or to patients in nonacademic centers. Our results may also not be generalizable to patients who receive skin cancer care from nondermatologists.
In addition, all of the patients in the present study participated in the larger study by completing the survey. While the demographic characteristics of patients who completed the survey and those who did not complete the survey are similar,15 it is possible that the 2 groups differ in important ways. Similarly, the demographic characteristics of patients with locatable charts and without locatable charts are comparable, but it is possible that these 2 groups may vary from each other.
We also do not know if patients were accurately told their skin cancer status by their physician. For example, some patients may never have found out the results of their biopsy, particularly if the lesion was treated at the same visit as the biopsy was performed. It is also possible that pathologic diagnoses such as "squamous cell carcinoma in situ" or "actinic keratosis with focal changes of squamous cell carcinoma," which were both considered malignancies in our study, might have been discussed with the patient in such a way that the patient did not consider them to be cancers. Conversely, "atypical melanocytic proliferation" or "hypertrophic actinic keratosis," both considered nonmalignant in our study, might be considered by some physicians to be an early stage of malignancy.
We were also unable to obtain cancer information for time periods outside of the period when patients were observed at our center, and we were also unable to determine whether patients saw other dermatologists during the time period when they were also observed at our center. Although the "best-case" scenario did not change most results greatly, the positive predictive value for melanoma changed markedly. Finally, because of the method by which our patients were enrolled, our study population had a higher rate of skin cancer than that found in the general population.
In general, we found that self-reported history of skin cancer had a high degree of sensitivity and specificity and a high negative predictive value. It also had a high positive predictive value in our study population, although that positive predictive value varied with differing prevalences of skin cancer. Thus, the time-honored method of obtaining medical information by patient report appears to be a valid way to determine patient history of skin cancer and is a useful tool for patient care.
Corresponding author and reprints: Michael E. Ming, MD, MSCE, Department of Dermatology, University of Pennsylvania Health System, 3600 Spruce St, 2 Maloney Bldg, Philadelphia, PA 19104 (e-mail: firstname.lastname@example.org).
A cooperative effort of the Clinical Epidemiology Unit of the Istituto Dermopatico dell' Immacolata–Istituto di Ricovero e Cura a Carattere Scientifico (IDI-IRCCS) and the Archives of Dermatology
Accepted for publication March 3, 2004.
This work was partially funded by a research grant from Novartis Pharmaceuticals to the Trustees of the University of Pennsylvania. Dr Ming is supported by a Clinical Career Development Award from the Dermatology Foundation.
A portion of this work was presented at the International Investigative Dermatology meeting; May 4, 2003; Miami, Fla.
We thank DuPont Guerry IV, MD, for his help in the preparation of the manuscript for this article.
Ming et al report that the self-reported history of skin cancer has a high degree of sensitivity and specificity in a study population from an academic medical center. The likelihood ratios, positive and negative (and 95% confidence intervals), for a history of nonmelanoma skin cancer were 7 (4-12.6) and 0.07 (0.04-0.13), respectively. The corresponding values for a history of melanoma were 18 (10.3-31.8) and 0.06 (0.01 to 0.42), respectively (see Simel DL, Samsa GP, Matchur DB. Likelihood ratios with confidence: sample size estimation for diagnostic studies. J Clin Epidemiol. 1991;44:763-770). Likelihood ratios higher than 10 or lower than 0.1 generate large and often conclusive changes from pretest to posttest probability. Likelihood ratios of 5 to 10 and 0.1 to 0.2 generate moderate shifts in pretest to posttest probability (for discussion of use and interpretation of likelihood ratios see Jaeschke R, Guyatt GH, Sackett DL, for The Evidence-Based Medicine Working Group. Users' guides to the medical literature, III: how to use an article about a diagnostic test, B: what are the results and will they help me in caring for my patients? JAMA. 1994;271:703-1707). Confirmation of the results reported in this article in other clinical settings and in population-based samples would be helpful to make the results generalizable.
Michael Bigby, MD