Participant recruitment and randomization flowchart.
Reductions in prostate-specific antigen (PSA) screening from pretest to posttest evaluation by group. Combined indicates combined traditional decision aid and chronic disease trajectory model. *P = .047 for interaction effect.
Concern about prostate cancer by group for those who chose or did not choose a prostate-specific antigen (PSA) test. Combined indicates combined traditional decision aid and chronic disease trajectory model.
Decisional conflict by group (higher scores indicate more decisional conflict). Combined indicates combined traditional decision aid and chronic disease trajectory model. *P< .05 vs other groups.
Frosch DL, Bhatnagar V, Tally S, Hamori CJ, Kaplan RM. Internet Patient Decision SupportA Randomized Controlled Trial Comparing Alternative Approaches for Men Considering Prostate Cancer Screening. Arch Intern Med. 2008;168(4):363-369. doi:10.1001/archinternmed.2007.111
We conducted a randomized controlled trial to evaluate the effects of patient decision support Web sites on decision quality for men considering prostate cancer screening.
Men older than 50 years (N = 611) were randomly assigned to 1 of 4 Internet conditions: traditional didactic decision aid providing information about prostate-specific antigen (PSA) screening options and outcomes; chronic disease trajectory model for prostate cancer followed by a time–trade-off exercise; both the didactic decision aid and the chronic disease trajectory model; or links to public prostate cancer–specific Web sites from credible sources (control condition). Participants completed questionnaires at baseline and after their physical examination. Primary outcome measures were PSA test choice, prostate cancer treatment preferences, knowledge and concern about prostate cancer, and decisional conflict.
Participants assigned to view public Web sites were less likely to review information (116 participants [76.8%] reviewed) than those assigned to experimental groups (399 [86.7%] reviewed; P = .004). Greater reductions in PSA screening from pretest to posttest were observed among participants assigned to the traditional decision aid (−9.1%) or chronic disease trajectory model (−8.7%), compared with participants assigned to the combination (−5.3%) or control (−3.3%) groups (P = .047). Preferences for watchful waiting increased significantly in all 4 groups (baseline, 219 [35.8%]; follow-up, 303 [66.2%]; P < .001). Knowledge scores were lowest for those assigned to public Web sites (mean [SD] score, 7.49 [0.19] of questions correct) and highest for the traditional decision aid (8.65 [0.18] of questions correct; P = .005).
Public Web sites about prostate cancer provide less effective decision support than a specially designed Internet decision aid.
Many medical decisions involve multiple options with no clear best choice, either because the available scientific evidence cannot identify one or because the choice is expected to vary depending on patient preferences.1 In these situations, a high-quality decision is defined as one made by a patient with a high level of decision-specific knowledge and that reflects the values and preferences the patient has for related outcomes.2 Recent data continue to suggest room for improvement in the quality of prostate cancer screening decisions. The benefits of population-based prostate cancer screening are uncertain because randomized controlled trials that demonstrate a significant benefit on all-cause mortality have yet to be completed.3,4 Men diagnosed as having prostate cancer must choose between multiple potential treatment options, and each choice is associated with significant risks of side effects that can impair quality of life. Because of these uncertainties, most professional groups recommend that men engage in informed decision making in choosing whether to be screened.5- 8 However, despite the caution from consensus panels, evidence suggests that many older men, for whom the potential benefits are limited, do undergo screening, often without any discussion of the pros and cons.9,10
The recommendation to engage in informed decision making presents significant challenges. Educating men about prostate cancer screening can easily expand a clinical encounter beyond the average 20 minutes available.11,12 Decision aids are tools that describe the benefits and drawbacks of different options without advocating a particular choice. A Cochrane review found that decision aids increase patient knowledge, make expectations more realistic, reduce decisional conflict, and frequently lead to different clinical choices.13 However, despite the evidence supporting the use of decision aids, their utilization in clinical practice remains low.14 The Internet presents a potential structural solution to providing access to decision aids by moving patient education outside of the clinic. Recent data indicate that 73% of Americans now access the Internet and many use it for health information.15 However, publicly available information is often inaccurate, difficult to follow, and confusing to patients.16
In this article, we report on a randomized controlled trial designed to evaluate 2 alternative informed decision-making interventions delivered through the Internet for men considering the prostate-specific antigen (PSA) test. Primary trial outcomes included the decision to receive the PSA test and its relationship to individual patient values, prostate cancer knowledge, treatment preferences, and decisional conflict.
The trial used a fully crossed 2 × 2 factorial design resulting in 4 groups. Factor 1 was a traditional didactic decision aid that provided comprehensive information about prostate cancer screening and treatment and included physician and patient testimonials contrasting different preferences and decisions.13 Factor 2 was a chronic disease trajectory model that activated patients to express utilities for outcomes associated with a prostate cancer life course by contrasting screening with no screening in its impact on quality of life and longevity. The intervention prompted men to complete a time–trade-off exercise and visual analog ratings of prostate cancer disease trajectories. Participants assigned to the control group were provided links to public Web sites on prostate cancer screening maintained by the American Cancer Society and the Centers for Disease Control and Prevention. The resulting 4 study groups were as follows: (1) traditional decision aid (TDA); (2) chronic disease trajectory model (DTM); (3) combined traditional decision aid and chronic disease trajectory model; and (4) control group. Of participants assigned to the combined condition, half received access to the TDA first and half received access to the DTM first.
Participants were recruited from the Health Appraisal Clinic of Kaiser Permanente, San Diego, California. The clinic conducts preventive screening examinations and has been described in detail elsewhere.17,18 The study protocol was approved by the institutional review boards of the University of California, San Diego, and Kaiser Permanente.
Men older than 50 years were sent a letter within 2 to 3 weeks before a scheduled appointment. The letter informed them of the availability of the PSA test as part of the health appraisal and a description of the study. Men received a unique arbitrary identification number and password, allowing them to anonymously access a secure Internet site that hosted all study materials. Participation required having broadband Internet access at home or at work. On the Web site participants found a brief study description. After log-in, the secure site presented the informed consent document. After indicating agreement, participants completed the preappointment questionnaire (pretest). A computer algorithm then randomly assigned participants to the 4 study groups.
Participants had unlimited access to their assigned intervention until the day of their appointment at the Health Appraisal Clinic. The Web site monitored whether participants reviewed the assigned intervention before their appointments. Men who had not clicked on the assigned link within a week before their appointment received an e-mail reminding them to review the intervention. After the clinic appointment had passed, participants received an e-mail asking them to return to the study Web site to complete the postappointment questionnaire (posttest).
Background questions included demographics, previous PSA tests, baseline screening preferences, and family and friends' history of cancer diagnosis. Our primary outcome was decision quality, a multidimensional construct that consists of several elements.2,19 First, a high-quality decision requires disease-specific knowledge. We measured knowledge with a validated 10-item questionnaire developed specifically for the context of prostate cancer screening.20 Second, a high-quality decision should show concordance between the patient's preferences and values for potential outcomes related to the decision and the choice made. A key factor in making a prostate cancer screening decision is how a man values the uncertain and potentially limited mortality benefit of screening relative to the high incidence of the malignant neoplasm. A man who has lower concern about the limited risk of dying of prostate cancer should be less likely to want to screen for it. Concern about prostate cancer was measured with an ordinal 5-point Likert scale from “not at all” to “extremely concerned.” Because previous studies have found that informed men are less likely to desire aggressive treatment for prostate cancer, we queried men's treatment preferences, with options of active treatment (surgery or radiation) and watchful waiting.18,21 Men also completed the Decisional Conflict Scale, a widely used measure that quantifies modifiable uncertainty experienced when choosing among different clinical options.13,22 The questionnaire has 5 subscales: (1) feeling informed, (2) values clarity, (3) support in decision making, (4) uncertainty, and (5) having made an effective decision. Men's PSA decisions were verified through medical records.
Effects of the interventions on the rate of PSA testing and treatment preferences were examined by means of logistic regression, controlling for the number of previous PSA tests. Pretest PSA testing or treatment preferences were entered in the analysis followed by the group assignment variables. We report odds ratios (ORs) with 95% confidence intervals (CIs) for each variable of the analysis. Rates of review of the educational materials in the 4 groups were also compared by logistic regression. Prostate cancer knowledge and decisional conflict were analyzed with factorial analysis of variance. The number of previous PSA tests was included as the covariate. Prostate cancer knowledge was analyzed by an intention-to-treat analytic approach. We imputed missing data for participants who did not complete follow-up assessments according to an approach used in a previously published trial.21 For participants who did not complete follow-up but reviewed the intervention materials, we substituted the posttest mean of the corresponding group. For participants who did not complete follow-up and did not review the intervention materials, pretest scores were carried forward. We examined the effect of these imputations by comparing these results with an analysis using uniformly imputed data, in which all missing data are replaced by pretest scores, and an analysis using complete cases only. The presence of a significant interaction effect in logistic regression or analyses of variance indicates that an independent variable has differential effects at different levels of another independent variable. Categorical variables with more than 2 levels were analyzed with Pearson χ2 tests.
Participants were recruited between March 14, 2005, and May 19, 2006. Figure 1 shows the randomization of participants into the 4 study groups, review of intervention materials, and completion of follow-up assessments. Before evaluating our main study hypotheses, we compared the intervention groups on demographic and other baseline characteristics (Table 1). Differences between groups were nonsignificant for all variables. We also compared demographic characteristics of those who completed follow-up assessments and those who did not. Participants who completed follow-up were more educated (P = .001), were more likely to have a personal (P = .009) or family (P < .001) history of cancer, and were more likely to have Internet access at home (P = .03).
Participants assigned to an experimental group were significantly more likely (399 participants [86.7]%) to click on the link for the assigned intervention (ie, TDA and/or DTM) than participants assigned to the control condition were to click on the public links to the American Cancer Society and Centers for Disease Control and Prevention Web sites (116 participants [76.8%]; P = .004). In the control group, 69 participants (45.7%) clicked on both links, and in the combined experimental condition, 32 (21.1%) clicked on both links. Participants in the control condition were significantly more likely (22 participants [23.4%]) to report accessing other Web sites than those assigned to experimental interventions (27 [8.1%]; P < .001).
At pretest, 588 participants (96.2%) indicated a preference for PSA screening. Controlling for the number of previous PSA tests, logistic regression analysis showed a significant drop in the number of men requesting a PSA test at the posttest (543 [88.9%] requesting; OR, 0.06; 95% CI, 0.02-0.15; P < .001). The next variables in the model were assignment to the TDA followed by assignment to the DTM (both variables coded dichotomously) and an interaction term. The effect of assignment to the TDA was marginally significant (OR, 0.426; 95% CI, 0.18-1.00; P = .051), and the effect of assignment to the DTM was nonsignificant. However, there was a significant interaction effect of the intervention assignment variables (OR, 3.31; 95% CI, 1.02-10.74; P = .047). As shown in Figure 2, men receiving a single intervention showed an approximately 2 times greater decline in PSA test requests than men assigned to the combined condition or the control group.
At baseline, 219 men (35.8%) indicated a preference for watchful waiting if diagnosed as having prostate cancer. Controlling for the number of previous PSA tests, logistic regression analysis showed a significant increase in the number of men expressing a preference for watchful waiting (OR, 12.62; 95% CI, 6.53-24.39; P < .001). However, there was no effect of assignment to the TDA or the DTM on posttest treatment preferences, and the interaction was nonsignificant. Across all 4 groups, 303 men (66.2%) at posttest indicated a preference for watchful waiting.
Our intention-to-treat analyses of prostate cancer knowledge did not differ by imputation method; we therefore focus our report on the differentially imputed data. There was a main effect of the TDA on knowledge (P = .003), but no effect of the DTM. There was also an interaction effect of the 2 group assignment variables (P = .005). Post hoc tests using the Tukey honestly significant difference test showed that the significant difference was between the TDA and control groups. Table 2 shows the proportion of men in the 4 groups answering each of the questions correctly.
There were no between-group differences at baseline or follow-up in how concerned participants were about prostate cancer. However, there was a significant decrease in concern about prostate cancer (P < .001). Figure 3 shows participants' ratings of concern about prostate cancer relative to their PSA decisions. Although the differences were not statistically significant, the results show that men assigned to the experimental interventions who chose not to have a PSA test rated their concern about prostate cancer lower than did men who requested a PSA test. By contrast, men assigned to the control group provided similar ratings of concern about prostate cancer, regardless of their PSA decision.
For the Decisional Conflict Scale, we found a significant effect of the TDA (P = .04) on the subscale measuring the sense of feeling informed, as well as a significant interaction effect (P = .04). There were also interaction effects on the subscales measuring values clarity (P = .01) and the sense of feeling supported in making a decision (P = .04). There were no significant effects for the subscales measuring uncertainty in making the decision and the sense of having made an effective decision. The significant differences are shown in Figure 4. High scores indicate more decisional conflict.
Our study is the first we know of that experimentally compared credible public Web sites with dedicated decision support interventions. Our findings suggest that guiding patients to public sources of information will not improve decision quality as much as a single, more intense decision support intervention. Participants assigned to the TDA group showed the highest prostate cancer knowledge scores and lowest decisional conflict scores, whereas the control group had the lowest prostate cancer knowledge scores. The TDA also appeared more effective than the DTM in improving prostate cancer knowledge and reducing decisional conflict. Men assigned to the control group did not show differential levels of concern about prostate cancer depending on what screening decision they made. By contrast, men assigned to the experimental interventions who chose not to screen indicated less concern about prostate cancer, suggesting better congruency between their values and their screening decisions than in the control group. Our combination condition appeared to provide too much information for participants. Only 21% of those assigned to this group reviewed both interventions, and prostate cancer knowledge scores were similar to those of the DTM group but lower than those of the TDA group.
The experimental interventions led to significant reductions in requests for PSA tests. Men assigned to a single intervention were less likely to choose PSA screening than men assigned to receive both experimental interventions or men assigned to the control group. Across all groups, there were also significant increases in the number of men expressing a preference for watchful waiting if diagnosed as having prostate cancer but no differences between groups. These results add to a growing body of literature that finds that informed patients frequently make different prostate cancer screening decisions and express more conservative treatment preferences than uninformed patients.23
In a previous study, our group compared Internet decision support with showing a video in clinic before an appointment.21 A substantial number of men assigned to the Internet group did not review the intervention before coming to the clinic, rendering it less effective. Building on this finding, the intervention Web site in the present trial monitored whether patients had reviewed the decision aid as their appointments approached. Reminders were sent to participants to increase use of the interventions. The reminders appeared to be effective, because 84.3% of participants reviewed the interventions, compared with only 53.5% of those assigned to the Internet group in our previous trial.21 Nevertheless, participants assigned to the control group in the present trial were significantly less likely to review the assigned interventions and more likely to use other information sources outside of the study materials.
Our study has several limitations. First, 74% of participants completed the posttest. This rate was good in comparison with other Internet trials, but not ideal.24 Our sample was predominately white and highly educated, limiting our ability to generalize our findings to older men, men from ethnic minority groups, and those with lower health literacy or computer literacy. Although similar to that in a previously published trial, our relatively low recruitment rate further limits our ability to generalize our findings.21 It is possible that men participating in this trial were more interested in prostate cancer screening than were men who did not enroll. Participants were recruited from a clinic focused on preventive health care and are likely to have greater concern about preventive health care than the general population is.
Participants assigned to the control group were significantly less likely to access the assigned decision support intervention. It is unclear to what degree the observed differences compared with the experimental groups are a function of the content of the control group intervention vs men being less likely to access the intervention in the first place. We speculate that some participants in the control group may have thought that public Web sites did not have anything new to offer to them. Although we think it unlikely, we cannot rule out the possibility that the results were affected by differential dropout.
Because of concerns about statistical power, we did not experimentally test the effects of reminders on uptake of the interventions. However, both this and our earlier trial took place in the same clinic, making it unlikely that differences in intervention uptake were attributable to varying patient populations. Our measure assessing patient values pertinent to the prostate cancer screening decision was limited to general concern about prostate cancer and did not assess specific values such as concern about sexual and urinary function or premature mortality. The differences in concern about prostate cancer among men who decided in favor of and against screening were not statistically significant, and our trial was not adequately powered for this outcome measure. Overall, however, the direction of our findings does suggest that participants assigned to the experimental interventions showed better congruence between their decisions and their concern than did those assigned to the control group.
The increased use of the PSA test in the past 15 years has identified many early-stage prostate cancers that pose a limited risk to men's mortality.25 In light of this, one might expect that the use of watchful waiting in managing prostate cancer would have increased. However, data from a national prostate cancer registry show that the opposite has happened, strongly suggesting less than optimal decision quality in prostate cancer care.25 Our trial suggests that a single dedicated Internet decision support intervention can increase decision quality at the earliest stage of prostate cancer care. By contrast, reputable public Web sites did not perform as well, as illustrated by lower prostate cancer knowledge scores and a lack of congruence between patient values and decisions in the control group.
Correspondence: Dominick L. Frosch, PhD, Division of General Internal Medicine and Health Services Research, Department of Medicine, University of California, Los Angeles, 911 Broxton Ave, Los Angeles, CA 90024 (firstname.lastname@example.org).
Accepted for Publication: September 26, 2007.
Author Contributions:Study concept and design: Frosch, Bhatnagar, Hamori, and Kaplan. Acquisition of data: Frosch, Bhatnagar, Tally, Hamori, and Kaplan. Analysis and interpretation of data: Frosch and Kaplan. Drafting of the manuscript: Frosch and Kaplan. Critical revision of the manuscript for important intellectual content: Frosch, Bhatnagar, Tally, Hamori, and Kaplan. Statistical analysis: Frosch and Kaplan. Obtained funding: Frosch and Kaplan. Administrative, technical, and material support: Frosch, Tally, Hamori, and Kaplan. Study supervision: Frosch, Hamori, and Kaplan.
Financial Disclosure: None reported.
Funding/Support: This study was supported by cooperative agreement U57/CCU920678 from the Centers for Disease Control and Prevention. Dr Frosch also received support from the Robert Wood Johnson Foundation Health & Society Scholars Program.