Research on Skin Cancer–Related Behaviors and Outcomes in the NIH Grant Portfolio, 2000-2014: Skin Cancer Intervention Across the Cancer Control Continuum (SCI-3C)

Key Points

Question  What are characteristics of the National Institutes of Health–funded behavioral intervention research portfolio across the skin cancer control continuum?

Findings  In this portfolio analysis of 112 grant applications for the years 2000 to 2014, 40 grants (35.7%) were funded, and male and female investigators did not differ in overall success rates. Less than half of the grants attempted to link behavior change to alteration in clinically relevant targets, and no grants addressed emotional sequelae or adherence behavior related to diagnosis or treatment.

Meaning  Gaps in intervention-related skin cancer research exist and should be addressed.

Importance  The Surgeon General’s Call to Action to Prevent Skin Cancer broadly identified research gaps, but specific objectives are needed to further behavioral intervention research.

Objective  To review National Institute of Health (NIH) grants targeting skin cancer–related behaviors and relevant outcomes.

Design, Setting, and Participants  A portfolio analysis of the title, abstract, specific aims, and research plans of identified grant applications from 2000 to 2014 targeting skin cancer–related behaviors or testing behavioral intervention effects on cancer-relevant outcomes along the cancer continuum.

Main Outcomes and Measures  Funding trends were compared along the cancer control continuum, with respect to investigator demographics and use of theory, technology, policy, and changes to environmental surroundings (built environment).

Results  A total of 112 submitted applications met inclusion criteria; of these, 40 (35.7%) were funded, and 31 of the 40 were interventions. Comparing the 40 funded grants with the 72 unfunded grants, the overall success rates did not differ significantly between male (33.3%) and female (37.3%) investigators, nor did the frequency of R01 awards (36.7% and 28.1%, respectively). Among intervention awards, most (24 of 31) addressed prevention. Fewer awards targeted detection alone or in conjunction with prevention (3) or cancer survivorship (4), and no grant addressed emotional sequelae or adherence behavior related to diagnosis or treatment. Fewer than half of funded grants aimed for clinically related targets (eg, sunburn reduction). Use of theory and technology occurred in more than 75% of grants. However, the full capability of proposed technology was infrequently used, and rarely did constructs of the proposed behavior change theory clearly and comprehensively drive the intervention approach. Policy or environmental manipulation was present in all dissemination grants but was rarely used elsewhere, and 19.4% included policy implementation and 25.8% proposed changes in built environment.

Conclusions and Relevance  Grant success rate in skin cancer–related behavioral science compares favorably to the overall NIH grant success rate (approximately 18%), and the success rate of male and female investigators was not statistically different. However, gaps exist in behavioral research addressing all points of the skin cancer control continuum, measuring interventions that hit clinically related targets, and leveraging technology, theory, and environmental manipulation to optimize intervention approach.

The Surgeon General’s Call to Action to Prevent Skin Cancer (SG-CTA) broadly identified research gaps for skin cancer prevention that rely on behavioral intervention.^1,2 However, as the principal funder of behavioral intervention research for skin cancer, it is necessary for the National Cancer Institute (NCI) to relate broadly identified research gaps as specific research needs to the behavioral research investigative and dermatology communities. Conceptualizing skin cancer intervention across the 5-point cancer control continuum (SCI-3C) provides a rubric to determine the degree to which intervention targets are being addressed in a research portfolio, and to assess the translational science phase of research (Figure 1).^3,4 Models identifying distinct intervention targets at each point along the cancer control continuum exist in other domains (eg, physical activity),^5-7 but no such model exists for skin cancer intervention research. Furthermore, knowing how grants use behavioral theory, technology, and incorporate built environment (changes to environmental surroundings) and policy environment offers insight into how these features may improve intervention reach and potency.^8-14 That is, such approaches can alter behavior for large segments of the population, or these elements can be used to tailor interventions to individuals or microenvironments. Finally, developing the next generation of skin cancer investigators is also central to the NCI mission. Considering the concerns raised regarding National Institutes of Health (NIH) support for skin cancer–related research, especially for women investigators, it is important to understand funding success generally and by specific demographics.^15,16

This report describes the NIH-funded behavioral intervention research portfolio from funding years 2000 to 2014 and identifies research gaps by points along the cancer control continuum to accelerate translation of scientific findings into practice.

A keyword search including “skin cancer,” “melanoma,” “sun protection,” “sun safety,” “tanning,” and “UVR exposure,” combined with “humans” identified extramural grant submissions in the NIH’s Information for Management, Planning, Analysis and Coordination (IMPAC II) grant database received between 2000 and 2014. The search yielded 49 funded grants (all but 1 by NCI) and 82 unfunded grants (Figure 2) with revisions collapsed into 1 application. Grant abstracts, project titles, and specific aims were reviewed to identify applications focused on prevention and control of skin cancer across the cancer control continuum. Nine funded grants and 10 unfunded grants were excluded from further consideration because their content was not focused on behavioral science. Of the remaining 40 funded grants (31 interventional-experimental and 9 observational), only the 31 funded intervention grants received further intervention characteristic coding, which included review of the entire research plan.

In addition to grant characteristics (eg, grant mechanism, investigator status, and general demographic and methodological components), intervention grants were coded on (1) inclusion of technology, (2) manipulation or introduction of the built environment or a policy, and (3) incorporation of theory and theoretical constructs. In addition, we coded point along the cancer control continuum, primary intervention target or outcome as specified in the specific aims, and study phase along a translational science continuum from T0 to T4.^3,4,17,18

Coding procedures were adapted from prior NCI grant portfolio analysis and developed as follows.^11,17 First, behavioral intervention scientists within the NCI with expertise in skin cancer (F.M.P., L.A.D, J.M.T., W.E.N., A.M.H., and an extramural skin cancer investigator (A.C.G.), developed a codebook of grant features, coding criteria, and decision rules. Owing to proprietary content, NCI staff conducted all grant coding. Next, 5 grants were pretested by 4 NCI coders and discussed to clarify criteria and exemplars, refine decision rules, and enhance consistency prior to coding additional grants. All grants, including the 5 pretested grants, were then independently double-coded using all possible combinations of coders. Discrepant codes were resolved by discussion resulting in 100% agreement. Across all grants and coding items (265 possible), 68.4% had good to excellent interrater reliability (Fleiss κ > 0.40) prior to discussion.¹⁹ Items concerning application of theory generally had poorer agreement and required discussion. The codebook is available online in the online Supplement. In this subsection we describe coding for inclusion of technology, manipulation of the environment or policy, and incorporation of theory.

We coded for the presence and type of skin cancer–specific and non–cancer-specific technology, such as infrared and/or UV photography, light dosimeter, dermoscope, reflectance spectroscopy, text messaging, mobile applications, internet and/or email, videos, ecological momentary assessment and/or daily diaries, social media, or other.

Environmental manipulations were coded if interventions sought to manipulate either policy or a feature of the built environment. Policies were defined as broad, local, or specific policies in workplaces, schools, or health care settings. The built environment included architectural and landscaping features, shade structures, or signage. Coders determined whether the intervention used policy or a feature of the built environment to address (1) access to indoor tanning; (2) use of sun-protective behaviors (clothing, hats, eyewear); (3) sunscreen use; (4) sun exposure, or other, and (5) whether the built environment changed, altered, or created shade structure(s), planting or shaded-trail use, or architectural features to minimize sun exposure; and (6) signs to prompt behavior, or other.

Theory was coded to reflect whether the intervention content or methodology was “based on any theory (ie, does any theory drive the intervention?),” assessed theoretical constructs, or included mediation testing as a specific aim. Mediation analyses seek to explain the mechanism or process (represented by a third mediating variable) by which the independent variable (ie, intervention) influences the dependent variable (ie, skin cancer-relevant target outcome).²⁰ Coders also determined the extent to which specific constructs mapped onto core elements of the intervention.

Portfolio analyses are exploratory by nature, rarely contain a priori hypotheses, and have a large number of comparisons. With 1 exception, we report only descriptive analyses. Because a recent study reported a sex disparity in NIH funding success to dermatology departments,¹⁶ χ² analyses compared the funding success rate of male and female behavioral researchers.

Of the 112 NIH skin cancer–related grant applications meeting inclusion criteria, 40 (35.7%) were funded and 31 of these 40 met intervention research criteria (Figure 2). Women submitted more applications (59.8% [67 of 112]) than men (40.2% [45 of 112]) (P = .047), but there was a nonsignificant difference in the percentage of funded applications submitted by women (37.3% [25 of 67]) and men (33.3% [15 of 45]) (P = .67). The percentage of successful R01 applications submitted by men (36.7% [11 of 30]) and women (28.1% [9 of 32]) was also not significantly different (P = .47). Among those applications receiving funding (Table 1), the proportion of total awards to men (37.5% [15 of 40]) and women (62.5% [25 of 40]) was not statistically different (P = .11). There were similar proportions of men and women receiving R01 awards (45% and 55%, respectively) and new investigator awards (33% and 36%, respectively) (see eTable in the Supplement). Among the 8 investigators receiving multiple grants, 6 were women, 5 of whom received an R01 grant.

Among the 40 awarded grants, 50% were R01s, and 60% of the awards were related to prevention (Table 1). The overall number of awards increased over the three 5-year periods from 3 to 15 to 22, and R01 awards similarly increased, especially in the 2010-2014 period. Eighty percent of awards (32 of 40) went to investigators holding a PhD or equivalent degree.

Among the 31 funded intervention grants, 24 were related to prevention. Three awards included work related to detection and/or screening (2 of which also included a prevention aim). Three of the 4 survivorship-related grants also included a detection aim (ie, skin examination by self, partner, or physician). The number of intervention-related awards increased over the three 5-year periods from 2 to 10 to 19, and R01 awards similarly increased from 2 to 4 to 14 over the award periods (Table 1).

A broad examination of Table 2 reveals funded behavioral intervention grants pertained to 3 of the 5 points along the cancer control continuum, but no funded grants addressed behavioral factors associated with diagnosis or treatment. The prevention focused grants sampled a wide age range of participants, but children were absent from survivorship research. Across the cancer control continuum, 17 of 31 grants (54.8%) targeted sun protection as a primary aim, but only 8 (25.8%) included sunburn or other objective measure of outdoor UV exposure. Indoor tanning was targeted in 5 grants (16.1%). Six grants (19.4%) included some form of skin examination. Psychosocial outcomes were targeted as a primary outcome in 6 grants (19.4%). Most projects were at early translational T1 and T2 stages (45.2% and 35.5%, respectively). Six projects were focused on dissemination and implementation (T3 or T4).¹⁸ Most grants included an information delivery approach. General health messaging about skin cancer risk and behavior was the primary intervention approach in 22 grants (71%); more specific, tailored approaches were used in 12 projects (38.7%) (Table 2).

Mention of a conceptual framework (30 of 31 [96.7%]) and general rationale for the theories or theoretical constructs used (27 of 31 [87.1%]) occurred in almost all of the funded intervention projects (Table 3). However, less than a quarter of these projects (22.6%) mapped the named theoretical constructs clearly and comprehensively to the proposed intervention approach, and mediation analysis occurred as a specific aim in less than half of the projects (45.6%).

Over the portfolio period, most of the grants (26 of 31 [83.9%]) incorporated some form of general technology (eg, use of the internet in 19 of 31 [61.3%]), typically to present information on skin cancer risk behavior, demonstrate protective behavior, or convey a story (Table 3). The proportion of any technology use increased over the 3 time periods and was generally consistent with the number of awards (eg, 17 of the 19 awards from 2010 to 2014 included technology use vs 7 of 10 awards from 2005 to 2009). Use of text messaging, mobile applications, or social media occurred in less than 10% of grants overall and were largely confined to the 2010-2014 award period. Use of skin cancer–specific technology occurred in 11 grants (35%), with reflectance spectroscopy reported most frequently (6 grants [19.4%]).

Manipulation of the built or policy environment was proposed in all 6 of the dissemination-related projects but was proposed in only 2 other grants. Policy approaches involved change in workplace or school procedures and participant education. Change to the built environment involved introduction of signs promoting sun safety or some form of shade structure.

In reference to the overall NIH funding success rate, approximately 18%,²¹ the success rate (37.5%) of behavioral science grants in skin cancer seems high, and R01 success rates were also similar between men and women, which contrasts with reports of sex disparity for general dermatology–related research.^15,22 However, success rate can be relatively more influenced when there are relatively few applications. Coinciding with recent reviews of the intervention literature related to skin cancer,^23-26 our portfolio analysis identified 3 notable gaps in intervention-related research: (1) lack of research at the diagnosis and treatment points in the cancer control continuum, (2) few projects aimed at clinically related targets, and (3) possible suboptimal leveraging of technology, theory, and environmental approaches.

With respect to the cancer control continuum, most of the portfolio concerned prevention, and in comparison with other health behavior research, there was good representation across the translational research continuum.^4,17 However, unlike behavioral research with other types of cancer, no skin cancer grants were related to diagnosis or treatment in which behavioral intervention addresses amelioration of distress, adverse events, or treatment compliance.²⁷ Behavioral interventions may also improve diagnostic accuracy by evaluating different teaching methods and use of visualization technology. In turn, this work may inform criteria, training protocols, and deployment decisions regarding new diagnostic approaches, such as dermoscopy. Such screening methods may become increasingly important for larger segments of the population given recent US Food and Drug Administration recommendation for regular skin examinations among those who have tanned indoors.^28,29

The portfolio analysis and SCI-3C logic model suggest that the next generation of prevention research demonstrates effects beyond achieving statistically significant change in target sun safety behaviors, and aim to demonstrate improvements in clinically related proximal targets (eg, reduction in sunburn and/or tanning) associated with skin cancer.²⁴ For example, despite the number of prevention grants and survivorship projects addressing UV safety, relatively few grants addressed indoor tanning or included change in objective or clinically related targets of sun exposure (ie, sunburn) as a primary aim. Research targeting these outcomes is needed because indoor tanning rates among young adults and teens (eg, 20.2% overall and 30.7% among non-Hispanic white teens) are a cause for concern, and proposed regulation of indoor tanning will not address teens older than 17 years or restrict access to many tanning sites.^10,30,31 Concomitantly, research addressing measurement, especially the integration and benchmarking of self-report and objective assessment of exposure (eg, UV radiation, sunburn) and behavior (eg, sunscreen application), will facilitate intervention and surveillance approaches. Furthermore, unlike recommendations for other cancer-related health behaviors (eg, physical activity for 150 minutes/week), an evidence-based, minimally effective dose of sun protection remains elusive, and with few exceptions, reported sun-protective behavior has not been commonly related to clinically relevant outcomes, such as sunburn or melanoma.^32-34 Research to improve measurement approaches deployable on a large scale that better captures the temporal relationship between behavior and sunburn and/or skin-darkening and defines the context of sunburn will likely improve behavior-outcome relationships. However, many individuals will be unlikely to adhere optimally to sun safety recommendations.³⁴ Future research must demonstrate that interventions are potent enough to hit clinically related targets (eg, reduced sunburn), as well as to achieve a statistically significant change in sun-protective behavior, particularly among those who have incurred sunburn (eg, approximately half of all adults and 65% of white adults) and those at heightened risk owing to their behavior (eg, regularly physically active).³⁵

Incorporation of theory and attendant constructs may improve behavioral intervention.³⁶Almost all funded intervention projects were informed by behavioral theory, but they relied heavily on nontailored information delivery rather than using theory to select salient points that may differ across individuals. Furthermore, interventions rarely examined the pairing of health and risk communication with a behavior change approach (eg, health messaging and motivational interviewing). Approximately a quarter of the funded interventions also did not integrate theory by having constructs comprehensively map to intervention approaches, or test whether an intervention effect on an intermediary marker linked the intervention to the intended target behavior or outcome. Determining the intervention effect on the explicit or implied mechanism of behavior change is important considering that most funded grants were at an early translation stage (eg, T1 and T2). While sample size may preclude traditional mediation analyses, small sample approaches and even single-subject designs can be used for this purpose.^37,38 The exclusion of mediation analyses are in line with reviews of behavior research and the trans-NIH Science of Behavior Change Initiative.^11,36,39 Incorporation of these intervention fidelity steps and mediation assessments in future research will better inform if interventions function as intended.

Manipulation of the built or policy environment principally involved use of signage, specific shade structure, and education at worksites and schools. No grants tested availability of sunscreen dispensers, which may better promote sun protection, or architectural site-planning to increase shade in overall design consideration. When included, use of shading was typically isolated (eg, waiting line area) rather than comprehensive. Only 2 funded grants sought to determine how behavior changed in microenvironments (shaded and unshaded). These data are vital because shade structures or plantings may be used more frequently if they can increase commercial patronage or facility use. Opportunity for skin cancer research and public health occurs at multiple levels of influence,¹ and may also coincide with efforts to shape the built environment to promote physical activity, healthy eating (eg, farmer’s markets), or attendance at community events. Natural experiment research to capture sun-relevant behavior in these settings may provide synergy with these other healthy lifestyle approaches.⁴⁰ While it is understandable that policy and environment manipulations would occur in dissemination and implementation studies, these approaches are ripe for intervention proof of concept testing (ie, T1 and T2) but were rarely used.

Use of technology in some form was present in almost all grants, and seemed to increase in the 2010-2014 period, but skin cancer–specific technology occurred in only approximately one-third of intervention grants. Given the promise of dermoscopy^28,29 and the need for objective UV exposure and accompanying methodology, opportunities exist to better incorporate and test skin cancer–specific technology in future research. While general technology use was common and typically involved the internet and incorporation of informational videos, full internet capability and mobile technologies were not optimally leveraged. For example, mobile applications and social media functions were rarely incorporated, and text messaging was used infrequently. Technology platforms integrating sensor, behavioral, affective, social, and location data are used with other health behaviors and offer promise for skin cancer intervention.^41,42 For example, mobile platforms, ideal for just-in-time interventions, are applicable to skin cancer because UV exposure often occurs in fixed locations. While initial studies yielded only modest success for sunscreen use,⁴¹ other research supports incorporation and integration of affective or social data, obtained by self-report or inferred from passive data collection, that may enhance delivery of just-in-time intervention.^43-45 Such approaches have been used to reduce message fatigue on an individual level, among college students possibly at risk for UV exposure, and can possibly be harnessed to survey public dialogue and perceptions of public health approaches for cancer control.^43-45

Limitations in our data warrant consideration. Portfolio analysis procedures are evolving, some coding discrepancies existed especially concerning use of theory, and a small grant sample limits data analyses and conclusions drawn herein. Mediation analyses may have occurred outside of a grant’s specific aims. Resource availability also limited full coding only to funded grants; it is unknown if unfunded grants differed in their use technology, theory, and environmental manipulation.

Providing evidence to advance skin cancer–related behavioral intervention requires an understanding of the NIH-supported state-of-the-science, analysis of gap areas, and a coordinated research agenda for the future. Our review suggests a need for intervention research to aim for proximal clinically relevant targets as well as behavioral targets at all points across the cancer control continuum.²⁴ Similar to findings of a recent review of the skin cancer intervention literature, increased use of behavioral theory, technology, and manipulation of the built and policy environments should be explored as a means to improve intervention reach and impact.²⁶

Corresponding Author: Frank M. Perna, EdD, PhD, Behavioral Research Program, Division of Cancer Control & Population Sciences, National Cancer Institute, 9609 Medical Center Dr, East Tower, Room 3E104, Bethesda, MD 20892 (pernafm@mail.nih.gov).

Accepted for Publication: December 26, 2016.

Published Online: March 22, 2017. doi:10.1001/jamadermatol.2016.6216

Author Contributions: Drs Perna and Dwyer had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Perna, Dwyer, Tesauro, Taber, Hartman, Geller.

Acquisition, analysis, or interpretation of data: Dwyer, Tesauro, Taber, Norton.

Drafting of the manuscript: Perna.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Perna, Dwyer, Taber.

Obtained funding: Perna.

Administrative, technical, or material support: Perna, Tesauro, Norton.

Supervision: Perna.

Conflict of Interest Disclosures: None reported.

Funding/Support: This work was partially supported by contract HHSN2612012000028I to the Westat Corporation and a Westat subcontract to Mr Geller.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The opinions in this article are those of the authors and do not represent the NCI or the US Government. The NCI reviews and approves manuscripts prior to submission by their staff scientists.

Additional Contributions: We thank Tracey Goldner, MA, and Kasey Morris, PhD, both employees of the NCI, for preparing the figures and editing earlier versions of this report. No additional compensation was provided for their contributions.