Measuring the Severity of Infantile Hemangiomas: Instrument Development and Reliability

Objectives To develop instruments that measure the severity of infantile hemangiomas (Hemangioma Severity Scale [HSS]) and the complications of infantile hemangiomas for longitudinal use (Hemangioma Dynamic Complication Scale [HDCS]).

Design Instrument development and reliability study.

Setting Academic research.

Participants The HSS and the HDCS were developed through the collaborative effort of members of the Hemangioma Investigator Group Research Core, an expert multi-institutional research group. After development of the scales, 13 pediatric dermatologists used the HSS to score 20 different hemangiomas. In addition, 12 pediatric dermatologists used the HDCS to score hemangioma-related complications for 24 clinical scenarios. Interrater and intrarater reliability was measured for both scales.

Main Outcome Measures Interrater and intrarater reliability.

Results For the HSS, interrater reliability and intrarater reliability exceeded 99%. Similarly, the HDCS had a high rate of interrater agreement; for individual items, agreement among raters was 67% to 100%, with most clinical scenarios demonstrating greater than 90% agreement. Intrarater reliability was excellent for all individual items of the HDCS.

Conclusion The HSS and the HDCS are reliable scales that can be used to measure the severity of infantile hemangiomas, including the severity of complications for longitudinal use.

Infantile hemangiomas (IHs) are the most common benign tumors of infancy. They range from small, innocuous, localized papules on the face, trunk, or extremity to large, disfiguring, and function-threatening vascular tumors of the head and neck. Attempts to describe the heterogeneity of IHs include classification schema commonly cited in the literature based on depth (superficial, deep, or mixed), morphologic structure (localized, segmental, indeterminate, or multifocal), and clinical behavior. In addition, the growth cycle is typically divided into early proliferative, late proliferative, plateau, and involuting stages, but the duration of these phases can vary among hemangioma subtypes.¹ Of note, some hemangiomas are inherently more complicated and may be cutaneous markers for neurologic, ocular, arterial, or cardiac anomalies (PHACES syndrome; OMIM 606519) or the presence of visceral or airway hemangiomas. Given the wide clinical heterogeneity of hemangiomas, it is imperative to carefully assess the risk associated with an individual hemangioma to help guide clinical decision making, including the need for referral to an expert individual or center and the choice of treatment. In the research setting, risk stratification is critical when designing and conducting clinical trials.

For several decades, the literature has been sparse for new treatments for IHs. Most complicated IHs were treated with high-dose glucocorticosteroids and less commonly with interferon or vincristine. However, since the discovery that systemic β-blockers are efficacious for the treatment of IHs, there has been a surge in the literature, including case reports and case series of patients treated with systemic β-blockers. A multicenter international randomized controlled trial is under way examining the safety and efficacy of this treatment (Study to Demonstrate the Efficacy and Safety of Propranolol Oral Solution in Infants With Proliferating Infantile Hemangiomas Requiring Systemic Therapy [HEMANGIOL]; clinicaltrials.gov identifier: NCT01056341).

As the number of prospective studies of IHs increases, it is clear that validated and reliable instruments to measure disease severity are needed for use in clinical trials. The Hemangioma Investigator Group Research Core (HIG-RC) has developed the following 2 scales to measure the severity and complications of hemangiomas: the Hemangioma Severity Scale (HSS), which measures the overall severity of an IH, and the Hemangioma Dynamic Complication Scale (HDCS), which assigns severity grades to hemangioma complications.

The HSS and the HDCS were developed by the HIG-RC, a multi-institutional research group of 14 pediatric dermatologists and 1 hematologist/oncologist representing 9 academic medical centers in the United States. Institutional review board approval was obtained for this study.

To draft the HSS, HIG-RC members identified clinical hemangioma variables and assigned values to the morbidity associated with each variable based on published work and clinical experience. The HSS was drafted, and using a database of archived photographs with known outcomes, the group compared severity scores of a wide variety of hemangiomas based on the drafted instrument. Modifications to the HSS were made by discussion and consensus of the expert group. Content validity of the HSS is ensured by a comprehensive literature review and adequate clinical input from multiple expert clinicians.

To assess the ease of use and practicality, the drafted HSS was tested in a pilot study by 10 members of the HIG-RC, who applied the HSS to 20 clinical scenarios with accompanying photographs. The results of the pilot study led to minor revisions before field testing. Clinical variables were categorized as objective (size, location, presence of risk factors for structural anomalies, and complications) and as subjective (pain and risk of disfigurement). Subscores were recorded for the objective and subjective portions of the scale, in addition to a total severity score that represented the sum of the objective and subjective variables (Figure 1).

A subcommittee of the HIG-RC similarly developed the HDCS. This scale provides a 6-point severity grading system for 12 individual hemangioma-related complications from the HSS (grade 0 represents absent to minimal; grade 5, the most severe complication), allowing for longitudinal follow-up study of IHs in clinical or research settings. The grading scales were written and stratified based on the clinical experience of the expert group and were modeled on existing grading scales for reporting of adverse effects in clinical trials (the National Cancer Institute guidelines for Common Terminology Criteria for Adverse Events [CTCAE], version 4.0; http://ctep.cancer.gov/protocolDevelopment).

A manual was created to accompany the HSS and the HDCS to facilitate and explain their use. The manual is available online (eAppendix). The manual included detailed instructions for individual items and provided definitions of terms (eg, central face, peripheral face, nasal tip, and lip) (Figure 2). Clinical examples were included to address any potential ambiguities.

Thirteen manual-trained investigators, 12 pediatric dermatologists, and 1 hematologist/oncologist assigned severity scores for 20 hemangiomas in heterogeneous clinical scenarios using accompanying photographs of patients with known outcomes from previous HIG-RC studies. These investigators were not members of the HIG-RC, were not involved in HSS development or pilot testing, and were not participants in previous HIG-RC studies. They had a mean of 11.5 years of clinical practice experience (range, 1.5-25.0 years). The investigators completed 2 scoring sessions, with the second scoring session occurring 7 to 10 days after the first. Interrater and intrarater reliability was measured using the intraclass correlation coefficient, which was estimated as the proportion of variance relative to a clinical scenario from a variance components analysis that included effects for scenario, rater, testing session, and all interactions. The subjective and objective portions of the HSS and the total severity score were analyzed separately.

Twelve members of the HIG-RC, not involved in HDCS development, used the HDCS (Figure 3) to evaluate 24 unique clinical scenarios with accompanying photographs on 2 separate occasions 7 days apart. For complications that could vary in severity or clinical presentation, more than 1 clinical scenario was developed. For example, 4 cases addressing ulceration were developed to represent various anatomic locations, depth of ulceration, and associated symptoms, including pain and effect of daily activities, such as feeding. Intrarater reliability was measured for individual items of the HDCS using a weighted κ statistic. The absolute percentage agreement between occasions was also calculated for each item of the HDCS. Interrater reliability for each HDCS item was summarized as the frequency at the mode (most common score for that item) divided by 12.

Twenty cases were evaluated by 13 raters on each of 2 occasions for estimation of interrater reliability and intrarater reliability. Both exceeded 99% for objective, subjective, and total severity ratings.

Twelve reviewers evaluated 24 clinical scenarios on 2 occasions. Intrarater reliability is shown in the Table. Because of the low level of variability in scores, the κ statistic was generally not estimable, and percentage agreement is presented instead. Intrarater reliability (as given in the percentage agreement column) exceeded 90% for 16 clinical scenarios tested and was less than 70% for 1 clinical scenario (related to cartilage distortion and destruction on the ear). Similar results were found for the agreement among raters for all clinical scenarios, where 67% to 100% agreement was seen at the first occasion, with most scenarios demonstrating greater than 90% agreement. In the analysis among raters for all clinical scenarios, the only scenario with agreement of less than 70% was related to 2 scenarios, one with bilateral mandibular –airway complications and the other with cartilage distortion and destruction on the ear.

The HSS and the HDCS comprise outcome measures designed by expert clinicians based on published work and clinical experience. Their high reliabilities support their potential use in clinical studies of IHs. The HSS provides a severity score for an individual hemangioma based on risk stratification of numerous clinical variables, while the adjunct HDCS assigns a grade to each complication for longitudinal use.

Quiz Ref IDThe primary objective variables of the HSS include hemangioma size, location, and the presence of established risk factors for systemic anomalies. Larger size and facial location portend a higher risk for complications.² For the HSS, 2 distinct size classification schemas were developed to distinguish the severity of a hemangioma located on the head and neck vs similar-sized hemangiomas elsewhere on the body. For example, a 5-cm hemangioma on the face is assigned a higher severity score than a 5-cm hemangioma on the trunk. Involvement of critical facial locations, such as the periorbital or lip regions, merits a higher severity score because of the potential effect on vital functions, including vision and feeding. Facial location is considered an important factor for assessing the potential for long-term disfigurement. Specifically, the effect on hemangioma severity was assumed by the investigators to be greater for hemangiomas at the central portion of the face than for those at the lateral portions of the face; specific involvement of periocular skin, nasal tip, ear, and lip are considered the most severe. Therefore, these locations were weighted accordingly relative to their higher risk for disfigurement and function impairment.

Quiz Ref IDAlthough extremity and trunk hemangioma locations were considered less worrisome, important subsets, including anogenital hemangiomas, breast hemangiomas, and lumbosacral hemangiomas, were thought to be more concerning. Anogenital hemangiomas frequently ( ≤50%) ulcerate, resulting in pain, bleeding, and potential infections.³ Breast hemangiomas, especially those with a significant deep component, can potentially interfere with normal breast bud development in girls in the future. Last, lumbosacral hemangiomas may be associated with underlying spinal anomalies in approximately 30% to 50% of individuals.⁴

Quiz Ref IDCertain hemangiomas represent markers for internal anomalies. PHACES syndrome is present in 31% of individuals who have a facial hemangioma of 22 cm² or larger.⁵ Recently published diagnostic criteria for PHACES syndrome consider at-risk hemangiomas to be facial hemangiomas exceeding 5 cm in diameter.⁶ The HSS assigns higher severity to such hemangiomas to account for the extensive evaluation needed and the higher risk of significant cerebrovascular and cardiac anomalies. Awareness of higher HSS severity in these patients will help ensure appropriate clinical management and choice of treatment. Similar importance was given to lumbosacral and anogenital hemangiomas, which can be associated with regional anomalies, including spinal dysraphism and anomalies of the gastrointestinal and genitourinary system.⁷

Quiz Ref IDThe risk of long-term disfigurement is the most common reason why hemangiomas are treated.² Although hemangiomas involute over time, they often leave fibrofatty residua. In a thin plaque-type hemangioma, the fibrofatty residua may manifest as minimal textural change with or without telangiectasias. However, more exophytic lesions and those with both superficial and deep components are expected to leave a more prominent and disfiguring scar and redundant skin with altered texture. The HSS recognizes that the worst cosmetic outcomes tend to involve hemangiomas that permanently distort anatomic landmarks, such as cartilage of the nose or ear or shape of the lip, especially when there is loss of the normal concavity of the philtrum. These anatomical units are difficult to recreate if the fibrofatty tissue significantly alters the contour or if the hemangioma has resulted in distortion or destruction of anatomical landmarks. Although intrarater reliability of the HDCS was high, the lowest percentage agreement (67%) occurred for a case involving the complication of cartilage distortion and destruction of the ear. Two other cases involving cartilage destruction had 100% agreement. Because developing the grades for this complication was difficult and controversial among members of the HIG-RC, we added clarification to this section of the manual (Appendix).

Results of the HSS are reported as objective, subjective, and total severity scores because it was hypothesized that the subjective score would have greater variability among raters. Although in testing of the HSS, the objective and subjective portions had similar low variability, the 3 scores differentiate the objective clinical features, including complications, from the subjective physician-rated findings of pain and disfigurement. Further prospective studies are needed to determine whether distinguishing objective from subjective variables is useful or if the separate scores may be clinical predictors. For example, a higher subjective score may be associated with the need for treatment, or a higher objective score may indicate greater likelihood for significant morbidity.

Each complication listed in the HSS is linked to an HDCS grade. The intent of the HDCS is to facilitate documentation of hemangioma-specific complications for longitudinal use during research trials or routine clinical care. The format of the HDCS was modeled on CTCAE (version 4.0), which is used to assign severity grades for toxic effects related to cancer therapy. The high degree of interrater reliability of the HDCS may also reflect the similar level of experience of those participating in testing because most of the investigators have extensive experience in treating vascular anomalies and many of them do so in a multidisciplinary clinic setting. The manual accompanying the scales was created to be detailed and comprehensive to increase interrater reliability. Optimally, the HSS and the HDCS should be used in clinical trials to measure size changes, complications, and disfigurement risks of hemangiomas. It is estimated that the HSS and the HDCS take no longer than 5 minutes to complete.

Quiz Ref IDIn conclusion, given the tremendous heterogeneity of hemangiomas, it is important to standardize measurement of clinical severity and outcomes in clinical practice and in therapeutic trials so that treatment efficacy can be accurately measured. These scales are easy to use, take little time to complete, and can be used across specialties for documenting morbidity, stratifying patients for clinical trial enrollment, and assessing objective outcome measures. The HSS and the HDCS should be invaluable for assessing clinical outcomes and for standardizing hemangioma research.

Correspondence: Anita N. Haggstrom, MD, Department of Dermatology, Indiana University School of Medicine, 550 N University Blvd, Room UH3240, Indianapolis, IN 46202.

Accepted for Publication: September 2, 2011.

Author Contributions: Drs Haggstrom and Chamlin had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Haggstrom, Adams, Drolet, Frieden, Garzon, Holland, Horii, Mancini, Metry, Morel, Newell, Nopper, Siegel, Swigonski, Cella, and Chamlin. Acquisition of data: Haggstrom, Adams, Drolet, Frieden, Garzon, Holland, Horii, Mancini, Metry, Morel, Newell, Nopper, Siegel, and Chamlin. Analysis and interpretation of data: Haggstrom, Beaumont, Lai, and Chamlin. Drafting of the manuscript: Haggstrom, Beaumont, Lai, and Chamlin. Critical revision of the manuscript for important intellectual content: Haggstrom, Beaumont, Lai, Adams, Drolet, Frieden, Garzon, Holland, Horii, Lucky, Mancini, Metry, Morel, Newell, Nopper, Siegel, Swigonski, Cella, and Chamlin. Statistical analysis: Beaumont and Lai. Obtained funding: Haggstrom and Chamlin. Administrative, technical, or material support: Haggstrom and Chamlin. Study supervision: Haggstrom and Chamlin.

Funding/Support: This study was supported by grant 1RC1AR058767 from the National Institutes of Health.

Role of the Sponsors: The sponsors had no role in the design or conduct of the study; in the collection, analysis, or interpretation of data; or in the preparation, review, or approval of the manuscript.

Additional Contributions: Eulalia Baselga, MD, Jane Bellet, MD, Francis Blei, MD, Anna Bruckner, MD, Kelly Cordoro, MD, Mary Wu Chang, MD, Sheila Galbraith, MD, Moise Levy, MD, Sheilagh Maguiness, MD, Elena Pope, MD, Howard Pride, MD, Julie Prendiville, MD,Maureen Rogers, MD, Amy Schmidt, MD, Sarah Stein, MD, Patricia Wittman, MD, and Albert Yan, MD, participated in the development and reliability testing of the Hemangioma Severity Scale and the Hemangioma Dynamic Complication Scale. Treana Porter, MD, and Donita Roettcher, MD, assisted with database design.