The Patient-Oriented Eczema Measure: Development and Initial Validation of a New Tool for Measuring Atopic Eczema Severity From the Patients’ Perspective

Objective  To develop a simple, valid, repeatable, and readily understandable patient-oriented assessment measure for monitoring disease activity in children and adults with atopic eczema.

Design  Qualitative semistructured patient interviews identified a list of symptoms of atopic eczema. These symptoms were quantitatively analyzed in a larger patient population to identify which symptoms were important to patients and amenable to monitoring as part of a scoring system.

Setting  The outpatient Department of Dermatology at the Queen’s Medical Centre, University Hospital, Nottingham, England, and 5 local general practices.

Patients  Four hundred thirty-five patients with atopic eczema.

Results  Seven symptoms were incorporated into the final patient-oriented eczema measure using a simple 5-point scale of frequency of occurrence during the previous week, with a maximum total score of 28. Validity testing against the Dermatology Life Quality Index, Children’s Dermatology Life Quality Index, and patients’ global severity assessments showed good correlation (r = 0.78, r = 0.73, and r = 0.81, respectively; P<.001). Internal consistency was high (Cronbach α = 0.88), and test-retest reliability was good, with 95% of scores falling within 2.6 points on repeat testing (mean score difference, 0.04; SD, 1.32). Individual variables in the measure demonstrated sensitivity to change during a 4-week in-clinic period and an 18-week randomized controlled clinical trial.

Conclusion  The patient-oriented eczema measure is a practical self-assessed measurement tool for monitoring aspects of atopic eczema that are important to patients in routine clinical practice or in the clinical trial setting.

Atopic eczema currently affects around 5% to 20% of children worldwide.¹ The increasing prevalence of the disease has had a profound impact on hospital and community health care services, with significant amounts of resources being directed into research to improve our understanding of disease mechanisms, resulting in the recent development of new treatments such as topical tacrolimus and pimecrolimus. Valid and clinically meaningful outcome measures play a central role in both the interpretation of clinical trial results and the ongoing assessment of health care intervention and are essential to ensure that such research can be translated into maximum patient benefit.

Outcome methodology for atopic eczema has been a relatively neglected and unstandardized area of research, with a bias toward objective measurements deemed to be important by physicians rather than patient-centered assessments.^2-4 More than 56 different objective scoring systems recording different combinations of physical signs were recently identified in published randomized controlled trials (RCTs) of atopic eczema between 1994 and 2001 alone, with many of the measures having no data on validity or reliability.⁴ Even validated objective measures such as the Scoring Atopic Dermatitis (SCORAD) Index,⁵ Eczema Area and Severity Index,⁶ and Six Area, Six Sign Atopic Dermatitis Index⁷ all measure different physical variables and combine them with variable weighting. Interpreting what a change in a score of 12 of 103 actually means in terms of patient morbidity can be extremely difficult. Furthermore, these scoring systems are primarily designed for clinical trial work and are generally too time-consuming for routine use in the outpatient clinic or for monitoring patient responses in general practice.

Recent years have witnessed a growing recognition of the importance of measuring disease severity and the impact of health care from the patient’s perspective.^8,9 Patient-based outcome measures addressing aspects such as patient symptoms, functional ability, satisfaction with care, and quality of life are increasingly being used as primary or secondary end points in clinical trials to supplement more traditional clinical or laboratory measures of disease status. These measures potentially provide a more holistic and relevant evaluation of the impact of health care intervention. Compared with other chronic diseases such as asthma^10,11 and rheumatoid arthritis,¹² where patient-oriented symptom measurements have been successfully developed and incorporated into clinical practice, patient-based assessments in atopic eczema remain underused or neglected. In our recent review of atopic eczema RCTs (93 trials), patient symptoms were assessed using a wide range of scales with no standardized scoring system identified and were restricted to an assessment of itch alone or itch and sleep disturbance in more than three quarters of the trials.⁴ Furthermore, quality of life was assessed in only 3 (3%) of these RCTs, despite the availability of validated dermatology-specific measures.^2,13,14

We performed this study to investigate which atopic eczema symptoms are important to patients, to develop a simple patient-oriented eczema measure (POEM) for research purposes, and to assist health care professionals such as general practitioners, dermatologists, pediatricians, and specialist nurses caring for patients in routine clinical practice.

Patients were recruited from primary and secondary care settings using 5 local general practices and the outpatient Department of Dermatology at the University Hospital in Nottingham, England. All patients satisfied the UK Working Party’s refinement of the diagnostic criteria for atopic eczema of Hanifin and Rajka.¹⁵ Primary care patients were recruited by writing to all patients listed by the general practice with a diagnosis of atopic eczema and arranging local interviews at the practice at a convenient time for patients. Secondary care patients were recruited consecutively from the dermatology outpatient clinic. All primary care patients were being treated with topical therapy only (eg, emollients and topical corticosteroids), whereas 12% of the secondary care patients were receiving additional systemic therapy with cyclosporine, azathioprine, or oral corticosteroids. Ethical permission for the study was obtained from the Local Ethics Committee before commencement of the study.

Thirty-five patients were recruited from primary and secondary care settings for qualitative semistructured interviews by a single interviewer (C.R.C.). Patients were asked open-ended questions about the aspects of their disease that affected them, including “What bothers you about your eczema?” and “What troubles you most about your eczema?” All interviews were audiotaped for transcribing, and all symptoms identified by patients were recorded. For young children, information was obtained from the child and parents, depending on the child’s age and understanding.

Identified symptoms were incorporated into a questionnaire that was administered to 200 patients with atopic eczema at interview (examples and the format of these questions are available from the authors on request). The aims of this questionnaire were (1) to establish the frequency of each symptom (using a 5-point scale), because measuring symptoms that occurred very rarely or every day in most of the patients would provide poor discrimination between individuals in the final outcome measure; (2) to investigate whether some symptoms were more important than others to patients (using a 5-point scale); and (3) to assess the clarity of the questions and identify symptoms that were difficult to understand or assess as part of a questionnaire.

To measure how important each symptom was to patients, we used the phrase “How much bother does the eczema cause?” The term bother has been used successfully in the development of a symptom-based outcome measure for asthma¹¹ and was easily understood by all patients when piloted before the study. An alternative phrase such as “How much trouble does the eczema cause?” may have provided similar information, but such terms have not, to our knowledge, been used previously in published outcome-measure studies. A 1-month recall period was used to capture relatively infrequent but important symptoms. Patients were given the opportunity to add other symptoms that they felt were important.

We performed statistical analysis using SPSS for Windows software version 11.0 (SPSS Inc, Chicago, Ill). We computed and tabulated the percentage of respondents falling into each frequency and morbidity category for each symptom. We used the Spearman rank correlation coefficient to assess the association between frequency and morbidity for each symptom.

Those symptoms confirmed to be important and comprehensible in the above sample were then incorporated into a POEM allowing patients to record how frequently they experienced each symptom on a 5-point scale. Although a 1-month recall period had been used to obtain preliminary information on the frequency of symptoms, a 1-week recall period was considered more appropriate for the final outcome measure, to monitor short-term changes in clinical trials and daily practice.

The POEM was given to an additional 200 patients recruited from the outpatient clinic. Patients and parents were also asked to complete a validated dermatology-specific quality-of-life measure, the Dermatology Life Quality Index¹³ for patients older than 16 years or the Children’s Dermatology Life Quality Index¹⁴ for patients aged 4 to 16 years. In addition, all patients were asked to assess how much overall bother their eczema had caused them during the 1-week period on a scale of 0 to 10 and to rate the global severity of their eczema on a 5-point scale (clear, mild, moderate, severe, or very severe). The frequency distribution of scores was computed and tabulated.

We used the Cronbach α to assess internal consistency and the Spearman rank correlation coefficient to assess the correlation between the new patient-oriented measure, quality of life, and global disease severity assessments. Repeatability was tested by asking 50 additional patients from the outpatient clinic to complete the new patient-oriented measure twice, 24 to 48 hours apart. To minimize recall bias, patients were not informed that they would need to repeat the questionnaire. Test-retest repeatability was analyzed using the method suggested by Bland and Altman.¹⁶ Sensitivity to change was assessed in 40 new dermatology outpatients who completed the scoring system in the clinic and were telephoned at 1 and 4 weeks to complete another questionnaire. During development of the POEM, sensitivity of 6 of the variables was also assessed in an RCT of topical corticosteroid use in 207 children (aged 1-15 years) with atopic eczema.¹⁷

Of the 35 patients interviewed, 24 were recruited from secondary care and 11 from primary care settings, with eczema severities ranging from mild to very severe (global assessment by a dermatologist [C.R.C.]). Patient ages ranged from 1 to 58 years (median age, 17 years); 17 (49%) were male. Ethnicity included 31 white patients (89%) and 4 (11%) of Asian or Afro-Caribbean origin from a range of socioeconomic backgrounds. During the interview, the following 10 symptoms were identified by patients:

• Itching

• Soreness or pain

• Sleep disturbance

• Redness of the skin

• Bleeding

• Weeping/oozing of the skin

• Dryness/roughness of the skin

• Flaking of the skin

• Cracking of the skin

• Tightness of the skin

The questionnaire was completed by 110 patients from secondary care and 90 patients from primary care (age range, 1-58 years; median age, 10 years). This represented a recruitment rate of 98% from the dermatology outpatient clinic and 36% from the primary care setting. The low recruitment rate from the primary care setting predominantly resulted from failure to respond to the initial contact letter (68%), although other reasons included resolution of the eczema, failure to fulfill the diagnostic criteria for atopic eczema, lack of a mutually convenient time for interview, and failure to attend an arranged interview. No additional symptoms were identified by patients in this part of the study.

The frequency of symptoms during the 1-month period is shown in Table 1. Dryness, redness, itching, and soreness were the most commonly occurring symptoms in responding patients from the hospital and community settings. No symptoms were predominantly endorsed in 1 response category only, and therefore each could potentially provide useful information in the final outcome measure.

The morbidity (bother) caused by each symptom was strongly related to how frequently patients experienced the symptom. For example, sleep disturbance did not bother patients whose nights were only occasionally disturbed as much as those whose sleep was disturbed regularly. This was demonstrated by a strong significant correlation between frequency and symptom morbidity, with highest correlation for weeping (r = 0.88; P<.001), sleep disturbance (r = 0.70; P<.001), bleeding (r = 0.70; P<.001), flaking (r = 0.70; P<.001), and itching (r = 0.60; P<.001) and lowest for dryness (r = 0.51; P<.001), cracking (r = 0.51; P<.001), and redness (r = 0.45; P<.001). Therefore the amount of bother caused by each symptom was assessed only in patients experiencing the symptom on most days or every day of the month (Table 2). Analyzing hospital and community patients separately confirmed that all symptoms were important to the patients, with more than 65% of the patients who experienced each symptom regularly complaining of moderate bother or more.

Although 200 patients completed a questionnaire at the interview, questions concerning tightness and redness were completed in fewer than 75% of cases because of difficulties understanding or assessing these symptoms. Redness was found to be particularly difficult to assess for patients with Afro-Caribbean or Asian skin type. These 2 symptoms were therefore excluded from the final outcome measure. Assessment of soreness in young children was also difficult, and parents of 35 children (all aged <4 years) felt unable to assess this symptom accurately on their child’s behalf. Retention of soreness in the outcome measure would have necessitated a separate scoring system excluding soreness for children younger than 4 years. As we believed that this would reduce the measure’s simplicity and acceptability, this symptom was also excluded. Questions concerning the remaining symptoms were completed by all patients. The retained 7 symptoms were incorporated into a simple POEM that allowed patients to record how frequently they experienced each symptom during a 1-week period, with a maximum score of 28 (Figure 1). Weighting of individual symptoms was not used to keep the measure simple and practical.

An additional 200 patients (aged 12 months to 69 years; median age, 9 years) completed the POEM. Scores for the 7 symptoms are shown in Table 3, with dryness and itching being the most common symptoms and bleeding and weeping the most infrequent. The scores showed high homogeneity or internal consistency (Cronbach α = 0.88), confirming that they were measuring aspects of the same disease. A Cronbach α of 0.7 to 0.9 is thought to be ideal, with higher scores suggesting that items correlate so well that some are redundant and lower scores suggesting that items are not measuring the same trait.¹⁸

Content validity of the new scoring system was demonstrated by the fact that the measured domains were derived from patients themselves, with no further symptoms identified among the 200 patients questioned. Construct validity measures agreement with other related variables, in this case patient assessments of disease-related quality of life. Correlation showed reasonably good agreement between the new measure and the Dermatology Life Quality Index (n = 82; r = 0.78; P<.001) and Children’s Dermatology Life Quality Index (n = 68; r = 0.73; P<.001). Criterion validity ideally involves correlation against a gold standard measure. As there is no accepted gold standard patient-based measure of eczema severity, criterion validity was measured against patient global assessments of disease severity (5-point scale) and overall bother related to the eczema (10-point scale) during the 1-week period and demonstrated high correlation with both assessments (n = 200; r = 0.81 and r = 0.84, respectively; P<.001).

Fifty patients (age range, 12 months to 62 years; median age, 8.5 years) completed the new measure twice. The Bland and Altman plot is shown in Figure 2. The mean of the differences between scores was 0.04 (SD = 1.32). Total scores were the same before and after in 33 (66%) of the 50 patients, with identical individual symptom scores in 29 (58%). Overall, 46 patients’ scores (92%) fell within 2 points and 49 (98%) within 3 points on repeat testing, confirming acceptable repeatability.

The new measure was completed by 40 new outpatients (age range, 1-36 years; median age, 4 years) at 0, 1, and 4 weeks, as shown in Figure 3. All 7 symptoms showed a mean decrease during the 4-week period, although dryness was slower to improve than other variables such as itch. All variables tested during the 18-week topical corticosteroid RCT showed an improvement (Figure 4), with itching and dryness falling by more than a third. Scores for other variables also decreased but were generally low at baseline, reflecting the mild-to-moderate disease spectrum of patients in the trial.

The POEM is a simple, valid, easily interpreted, and reproducible tool for assessing atopic eczema and monitoring aspects of the disease that are important to patients. The measure captures the fluctuating and chronic nature of atopic eczema and provides a more comprehensive assessment of patient symptoms than that obtained by measuring itch and/or sleep disturbance alone, the 2 most commonly measured symptoms of the disease.^2,4 The total score (maximum, 28) has been shown to accurately reflect eczema-related morbidity, whereas analysis of individual variables can provide useful information on whether acute (eg, weeping, bleeding) or chronic (eg, itching, dryness) changes are predominant and can help target therapy accordingly.

The POEM is based on patients’ views of what constitutes disease severity rather than what physicians presume to be important to patients. One could argue that designing a scale containing a number of different variables is a roundabout way of assessing disease severity. Why not just ask patients for a global assessment of their disease severity? Direct global assessments (eg, mild, moderate, and severe or visual analog scales) can provide a useful overview of disease severity, but a single question may be more subject to individual external influences than a scale containing a number of items that require the patients to think about their disease in more detail. Global scales can also lack the sensitivity to detect small but clinically important changes in severity, and they do not provide information on how individual aspects of the disease change with treatment. By measuring symptom frequency rather than overall severity, the POEM can provide useful information about the pattern of relapses and remissions, the periodicity of which is more rapid than in other inflammatory skin diseases such as psoriasis and can be difficult to capture by intermittent clinical examination. Patient diaries for daily recording of symptoms could be used to further supplement the POEM, but were not used in the development of the outcome measure.

The POEM can be completed by most patients in 1 to 2 minutes and has proved a useful tool for routine clinical practice in our dermatology department, being quick for patients to complete in the waiting room before consultation or for monitoring at home on a regular basis to provide a clearer picture of long-term disease activity. In the clinical trial setting, the POEM has proved to be practical to administer, and it is hoped that it will provide a standardized patient-based symptom score for use as a primary outcome measure or to supplement objective scoring systems and dermatology quality-of-life measurements. As the POEM does not involve clinical examination by a dermatologist, it could have significant advantages in long-term trials where information about the variation of individual disease variables is required for several months. The new measure has been specifically developed using patients from both primary and secondary care to ensure that it is applicable to use in community and hospital settings. In the community, the POEM could potentially be used to audit and monitor patient care by primary care physicians or specialist nurses or to supplement referral guidelines. As clinical examination is not required, the new measure provides a potentially useful tool for epidemiological studies, to guide allocation of health care service resources and improve our understanding of the environmental factors contributing to disease development.

One of the limitations of this study is the relatively low variable scores of patients recruited for sensitivity testing in the RCT of topical corticosteroid use.¹⁷ In this trial, 84% of patients were recruited from a primary care setting, with overall objective clinical scores showing a mild disease spectrum (Six Area, Six Sign Atopic Dermatitis Index at baseline, 9.6/108; at week 18, 6.2/108). Although bleeding was not assessed during this trial, this variable showed sensitivity to change in the outpatient clinic setting. The recruitment rate from general practice was low, although it is anticipated that patients failing to reply were more likely to have had mild or cleared disease, and that such an exclusion would not have significantly altered our results.

The POEM is a useful new tool for monitoring the care of patients with atopic eczema. Further studies are needed to confirm its usefulness in the clinical trial setting, particularly in patients with moderate-to-severe disease, and extension of the POEM’s use across different cultural backgrounds will require ongoing validation. Objective scoring systems such as the SCORAD Index and Eczema Area and Severity Index continue to have an important role in monitoring disease pathology, but focus on the disease process rather than the illness experienced by the patient and exclude symptoms (Eczema Area and Severity Index and Six Area, Six Sign Atopic Dermatitis Index) or combine sign and symptom scores (SCORAD), making interpretation from the patients’ perspective difficult. Inclusion of a patient-based symptom measure provides a more holistic evaluation, and it is hoped that incorporation of the POEM into clinical practice will enable patients to be more actively involved in their disease management and significantly improve our interpretation of patient benefit from health care intervention.

Correspondence: Carolyn R. Charman, BM, BCh, MRCP, Department of Dermatology, Queen’s Medical Centre, Nottingham NG7 2UH, England (carolyn.charman@nottingham.ac.uk).

Accepted for Publication: July 30, 2004.

Funding/Support: This work was supported by a Health Services Research Training Fellowship funded by the former National Health Service Research and Development Board, Trent Region, Sheffield, England.

Acknowledgment: We thank the patients and outpatient reception staff who helped with this study; the Trent Focus Group and general practice surgeries for their administrative assistance; and Kim Thomas, PhD, and Sandra Lawton, MSc, for their help with validation of the scoring system.

Financial Disclosure: None.

A cooperative effort of the Clinical Epidemiology Unit of the Istituto Dermopatico dell'Immacolata Istituto di Ricovero e Cura a Carattere Scientifico (IDI-IRCCS) and the Archives of Dermatology