Figure 1. Study procedures. PASI indicates Psoriasis Area and Severity Index.
Figure 2. Psoriasis Area and Severity Index (PASI) mean (SE) score changes before and after online video training.
Armstrong AW, Parsi K, Schupp CW, Mease PJ, Duffin KC. Standardizing Training for Psoriasis MeasuresEffectiveness of an Online Training Video on Psoriasis Area and Severity Index Assessment by Physician and Patient Raters. JAMA Dermatol. 2013;149(5):577-582. doi:10.1001/jamadermatol.2013.1083
Author Affiliations: Department of Dermatology, University of California, Davis, Sacramento (Drs Armstrong, Parsi, and Schupp); Division of Rheumatology, Department of Medicine, University of Washington, Seattle (Dr Mease); and Department of Dermatology, University of Utah, Salt Lake City (Dr Duffin).
Importance Because the Psoriasis Area and Severity Index (PASI) is the most commonly used and validated disease severity measure for clinical trials, it is imperative to standardize training to ensure reliability in PASI scoring for accurate assessment of disease severity.
Objective To evaluate whether an online PASI training video improves scoring accuracy among patients with psoriasis and physicians on first exposure to PASI.
Design This equivalency study compared PASI assessment performed by patients and PASI-naive physicians with that of PASI-experienced physicians at baseline and after standardized video training. The study was conducted from March 15, 2011, to September 1, 2011.
Setting Outpatient psoriasis clinic at University of California, Davis.
Participants Forty-two psoriasis patients and 14 PASI-naive physicians participated in the study. The scores from 12 dermatologists experienced in PASI evaluation were used as the criterion standard against which other scores were compared.
Main Outcome Measures Aggregate and component PASI scores from image sets corresponding to mild, moderate, and severe psoriasis.
Results After viewing the training video, PASI-naive physicians produced equivalent scores for all components of PASI; patients provided equivalent scores for most PASI components, with the exception of area scores for moderate-to-severe psoriasis images. After the online video training, the PASI-naive physicians and patients exhibited improved accuracy in assigning total PASI scores for mild (Meanexperienced physician − MeanPASI-naive physician: 1.2; Meanexperienced physician − Meanpatient: −2.1), moderate (Meanexperienced physician − MeanPASI-naive physician:0; Meanexperienced physician − Meanpatient: −5.7), and severe (Meanexperienced physician − MeanPASI-naive physician: −5.1; Meanexperienced physician − Meanpatient: −10.4) psoriasis, respectively.
Conclusions and Relevance Use of an online PASI training video represents an effective tool in improving accuracy in PASI scoring by both health care professionals and patients. The video-based online platform for disseminating standardized training on the use of validated instruments in dermatology represents a novel form of standardized education.
Psoriasis is a common, chronic inflammatory skin disease that affects 2% to 4% of the adult population worldwide.1,2 Several instruments have been developed to measure disease severity in psoriasis.3- 5 Among them, the Psoriasis Area and Severity Index (PASI) is the most frequently used instrument to measure disease severity in clinical trials.6,7 The PASI is determined by assessing the severity of erythema, induration, scale, and area of involvement; entering the severity scores into a formula results in a score from 0 to 72. The PASI not only quantifies the extent of the disease but also is considered a validated measure of response to treatment.8
Although the PASI is the most frequently used instrument to measure psoriasis severity in clinical trials, it has many limitations. First, the PASI is a nonnormally distributed scale. Approximately 80% of patients with psoriasis have less than 10% body surface area (BSA) involvement; therefore, the upper end of the PASI is used infrequently.9 Second, the PASI does not discriminate well among patients with low BSA.10 Third, although the PASI has been the criterion standard for assessing response to new medications, it is rarely used routinely in clinical practice because of the complexity in calculating the score and its inherent lack of intuitive meaning compared with measures such as BSA.
Without training, evaluators assign PASI scores with substantial variability.11 Because the PASI is a subjective measure based on visual assessment of the patients' skin involvement, untrained evaluators tend to overestimate the area involved.12 Therefore, standardized training is important to ensure uniformity in PASI scoring.
Traditionally, PASI training for health care professionals occurs before initiation of clinical trials in varied ways. In some instances, experienced investigators conduct in-person training sessions for a group of PASI-naive physicians, research staff, and other health care providers; this process usually is not reproducible and not standardized. Furthermore, such training sessions are costly, and the knowledge is not readily shared beyond those who attend. As video-based education becomes more prevalent in dermatology, the use of video-based training modules for the PASI can be valuable in training a large group of evaluators and ensure uniformity in training standards.
Psoriasis patients can score their disease severity with the use of the Self-administered Psoriasis Area and Severity Index, which is the same PASI instrument used by physicians.13,14 If the patients are able to score the PASI more accurately after video training, new opportunities exist in monitoring disease severity using patients' scores and carrying out data collection in patients' homes. The primary aim of this study was to evaluate whether an Internet-based PASI training video improves scoring accuracy among psoriasis patients and physicians new to the instrument.
This study assessed the effect of an online training video on the accuracy of PASI scoring in psoriasis patients and physicians new to the instrument using experienced raters' scores as the criterion standard. The study was approved by the institutional review board at the University of California, Davis (UC Davis).
The patients were recruited from the psoriasis clinic at the Department of Dermatology at UC Davis between March 15, 2011, and September 1, 2011. These patients were aged 18 years or older, English-speaking, had psoriasis diagnosed by a dermatologist, had access to the Internet, and could demonstrate their ability to view videos online. The PASI-naive physicians were dermatology residents and board-certified dermatologists not trained on PASI scoring before the study. The scores of the psoriasis patients and PASI-naive physicians were compared with the criterion standard scores of the PASI-experienced dermatologists. These experienced raters (including A.W.A. and K.C.D.) were members of the Group for Research and Assessment of Psoriasis and Psoriatic Arthritis and had participated in the development of the online module.
The study flow is depicted in Figure 1. At the beginning of the study, the patients and PASI-naive physicians were asked to provide baseline PASI scores for 3 sets of images, with each set representing 1 of the following severity states: mild, moderate, and severe. Approximately 10 days after completion of the baseline image scoring, the patients and PASI-naive physicians completed the online training video. They then rated the PASI image sets again, and the scores were compared with those of the PASI-experienced physicians.
The online PASI training video was created as a part of the 2010 Psoriasis and Psoriatic Arthritis Video Project by the Group for Research and Assessment of Psoriasis and Psoriatic Arthritis. The main objective of this project was to provide a set of standardized training modules for use in clinical studies.15 The intended use of the training videos included study-site training for new study personnel, reinforcement of training completed at investigator meetings, and certification of evaluator proficiency. As part of the development of the video, 12 dermatologists who are experienced PASI users scored the patients' photographs. These scores were used as the criterion standards in this study.
The video was produced professionally (KitDigital; http:// www.kitdigital.com) and was launched using an Internet-based media platform to provide access for interested users (http://www.grappanetwork.org/).15 The PASI training module consisted of an introduction to the measure, a demonstration of the presenter examining patients, and supplemental diagrams and photographs to reinforce teaching points.
For each rater group, the mean score for aggregate PASI as well as individual components (erythema, induration, scale, and area) were calculated from all 3 psoriasis severity states (mild, moderate, and severe). We compared the mean aggregate PASI scores as well as individual component scores from before and after the video training among the 3 rater groups—patients, PASI-naive physicians, and PASI-experienced physicians. The equivalence margin for individual PASI components of erythema, induration, scale, and area was defined as +1 or –1 from the mean score of the PASI-experienced physicians. Traditional correlation analysis requires a small number of raters with a large number of independent observations.16 In the context of this study design, in which there were many raters and few standardized image sets, the equivalency testing was deemed most appropriate.17 All statistical analyses were performed using commercial software (SAS, version 9.3; SAS Institute, Inc).
The 68 individuals who participated in this study were 42 psoriasis patients, 14 PASI-naive physicians, and 12 PASI-experienced physicians. The demographic information of the patients is summarized in Table 1.
Scoring for each component of the PASI score—erythema, induration, scale, and area—was examined separately (Table 2). With the scores from the PASI-experienced physicians serving as the criterion standard, the results of the equivalency testing for each component of the PASI score between the (1) patient raters and PASI-experienced physicians and the (2) PASI-naive physicians and the PASI-experienced physicians were presented. With an equivalency margin of 1 for each of the components of the score, the PASI-naive physicians produced equivalent scores for all components of the PASI after the training video. The patients provided equivalent scores for the components with the exception of the area scores for moderate and severe psoriasis images.
Overall, after the online video training, the PASI-naive physicians and patients exhibited improved accuracy in assigning total PASI scores for mild (Meanexperienced physician − MeanPASI-naive physician: 1.2; Meanexperienced physician − Meanpatient: −2.1), moderate (Meanexperienced physician − MeanPASI-naive physician: 0; Meanexperienced physician − Meanpatient: −5.7), and severe (Meanexperienced physician − MeanPASI-naive physician: −5.1; Meanexperienced physician − Meanpatient: −10.4) psoriasis, respectively.
Mean PASI comparison between the 3 groups before and after the video training modules for the images with varying psoriasis severity is shown in Figure 2. For images depicting mild psoriasis, the PASI-experienced physicians assigned a mean (SE) score of 5.6 (0.5). At baseline prior to the video training, the patients assigned a mean score of 11.5 (0.9). After the video training, the patients' mean score decreased by 3.8 points (33%), which approximated the mean of the PASI-experienced physicians. At baseline, the PASI-naive physicians assigned a score of 7.5 (1.0); after online video training, these physicians' mean score decreased by 3.1 points (41%).
For images depicting moderate psoriasis, the PASI-experienced physicians assigned a mean score of 10.4 (0.9). Before the video training, the patients assigned a mean score of 27.9 (1.9). After the video training, the patients' mean score decreased by 11.8 points (40%). At baseline, the PASI-naive physicians assigned a score of 17.2 (1.6); after online video training, these physicians' mean score decreased by 6.8 points (40%), which corresponded closely to the mean score of the PASI-experienced physicians.
When evaluating images depicting severe psoriasis, the PASI-experienced physicians assigned a mean score of 19.9 (0.8). Before the video training, the patients assigned a mean score of 38.0 (1.5). After the video training, the patients' mean score decreased by 7.7 points (20%). At baseline, the PASI-naive physicians assigned a score of 28.7 (1.7); after online video training, these physicians' mean score decreased by 3.7 points (13%).
The PASI is considered the most valid and most frequently used psoriasis severity assessment tool. Its use in clinical trials has enabled accurate assessment of psoriasis severity states as well as responses to treatments. However, without adequate training, raters can assign erroneous scores to individual components of the PASI, thereby leading to wide variations in scores resulting from poor intrarater and interrater reliability.11
Whereas traditional PASI training has relied on costly on-site training, online standardized training for medical education is ushering in novel training formats for teaching raters to use disease severity measures. There are several advantages of online video-based standardized training. First, the information is readily accessible and instantly disseminated, without costly in-person sessions. Second, the users can digest the training information at times and locations convenient to them, and they can pause and review any part of the training material. Last, online video-based training ensures standardization of training, such that all trainees receive consistent information.
In this study, after viewing the online video training, the psoriasis patients achieved equivalency in scoring all dimensions of PASI with the exception of the area component in moderate and severe psoriasis. Although patients did not provide area scores within the predefined equivalency margins, they were able to correct the initially overestimated area scores to more accurate estimates of the BSA involved. After the online video training, patients exhibited improved accuracy in assigning total PASI scores for all 3 psoriasis severity states compared with the PASI-experienced physicians.
Published documentation of the effectiveness of in-person PASI training sessions is lacking; reports on the effectiveness of online, video-based standardized training are necessary for ensuring training quality for future users. In this study, after the online video training, the PASI-naive physician raters were able to achieve equivalency in all individual components of the PASI. Furthermore, their mean total PASI scores for mild, moderate, and severe psoriasis improved in accuracy and were closer to those of the PASI-experienced physicians after the online video training.
One novel aspect of this study was the application of a training video originally designed for health care professionals to use in a patient population for self-assessment of disease severity. Recent studies18- 21 have shown that video-based education is effective in increasing patient knowledge and improving clinical outcomes in certain situations. For example, after viewing educational videos, patients with atopic dermatitis have increased knowledge regarding eczema care as well as reduced disease severity.19
Psoriasis patients are an ideal population to assess their disease severity, particularly as this applies to patient-centered care. As we continually define treatment goals for psoriasis and psoriatic arthritis, renewed emphasis has been placed on the treating-to-target concept and keeping disease activity suppressed rather than allowing psoriasis to flare before initiating treatment. Thus, the usefulness of self-monitoring for routine treatment of psoriasis and psoriatic arthritis is high. In addition, literature22,23 on rheumatoid arthritis suggests that, with simple training, most patients are able to accurately assess joint swelling, and their self-administered joint tenderness counts are reliable and responsive to changes in disease activity.
The study findings need to be interpreted in the context of the design and limitations. This study involved grading of photographs to evaluate assessors' abilities; future studies need to focus on grading of multiple patients with varying psoriasis severity in live settings. Even with video education, patients appeared to overestimate BSA involvement; the implications of this overestimation on disease perception and treatment adherence are unknown. Further improvements in video education specific for patients are necessary to improve accuracy of the estimation. Future studies also need to assess whether the videos are effective in improving patients' ability to grade the psoriasis severity. Most patients in this study were overall more educated and motivated than the general population. Creation of videos targeted toward patients of varying literacy levels will likely be more effective in improving patient-assessed PASI. Furthermore, larger sample sizes of PASI-naive physicians and PASI-experienced physicians would be useful to understand potential underlying factors that predict degree of improvement in PASI scoring after video training.
In conclusion, the findings of this study suggest that the online PASI training video is effective in improving accuracy in scoring by health care professionals new to PASI scoring as well as patients with psoriasis. The video-based online platform for disseminating standardized training on the use of validated instruments in dermatology represents a novel form of standardized education for researchers, clinicians, and patients.
Correspondence: April W. Armstrong, MD, MPH, Department of Dermatology, University of California, Davis, 3301 C St, Ste 1400, Sacramento, CA 95816 (firstname.lastname@example.org).
Accepted for Publication: August 30, 2012.
Published Online: February 20, 2013. doi:10.1001/jamadermatol.2013.1083
Author Contributions: All authors had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Armstrong, Parsi, and Duffin. Acquisition of data: Armstrong, Parsi, and Duffin. Analysis and interpretation of data: Armstrong, Schupp, Mease, and Duffin. Drafting of the manuscript: Armstrong, Schupp, Mease, and Duffin. Critical revision of the manuscript for important intellectual content: Armstrong, Parsi, Mease, and Duffin. Statistical analysis: Armstrong and Schupp. Obtained funding: Armstrong. Administrative, technical, and material support: Armstrong and Parsi. Study supervision: Armstrong, Mease, and Duffin.
Conflict of Interest Disclosures: Dr Armstrong is an investigator and consultant to Abbott, Amgen, and Janssen.
Funding/Support: The video production was supported by the Group for Research and Assessment of Psoriasis and Psoriatic Arthritis.
Additional Contributions: Julie Wu, BS, and Bhupinder Badwal, BS, from UC Davis assisted in the formatting and submission process of the article. Mss Wu and Badwal did not receive any compensation.