Comparison of Posttraumatic Stress Disorder Checklist Instruments From Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition vs Fifth Edition in a Large Cohort of US Military Service Members and Veterans

Key Points Question How well can posttraumatic stress disorder (PTSD) be assessed and compared spanning the transition between the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) and DSM-5, using different PTSD Checklist (PCL) instruments? Findings In this diagnostic study of 1921 individuals, there was substantial to excellent agreement when individual items, probable PTSD, and sum scores were compared between PCL-Civilian (PCL-C) version and PCL for DSM-5 (PCL-5); the 2 instruments had nearly identical associations with comorbid conditions. Meaning These results provide support for the transition from the PCL-C to the PCL-5 without losing the ability to monitor trends and associated comorbidities.


The American Psychiatric Association's Diagnostic and Statistical Manual of Mental Disorders, Fifth
Edition , released in 2013, provided updated criteria for mental disorders. 1 Compared with the previous version, DSM-IV-TR, 2 the DSM-5 made a number of notable revisions to the posttraumatic stress disorder (PTSD) diagnostic criteria, including adding new symptoms, modifying existing symptoms, and dividing the avoidance cluster into avoidance and negative alterations in cognitions and mood. These changes created challenges for longitudinal research and in medical settings, where it is important to maintain consistency in assessment of PTSD and retain the ability to monitor changes over time.
Although the Clinician-Administered PTSD Scale is considered the criterion standard for diagnosing PTSD, 3 4 This 20-item instrument has demonstrated excellent psychometric properties. [5][6][7] However, there are notable differences between instruments. The PCL-5 includes 3 additional items assessing the novel DSM-5 symptoms (persistent trauma-related negative emotions, persistent blame, and reckless or self-destructive behavior). 4,8 Of the remaining 17 items, the wording of 13 was modified to align with the DSM-5 changes that attempted to clarify symptom expression (11 slightly to moderately modified and 2 heavily modified).
Owing to these differences, the National Center for PTSD stated that "PCL-5 scores are not compatible with PCL for DSM-IV scores and cannot be used interchangeably," 4 and it has been recommended to administer 2 separate instruments when there is need to assess both sets of DSM PTSD criteria. However, this approach is not practical and doubles the burden on individuals. The redundancy of items could lead to frustration, problems with repeated testing, and order effects. 9 Thus, it is imperative to determine how to assess PTSD based on DSM-IV and DSM-5 criteria with either instrument.
Previous comparisons of the DSM-IV and DSM-5 PCL instruments among soldiers, veterans, and trauma survivors have shown substantial overlap between the measures. [9][10][11][12] The estimated prevalence of PTSD was found to be similar, and the 2 definitions had nearly identical associations with other psychiatric disorders and functional impairment. Direct comparison of individual items indicated substantial agreement (κ > 0.60) for 16 of the 17 corresponding items among soldiers, 9 although agreement was lower among trauma survivors (κ = 0.42-0.58). 12 Among the trauma survivors, a 20-item blended version of the PCL demonstrated excellent internal consistency and correlation with DSM-IV and DSM-5 PCL instruments. 12 Although these studies provide promising evidence that the 2 instruments have similar psychometric properties, questions remain about the ability to make meaningful clinical comparisons across time for military, veteran, and civilian populations. In 1 study, 9 for example, approximately 30% of soldiers who met criteria using 1 measure did not meet criteria using the other measure, despite highly comparable psychometric properties. Furthermore, both versions of the instrument are still used in research and clinical settings. Within the US Department of Defense, the PCL-5 has become the standard for assessing symptoms in mental health clinics; however, the DSM-IV PCL versions are still routinely used for screening purposes for deployment and annual periodic health assessments. 13 Although PTSD screening is becoming routine in the US Department of Defense, Veterans Health Administration, and other health care settings, the marked changes in definition and screening instruments over time pose challenges to the evidence foundation for these public health practices. Most evidence-based treatments for PTSD have been validated with older definitions. Therefore, it is crucial that researchers, clinicians, and medical professionals have clear guidance on transitioning between definitions and interpreting data over time that may have involved both instruments.
Using data from a subset of participants from the Millennium Cohort Study who were administered both the PCL-C and PCL-5, the study addressed 5 objectives: (1) determine agreement between individual items on each instrument, (2) assess agreement of probable PTSD and PTSD sum scores using different instruments, (3) assess whether appending the 2 heavily modified PCL-C items to the PCL-5 significantly changed the percentage of participants with probable PTSD, (4) compare the association of PTSD with comorbidities by using both instruments, and (5) determine the agreement between estimated and observed PCL-5 sum scores by using an established crosswalk. 11 This study was designed to test the hypothesis that there is substantial agreement between the PCL-C and PCL-5 and that DSM-IV and DSM-5 PTSD can be assessed with either instrument. Results from this study will be invaluable in making comparisons across studies and longitudinal efforts and of particular relevance for assessing PTSD among military and veteran populations.

Population and Data Sources
Launched in 2001, the Millennium Cohort Study was designed to examine the long-term health effects associated with military service. 14,15 Between July 1, 2001, and April 4, 2013, 4 separate cohorts of service members, referred to as panels, were randomly selected from military rosters to participate in the study. Participants were selected from the Army, Navy, Air Force, Marine Corps, and Coast Guard (including active duty, National Guard, and reserve personnel). Enrolled participants (approximately 200 000) were requested to complete a web-based or paper survey at baseline and then every 3 to 5 years during and after leaving military service. A detailed description of the study methods has been published elsewhere. 14,15 Participants provided voluntary, informed consent. The study was approved by the Naval Health Research Center institutional review board. This study followed the Reporting of studies Conducted Using Observational Routinely Collected Data (RECORD) reporting guideline.
The 2019-2021 follow-up survey transitioned from the PCL-C to the PCL-5. To examine the agreement between these measures, a sample of initial web responders was randomly assigned to 1 of 4 survey groups (labeled A1, A2, B1, and B2) that received both the PCL-C and PCL-5 within the overall questionnaire (separated by 21 web pages or more, depending on skip patterns). Two survey groups (A1 and A2) received the PCL-C and PCL-5, whereas the other 2 groups (B1 and B2) received the PCL-C and the PCL-5 with 2 additional PCL-C items, referred to as the expanded PCL-5. The 2 additional items ("Feeling as if your future will somehow be cut short" and "Feeling emotionally numb, or being unable to have loving feelings for those close to you") were the items most significantly modified. Survey groups A1 and B1 received the PCL-5 first and the PCL-C as the second instrument, whereas this order was reversed for groups A2 and B2. This counterbalancing of the 2 PCL instruments was designed to control for order effects; the surveys were otherwise identical ( Table 1).
Once survey completions for each group passed 500, random assignment was halted, which occurred within a week after the survey cycle opened. Participants who completed their questionnaire by October 30, 2019, (when data for this project were accessed) were included in the study (n = 2060). After exclusion of participants with missing PCL-C or PCL-5 items (n = 139), the study population consisted of 1921 individuals (Table 1).

Measures
Instructions for the 2 PCL measures were standardized to ensure comparability. Likewise, response options were identical for both instruments, ranging from not at all to extremely, although scoring was different (PCL-C items scored 1 to 5, range 17-85; PCL-5 items scored 0 to 4, range 0-80).
According to DSM-IV criteria, participants screened positive for probable PTSD if they reported moderate or higher on at least 1 intrusive reexperiencing symptom, 3 avoidance symptoms, and 2 hyperarousal symptoms. 2 In accordance with DSM-5 criteria, participants screened positive for probable PTSD if they reported moderate or higher on at least 1 intrusion symptom (previously reexperiencing), 1 avoidance symptom, 2 negative alterations in cognitions or mood, and 2 hyperarousal symptoms. 1

Major depressive disorder was measured with the 8-item module from the Patient Health
Questionnaire, 16 and generalized anxiety with the 2-item Generalized Anxiety Disorder scale. 17 The alcohol scale of the Patient Health Questionnaire was used to assess problem drinking (at least 1 of 5 items endorsed). 16 Demographic and military characteristics (ie, age, sex, race/ethnicity, marital status, education, service branch, pay grade, enrollment panel, and military service status) were assessed via survey and electronic personnel files managed by the Defense Manpower Data Center.

Statistical Analysis
Descriptive analyses, including frequencies and χ 2 tests, were used to compare characteristics between survey groups. Order effects, based on percentage of participants with probable PTSD, were assessed with generalized linear models with inference based on generalized estimating equations to improve inference validity, 18 adjusting for instrument type (PCL-C or PCL-5) and an interaction between order and instrument type.
For determining agreement between individual items on each instrument and assessing agreement of probable PTSD with DSM-IV and DSM-5 criteria on different instruments, Cohen simple κ (for binary data) and weighted κ using Cicchetti-Allison weights (for categorical data) were calculated. Intraclass correlation coefficients (ICCs) were calculated to examine the agreement of PTSD sum scores. 19 Because the PCL-C is missing 3 items present on the PCL-5, imputation was used to calculate the PCL-5 sum score using PCL-C items. The effectiveness of mean imputation and multiple imputation to recover this missing information was compared. For assessing whether appending the 2 heavily modified PCL-C items to the PCL-5 significantly changed the percentage or sum score of PTSD, χ 2 and t tests were used to compare probable PTSD and PTSD sum scores, respectively, using the 20 PCL-5 items based on DSM-5 criteria between survey groups A (PCL-5) and B (expanded PCL-5). For comparing the association of PTSD with comorbidities by using both instruments, the prevalence of comorbidities was calculated, and regression coefficients of associations between comorbidities with DSM-IV and DSM-5 criteria were compared. For determining the agreement between estimated and observed PCL-5 sum scores using an established crosswalk, ICCs were calculated. 11 All analyses were performed with SAS version 9.4 (SAS Institute). Statistical significance was set at P < .05, and all tests were 2-tailed.  Table 2). For all survey groups, prevalence of probable PTSD was significantly higher with the first PCL instrument than the second (Table 1)   In examining the association of comorbidities with PTSD, the proportions with comorbidities were highly comparable, with no statistical differences between the DSM-IV and DSM-5 instruments and criteria (  b Survey group was based on which PCL instruments were received and order of the instruments. Survey groups were as follows: A1, PCL-5 followed by PCL-C; A2, PCL-C followed by PCL-5; B1, PCL-5 plus 2 PCL-C items followed by PCL-C; and B2, PCL-C followed by PCL-5 plus 2 PCL-C items.

Discussion
This study of nearly 2000 US service members and veterans extends previous research comparing PTSD definitions using different PCL instruments. The results indicated that the PCL-C and PCL-5 yielded remarkably similar outcomes with both DSM-IV and DSM-5 criteria. The 17-item PCL-C was able to effectively estimate PCL-5 sum scores and PTSD prevalence (based on DSM-5 criteria); similarly, PCL-C sum scores and prevalence (based on DSM-IV criteria) were effectively estimated with the PCL-5. The addition of the 2 PCL-C items to the PCL-5 did not significantly alter estimates and thus may represent an unnecessary, albeit benign, addition. Last, the PCL-C and PCL-5 showed nearly identical associations with comorbid conditions. These findings were further strengthened by counterbalancing the order of PCL instruments. Taken together, these findings suggest that research and medical settings will successfully be able to assess PTSD over time among service members and veterans with these different instruments. In settings in which it is important to maintain consistency in assessment of PTSD, it is essential to be able to apply data from the DSM-IV PCL to estimate DSM-5 PTSD. This study demonstrated 2 applicable techniques. First, responses from the PCL-C were recoded to align with 17 corresponding PCL-5 items, and the 3 missing items were imputed. The estimated sum scores had excellent agreement with the observed PCL-5 sum scores. Although mean imputation is slightly less accurate, using it rather than multiple imputation would simplify analyses without creating substantial misclassification, although it may underestimate standard errors. Second, a previously established crosswalk 11 was used to estimate PCL-5 scores based on PCL-C sum scores. Estimated PCL-5 scores aligned closely with the observed PCL-5 scores, performing comparably well in this study population.
Thus, this crosswalk appears to be generalizable to both service members and veterans, suggesting either of these approaches may be used for research studies and surveillance efforts.  e Classified individuals who endorsed 5 or more of the 8 items as more than half the days or nearly every day, including the endorsement of anhedonia or depressed mood.
f Assessed according to a positive endorsement of any of the 5 Patient Health Questionnaire alcohol items (eg, drank while working or taking care of responsibilities, drove a car after drinking too much).
g Assessed with the standard scoring algorithm (scoring Ն3 points) of the GAD scale.
Although most settings will likely rely on DSM-5 criteria, there may be certain instances in which DSM-IV PTSD criteria still need to be estimated, for example, for comparison or replication purposes.
The current study found that the estimated prevalence of DSM-IV PTSD and sum scores were similar when the PCL-5 was used compared with PCL-C instruments. Because the expanded PCL-5 did not offer a meaningful advantage over the original PCL-5, the 20-item PCL-5 can be used to estimate PTSD with DSM-5 or DSM-IV criteria.
Findings from this cohort add to prior comparisons. 9,11,12 A study of soldiers found the PCL-5 to be highly comparable to the 17-item PCL-Specific. 9 Both that study 9 and this one found substantial agreement for 16 of 17 corresponding PCL items and no significant differences in prevalence of comorbidities of depression, generalized anxiety, or alcohol problems with PTSD measured with DSM-IV and DSM-5 criteria. 9 The discordance between instruments was also similar, although slightly lower among our study population. Taken together, these results suggest that the 2 scales are comparable for epidemiologic and clinical purposes, and replicate prior findings suggesting that the 2 definitions have comparable clinical utility with no clear advantage afforded by the changes that were made. 9,20

Limitations
A key limitation of this analysis was the absence of a clinical criterion-standard diagnosis of PTSD, which was not feasible in such a large population-based study. However, the PCL is a widely used instrument in clinical settings and research, and the true diagnostic status of participants would not change the agreement or comparison between PCL instruments. The sample was predominantly non-Hispanic White individuals and men, which may limit generalizability. The sample included both veterans and service members from all branches and components, and investigations of the Millennium Cohort Study have not demonstrated systematic sampling bias. 21,22

Conclusions
The current findings add to an increasing body of literature suggesting that the PCL-5 can be used to estimate DSM-IV PTSD and the PCL-C can be used to estimate DSM-5 PTSD. These results provide strong support for the transition from the 17-item PCL to the 20-item PCL-5 without losing the ability to monitor trends and associated comorbidities over time.