Validity and Responsiveness of the Pediatric Quality of Life Inventory (PedsQL) 4.0 Generic Core Scales in the Pediatric Inpatient Setting

Importance  Validated patient-reported outcomes responsive to clinical change are needed to evaluate the effectiveness of quality improvement interventions.

Objectives  To evaluate responsiveness, construct validity, and predictive validity of the Pediatric Quality of Life Inventory (PedsQL) 4.0 Generic Core Scales in the pediatric inpatient setting.

Design, Setting, and Participants  Prospective, cohort study of parents and caregivers of patients 1 month to 18 years old (n = 4637) and patients 13 to 18 years old (n = 359) admitted to Seattle Children’s Hospital between October 1, 2011, and December 31, 2013. Of 7184 eligible participants invited to complete the survey, 4637 (64.5%) completed the PedsQL on admission, and of these 2694 (58.1%) completed the follow-up survey 2 to 8 weeks after discharge.

Main Outcomes and Measures  Responsiveness was assessed by calculating improvement scores (difference between follow-up and admission scores). Construct validity was examined by comparing the mean improvement scores for known groups differing by medical complexity. Predictive validity was assessed using Poisson regression to examine associations among admission scores, prolonged length of stay (≥3 days), and 30-day readmissions or emergency department (ED) return visits. Similar models examined the association between improvement scores and risk for 30-day readmissions or ED return visits.

Results  The mean (SD) PedsQL improvement scores (scale, 0-100) were 22.1 (22.7) for total, 29.4 (32.4) for physical, and 17.1 (21.0) for psychosocial. The mean PedsQL total improvement scores were lower for patients with medically complex conditions compared with patients without chronic conditions (13.7 [95% CI, 11.6-15.8] vs 24.1 [95% CI, 22.4-25.7], P < .001). A 10-point decrement in the PedsQL total admission score below the established community-based mean was associated with an increase in risk for prolonged length of stay (15% [95% CI, 13%-17%]), 30-day readmissions (8% [95% CI, 3%-14%]), and ED return visits (13% [95% CI, 6%-20%]). A 5-point decrement in the PedsQL total improvement score below the study sample mean improvement score was associated with an increase in risk for 30-day readmissions or ED return visits (9% [95% CI, −1% to 19%]).

Conclusions and Relevance  The PedsQL demonstrated responsiveness, construct validity, and predictive validity in hospitalized pediatric patients. The PedsQL may be a useful patient-reported outcome for hospital-based clinical effectiveness research.

Patient-reported outcomes (PROs) are increasingly being used in pediatric health services research to evaluate outcomes and to inform clinical decision making from the patient and family perspective.^1-3 To measure the effectiveness of clinical interventions, guidelines recommend that these measures should be brief, demonstrate a high degree of reliability and validity in multiple patient populations, and include both age-appropriate and parallel versions for child and parent-proxy raters.^1,4,5 These measures should also be clearly linked to the delivery of specific health services and detect meaningful variation in outcomes over multiple assessments.^3,6

PRO measures have been used extensively in outpatient settings⁷; however, most of these measures used in the inpatient setting have focused on patient experience and satisfaction, largely in adult populations.^8,9 Expanding the evaluation of PRO measures to include health status, health-related quality of life (HRQOL), symptoms, and functional status provides a more comprehensive evaluation of the clinical effectiveness of inpatient health services from the patient and family perspective. While HRQOL measures have been validated in the adult inpatient setting,¹⁰ no HRQOL measures have been validated for use in the pediatric inpatient setting to our knowledge.

The Pediatric Quality of Life Inventory (PedsQL) 4.0 Generic Core Scales and the PedsQL Infant Scales are population HRQOL measures that have demonstrated good reliability and construct validity in a wide variety of general^11-14 and disease-specific^15,16 populations and have demonstrated responsiveness to meaningful change.^17-22 The PedsQL 4.0 Generic Core Scales instrument consists of the following 4 domains: (1) physical functioning, (2) emotional functioning, (3) social functioning, and (4) school functioning. It includes formats for typically developing children and adolescents 2 to 18 years old (parent-proxy report) and 5 to 18 years old (self-report). The minimal clinically important difference calculated in previous investigations is 4.50 to 6.92 points by parent-proxy report and 4.36 to 6.66 points by child self-report.¹² The PedsQL Infant Scales instrument consists of the following 5 domains: (1) physical functioning, (2) physical symptoms, (3) emotional functioning, (4) social functioning, and (5) cognitive functioning. It includes formats for typically developing children 1 to 24 months old (parent-proxy report).¹⁴ The format, instructions, response scale, and scoring methods of the Infant Scales are identical to those of the Generic Core Scales.

The aim of this study was to evaluate responsiveness, construct validity, and predictive validity of the PedsQL 4.0 Generic Core Scales and the PedsQL Infant Scales in a population of hospitalized children. We hypothesized that improvement in the PedsQL scores from admission to follow-up would be higher than the minimal clinically important difference published in the ambulatory literature,¹² demonstrating improvement responsiveness of the measure in the inpatient setting. We also hypothesized that the PedsQL improvement scores would be significantly lower for patients with chronic conditions compared with patients with no chronic conditions. We further hypothesized that lower PedsQL admission scores would be associated with both longer length of stay (LOS) and increased risk for 30-day readmissions and 30-day emergency department (ED) return visits. Finally, we hypothesized that lower PedsQL improvement scores would be associated with higher rates of 30-day readmissions and 30-day ED return visits.

All study procedures were reviewed and approved by the Seattle Children’s Research Institute Institutional Review Board. Informed verbal consent was obtained from all participants.

The study included patients admitted to Seattle Children’s Hospital medical or surgical units between October 1, 2011, and December 31, 2013. Patients were eligible unless they were younger than 1 month, were in protective isolation (ie, immunosuppressed), had been approached to participate within the past 2 months, were admitted for 24-hour video electroencephalogram or pH probe study, were admitted for suspicion of child abuse, or were developmentally delayed. The rationale for excluding patients with developmental delays was based on focus groups conducted with caregivers of children with atypical development at our hospital. These parents found the age-appropriate PedsQL survey items difficult to answer for their children, secondary to having low applicability to their physical and psychosocial abilities.

Eligible families were invited to participate within 72 hours of admission by a trained research assistant using a standard protocol for recruitment and consent or assent. Parents and caregivers (hereafter referred to as caregivers) of admitted patients 1 to 24 months old were asked to complete the PedsQL Infant Scales instrument, while caregivers of admitted patients 2 to 18 years old were asked to complete the PedsQL 4.0 Generic Core Scales instrument. Patients 13 to 18 years old were asked to complete the self-report PedsQL 4.0 Generic Core Scales instrument. Patients determined to be too ill to complete the survey were not invited to participate.

The survey was administered within 72 hours of admission and at 2 to 8 weeks after discharge from the hospital. Caregivers and patients were asked to reflect on the time since their child or they had to come into the hospital (admission survey) and on the past 7 days (follow-up survey) when responding to the PedsQL. The admission survey also collected patient age, sex, and race/ethnicity and caregiver age and education level. Participants were given the option to complete the admission survey electronically using a laptop computer or by telephone interview. Follow-up surveys were completed as an online web-based survey or by telephone interview. Surveys were offered in English, Spanish, Vietnamese, Russian, Somali, and Chinese, and caregivers or patients who did not speak any of these languages were considered ineligible for the study. Spanish surveys were created using forward and backward translation methods and were reviewed by professional interpreters at Seattle Children’s Hospital before administration.

Statistical analysis was conducted using scale scores from the PedsQL as the main outcome measure. Each PedsQL item was reverse scored and linearly transformed to a scale of 0 to 100; therefore, higher scores reflected better HRQOL. For each caregiver or patient respondent, a total, physical, or psychosocial summary score was computed as the sum of the items divided by the number of items answered. The PedsQL total score was composed of the physical and psychosocial scores. The physical score combined the physical functioning and physical symptoms scores for the Infant Scales. The psychosocial score was composed of the emotional, social, and school functioning (or cognitive functioning for the Infant Scales) scores. If more than 50% of the items in the instrument were missing, a total, physical, or psychosocial summary score was not computed (8.2% of the sample). Mean imputation was used to compute scale scores for surveys with 50% missing items or less.¹²

Improvement scores were derived by calculating the difference between the total, physical, and psychosocial scores on admission from the same scores at follow-up. Higher improvement scores reflected greater improvement in HRQOL from admission to follow-up.

Responsiveness was analyzed by computing the means (SDs) of admission and follow-up scores. Paired t test and Cohen d analyses were conducted to examine differences and effect sizes between admission and follow-up scores for data reported by caregivers and patients.

Construct validity was analyzed by comparing the means (SDs) of improvement scores across known groups differing in medical complexity (known-groups validity). We stratified our study population using the Pediatric Medical Complexity Algorithm,²³ which classifies patients as having no chronic illness (eg, febrile seizure), noncomplex chronic illness (eg, epilepsy), or complex chronic illness (eg, epilepsy with chronic respiratory insufficiency) on the basis of up to 3 years of retrospective International Classification of Diseases, Ninth Revision, Clinical Modification codes, beginning with the date of admission. We used linear regression to evaluate the difference in the mean scores between medical complexity categories. We adjusted for LOS and follow-up time (difference between the discharge date and the follow-up survey completion date) in our analysis because we hypothesized that prolonged LOS (≥3 days) and longer time to follow-up survey completion (4-8 weeks) may be associated with decreased and increased improvement scores, respectively. Cohen d effect sizes were derived by comparing unadjusted means (SDs) of each chronic condition group (noncomplex chronic illness and complex chronic illness) with the group having no chronic illness.

To assess predictive validity, we constructed 3 models using modified Poisson regression²⁴ to determine risk for (1) LOS of 3 days or longer, (2) 30-day unplanned readmissions, and (3) 30-day ED return visits associated with a 10-point decrement in the PedsQL admission score below the established community-based population means for the PedsQL.¹⁶ We used a similar model to determine risk for 30-day unplanned readmissions or 30-day ED return visits associated with a 5-point decrement in the PedsQL improvement score below the study sample mean improvement score. This latter analysis was limited to participants who completed a follow-up survey within 30 days of discharge (n = 1505) and participants who completed the follow-up survey before their readmission or ED return visit (n = 25). Readmissions were coded as unplanned using the method developed by Berry et al.²⁵ All models were adjusted for patient age, sex, race/ethnicity, language, and level of medical complexity and for caregiver age and education level.¹² For the model examining the association between the PedsQL improvement scores and risk for 30-day readmissions and ED return visits, we also adjusted for LOS of 3 days or longer.

For all analyses, we used a single set of PedsQL admission or improvement scores for each caregiver-patient dyad. These were computed using patient scores if the patient was 13 years or older and completed the survey, while caregiver scores were used otherwise.

Of 19 139 patients discharged from the medical or surgical units between October 1, 2011, and December 31, 2013, a total of 10 866 (56.8%) families were eligible for study participation. The primary reasons for ineligibility were having a developmental delay (29.0% [2399 of 8273]) or participation in the study within the past 2 months (28.0% [2316 of 8273]). A total of 33.9% (n = 3682) of potentially eligible families were not approached because of unavailability of research assistants during evening and weekend hours. The response rate for eligible families invited to complete the PedsQL on admission was 64.5% (4637 of 7184), and of these 58.1% (2694 of 4637) completed a follow-up survey using a combination of modes (online, followed by telephone). The range in time to follow-up survey completion was 2 to 8 weeks after discharge. The study sample was similar to the overall medical and surgical unit populations; however, study patients were older and healthier (Table 1). Only 1.4% of the study sample completed the survey in a language other than English or Spanish.

The PedsQL demonstrated responsiveness, with significant improvement in the PedsQL scores from admission to follow-up and with large effect sizes reported by caregivers and patients (Table 2). The mean (SD) PedsQL improvement scores were 22.1 (22.7) for the total score, 29.4 (32.4) for the physical domain, and 17.1 (21.0) for the psychosocial domain. Improvement scores were highest on the physical domain, and caregiver-patient dyads reported similar scores overall, with patients reporting slightly higher psychosocial scores. The PedsQL also demonstrated moderate variability in responsiveness by age and minimal variability for patients with medical vs surgical conditions (eTable 1 and eTable 2 in the Supplement).

The PedsQL demonstrated construct validity among known groups differing by medical complexity (Table 3). Caregivers and patients with no chronic illness reported significantly higher total, physical, and psychosocial improvement scores than patients with noncomplex chronic illness or patients with complex chronic illness.

In adjusted analyses, a 10-point decrement in the PedsQL total admission score below the established community-based mean was associated with a 15% (95% CI, 13%-17%) increase in risk for LOS of 3 days or longer, an 8% (95% CI, 3%-14%) increase in risk for 30-day unplanned readmissions, and a 13% (95% CI, 6%-20%) increase in risk for 30-day ED return visits. A 10-point decrement in the PedsQL physical admission score below the established community-based mean was associated with a 10% increase in risk for LOS of 3 days or longer, a 5% increase in risk for 30-day unplanned readmissions, and an 11% increase in risk for 30-day ED return visits. A 10-point decrement in the PedsQL psychosocial admission score below the established community-based mean was associated with a 12% increase in risk for LOS of 3 days or longer, a 9% increase in risk for 30-day unplanned readmissions, and a 7% increase in risk for 30-day ED return visits (Table 4).

The unadjusted analysis indicated significant associations between decrements in the PedsQL total and physical improvement scores and risk for 30-day unplanned readmissions or ED return visits. However, these associations were no longer statistically significant after adjusting for covariates (Table 5).

This study demonstrated improvement responsiveness of the PedsQL over short periods for recently hospitalized pediatric patients. The findings also provide support for the instrument’s construct validity and predictive validity in the pediatric inpatient setting. The PedsQL is brief and easy to complete, resulting in reasonable response rates for the present study and supporting its use to assess the clinical effectiveness of quality improvement interventions among pediatric inpatients.

To our knowledge, this is the first study to evaluate responsiveness of the PedsQL in a large diverse population of hospitalized pediatric patients. The observed magnitude of improvement in the PedsQL scores was larger than that in prior outpatient studies^17-21 measuring the PedsQL responsiveness and was significantly higher than the published minimal clinically important difference,¹² demonstrating improvement responsiveness of the instrument in the pediatric inpatient setting. This is likely a reflection of the degree of decline in physical health perceived by caregivers of patients requiring hospitalization compared with patients managed in the ED or outpatient setting. For example, children requiring hospitalization may not achieve activities of daily living such as maintaining adequate oral intake or full ambulation, resulting in admission scores that are lower than population means for ill children managed in the ED or as outpatients.^11,21 Improvement in both caregiver-reported and patient-reported PedsQL scores was highest for the physical domain. This is not surprising because short-term inpatient treatment for medical and surgical patients is primarily targeted toward improving a patient’s physical health. Those endeavoring to evaluate quality improvement interventions using HRQOL outcomes in this patient population should consider focusing on physical improvement scores because the use of the total improvement score may blunt the ability to detect the effectiveness of inpatient treatment. Finally, adolescent patients reported higher admission and follow-up scores compared with their caregivers, primarily on the psychosocial domain. This is consistent with investigations comparing patient-reported and observer-reported HRQOL assessments demonstrating that caregivers may rate HRQOL more accurately on visible domains such as physical health vs less visible domains such as psychosocial health.²⁶ This finding provides additional evidence for the importance of obtaining child and adolescent self-report when possible for measures of HRQOL.^27,28

Patients with noncomplex or complex chronic conditions and their caregivers reported significantly lower improvement in HRQOL compared with those with no chronic conditions, demonstrating construct validity of the PedsQL in the inpatient setting. We expect patients and caregivers of youth with no chronic illness to report lower scores on admission because the decline in physical health prompting inpatient treatment is perceived to be appreciably large relative to their healthy baseline functioning. In contrast, patients and caregivers of youth with complex chronic conditions may not perceive their physical health on admission to be remarkably lower relative to their baseline functioning. This is supported by the finding that patients with complex chronic illness reported lower scores at follow-up, suggesting that they return to a lower baseline health status after hospitalization. This is consistent with studies^11-13,16 in the ambulatory literature in which children with chronic illness report lower baseline HRQOL compared with healthy children.

Clinical effectiveness research in the pediatric inpatient setting has largely been limited to measuring morbidity, mortality, and health care utilization outcomes, primarily related to specific clinical diseases.²⁹ However, because mortality and readmission are rare events in pediatrics, using these metrics to assess the quality of hospital care has recently been called into question.^25,30,31 In contrast, PROs such as the PedsQL may be routinely collected in a population of hospitalized children, and this is the first study to our knowledge evaluating predictive validity of the PedsQL in the inpatient setting in relation to utilization outcomes. Although the PedsQL is a subjective measure based on caregiver or patient perceptions, the findings of this study demonstrate that this measure is associated with more traditional, objective outcomes commonly used to measure the effectiveness of quality improvement interventions. Evaluating quality improvement interventions using all these outcomes collectively may provide a more comprehensive assessment of the quality of hospital care that is relevant to multiple stakeholders.

The 2010 Patient Protection and Affordable Care Act challenges health systems, health care professionals, and researchers to routinely measure outcomes relevant to patients and families to assist them in making informed decisions that are consistent with their values, preferences, and goals. Understanding HRQOL improvement in pediatric patients during and after hospitalization provides an opportunity to incorporate PROs into comparative effectiveness research.^32,33 By systematically assessing PROs through an integrated clinical and research environment, the effectiveness of interventions aimed at improving care can be measured from the patient and family perspective.³⁴

Our study had several limitations. This study was conducted at a single institution, limiting the generalizability of the results; however, they are likely comparable to other US tertiary children’s hospitals with large referral bases. Response rates were on the lower end of acceptable for the admission and follow-up surveys and may have resulted in nonresponse bias; however, respondents were largely representative of the medical and surgical unit patient populations. The study had low participation among adolescent patients, and although caregiver scores were similar to adolescent self-reports, direct measurement of patient self-report is preferred. The study follow-up time frame was prolonged (2-8 weeks), and a narrower time frame (within 30 days of discharge) may have provided a more direct assessment of the effect of inpatient health services on the HRQOL among this study population. The number of respondents who completed a follow-up survey before a 30-day readmission or ED return visit was low, potentially resulting in reduced statistical power to detect a significant association between decrements in the PedsQL improvement scores and increased risk for utilization outcomes. Finally, although the findings represent gross associations that may be self-evident, establishing responsiveness and validity of this instrument was a necessary first step to incorporating HRQOL outcomes in the evaluation of inpatient health services.

The PedsQL demonstrated improvement responsiveness over short periods and construct validity and predictive validity in hospitalized pediatric patients. Given these findings, the PedsQL may be useful to assess the clinical effectiveness of hospital-based quality improvement interventions from the patient and family perspective.

Accepted for Publication: July 13, 2014.

Corresponding Author: Arti D. Desai, MD, MSPH, Department of Pediatrics, University of Washington, PO Box 5371, Mail Stop CW8-6, Ste 400, Seattle, WA 98145 (arti.desai@seattlechildrens.org).

Published Online: October 27, 2014. doi:10.1001/jamapediatrics.2014.1600.

Author Contributions: Drs Desai and Mangione-Smith had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Desai, Zhou, Varni, Mangione-Smith.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Desai, Mangione-Smith.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: All authors.

Obtained funding: Mangione-Smith.

Administrative, technical, or material support: Mangione-Smith.

Study supervision: Mangione-Smith.

Conflict of Interest Disclosures: Dr Varni reported holding the copyright and the trademark for the Pediatric Quality of Life Inventory and reported receiving financial compensation from the Mapi Research Trust, which is a nonprofit research institute that charges distribution fees to for-profit companies that use the Pediatric Quality of Life Inventory. No other disclosures were reported.

Funding/Support: This research was supported by the Continuous Performance Improvement Department, Seattle Children’s Hospital.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank the staff of the Outcomes Assessment Program, Seattle Children’s Hospital, for their contributions to data collection and management.