Use of the Pediatric Symptom Checklist to Screen for Psychosocial Problems in Pediatric Primary Care: A National Feasibility Study

Background  Routine use of a brief psychosocial screening instrument has been proposed as a means of improving recognition, management, and referral of children's psychosocial morbidity in primary care.

Objective  To assess the feasibility of routine psychosocial screening using the Pediatric Symptom Checklist (PSC) in pediatrics by using a brief version of the checklist in a large sample representative of the full range of pediatric practice settings in the United States and Canada. We evaluated large-scale screening and the performance of the PSC in detecting psychosocial problems by (1) determining whether the prevalence of psychosocial dysfunction identified by the PSC was consistent with findings in previous, smaller samples; (2) assessing whether the prevalence of positive PSC screening scores varied by population subgroups; and (3) determining whether the PSC was completed by a significant proportion of parents from all subgroups and settings.

Patients and Methods  Twenty-one thousand sixty-five children between the ages of 4 and 15 years were seen in 2 large primary care networks: the Ambulatory Sentinel Practice Network and the Pediatric Research in Office Settings network, involving 395 pediatric and family practice clinicians in 44 states, Puerto Rico, and 4 Canadian provinces. Parents were asked to complete a brief questionnaire that included demographic information, history of mental health services, the 35-item PSC, and the number of pediatric visits within the past 6 months.

Results  The overall prevalence rates of psychosocial dysfunction as measured by the PSC in school-aged and preschool-aged pediatric outpatients (13% and 10%, respectively) were nearly identical to the rates that had been reported in several smaller samples (12%-14% among school-aged children and 7%-14% among preschoolers). Consistent with previous findings, children from low-income families were twice as likely to be scored as dysfunctional on the PSC than were children from higher-income families. Similarly, children from single-parent as opposed to those from 2-parent families and children with a past history of mental health services showed an elevated risk of psychosocial impairment. The current study was the first to demonstrate a 50% increase in risk of impairment for male children. The overall rate of completed forms was 97%, well within an acceptable range, and at least 94% of the parents in each sociodemographic subgroup completed the PSC form.

Conclusions  Use of the PSC offers an approach to the recognition of psychosocial dysfunction that is sufficiently consistent across groups and locales to become part of comprehensive pediatric care in virtually all outpatient settings. In addition to its clinical utility, the consistency and widespread acceptability of the PSC make it well suited for the next generation of pediatric mental health services research, which can address whether earlier recognition of and intervention for psychosocial problems in pediatrics will lead to cost-effective outcomes.

CHILDHOOD psychosocial dysfunction, considered a "new morbidity" 20 years ago, has become widely acknowledged as the most common chronic condition of children and adolescents.^1,2 Epidemiological studies report that 14% to 20% of all school-aged US children and 13% of preschoolers have an emotional and/or behavioral disorder.^3-8 The rates of psychosocial impairment seem to be higher in risk groups such as low-income, single-parent households.

Pediatricians have long been an important first resource for parents who are worried about their children's behavioral problems.⁹ With the advent of managed and especially capitated health systems,¹⁰ primary care providers assume an even greater "gatekeeping" responsibility to identify, manage, and refer children with emotional and/or behavioral disorders.¹¹ Yet, recent studies estimate that only about 50% of these children are identified by their primary care physicians and that, once identified, only a fraction of them receive appropriate mental health treatment.^9,12-15

Despite the growing burden of psychosocial morbidity, pediatricians do not receive adequate training concerning psychosocial problems, are hesitant to attach potentially deleterious labels to children,¹³ do not have time during office visits to address psychosocial needs, and may have limited access to mental health referral networks.¹⁵ Recent efforts such as the American Board of Pediatrics increasing ambulatory and behavioral training requirements² and publications such as Bright Futures¹⁶ and the Diagnostic and Statistical Manual for Primary Care¹⁷ may help to increase awareness of psychosocial morbidity over the long term, but now primary care pediatricians still struggle to provide psychosocial services.^2,16,17 Given the focus on productivity and profitability, the move to managed care approaches in medicine has created additional pressure for pediatric clinicians to limit attention on psychosocial problems.

One approach to facilitating recognition and referral of psychosocial problems is to use a parent-completed screening questionnaire as part of routine primary care visits.¹⁸ The Pediatric Symptom Checklist (PSC) was developed for this purpose. The PSC is a 1-page questionnaire of children's emotional and behavioral problems that reflects parents' impressions of their children's psychosocial functioning. Cutoff scores for school-aged and preschool-aged children indicating clinical levels of dysfunction have been empirically derived using receiver operator characteristic analyses in studies comparing the performance of the PSC with other validated questionnaires and clinicians' assessments of children's overall functioning.^19,20 In validity studies, PSC case classifications agreed with case classifications on the Children's Behavior Checklist (CBCL), clinicians' Global Assessment Scale ratings of impairment, and the presence of psychiatric disorder in a variety of pediatric and subspecialty settings representing diverse socioeconomic backgrounds.^19-25 When compared with the Children's Global Assessment Scale scores in both middle- and lower-income samples, the PSC has shown high rates of overall agreement (79% and 92%), sensitivity (95% and 88%), and specificity (68% and 100%).^21,22 Studies using the PSC have found prevalence rates of psychosocial impairment in middle-class or general settings that are quite comparable to national estimates of psychosocial problems.^19-25 More recently, efforts are under way to develop specific subscales of the PSC for use in identification of attentional, internalizing (depression/anxiety), and behavior problems (W. Gardner, MD, unpublished data, 1998).

Previous studies using a variety of measures have consistently shown that the prevalence of psychosocial impairment varies considerably based on several sociodemographic risk factors, and research with the PSC has paralleled many of these findings. For example, low socioeconomic status (SES),²⁶ living with a single parent,²⁷ parental mental illness,²⁸ family discord,²⁹ the child's temperamental characteristics, and male sex^30,31 have all been shown to increase the probability of psychosocial dysfunction. Consistent with these findings, studies using the PSC have shown the prevalence of child psychosocial dysfunction to be 2 to 3 times higher in children with low family incomes, single parents,²² or mentally ill parents.²³

Some investigators have recommended that the PSC should be considered "basic office equipment" in pediatrics¹⁸ and others have argued that the PSC should become a mandated part of all well-child visits in managed care settings or large programs like Medicaid Early Periodic Screening Diagnosis and Testing.²⁴ Routine use of a brief screening instrument like the PSC could provide a counterbalance to the previously mentioned pressures toward underrecognition, but such recommendations presuppose that the use of the PSC would be feasible across the full range of practice settings among diverse populations and regions.

Previous studies have shown that the PSC is a feasible method of early detection in selected pediatric sites, but these findings have been limited to samples of less than 500 patients. Sample sizes have been too small to investigate systematically the feasibility of using the PSC to screen for dysfunction in diverse types of pediatric practices that use varying methods of insurance reimbursement, are located in a variety of geographic regions in the United States, and that include family practice clinicians.

Our study assessed the feasibility of routine psychosocial screening with the PSC in a large national sample involving more than 21,000 pediatric outpatients drawn from the practices of 395 primary care clinicians representing 44 states, Puerto Rico, and 4 Canadian provinces. Specifically, the assessment of the PSC's feasibility was operationalized by examining whether (1) in the current sample the overall prevalence of positive PSC screening scores was consistent with previously reported findings; (2) the prevalence of positive PSC screening scores varied by sociodemographic subgroups as in previous studies; and (3) the use of the PSC was feasible for use with the full range of parents as evidenced by high and comparable rates of questionnaire completion across groups.

The study was conducted in 2 large practice-based primary care research networks: the Pediatric Research in Office Settings (PROS) network³² and the Ambulatory Sentinel Practice Network (ASPN).³³ The PROS is a pediatric network that was established in 1986 and currently comprises more than 1300 clinicians from more than 475 practices in all 50 states and the Commonwealth of Puerto Rico. The ASPN is a family medicine network that was established in 1978 and currently consists of 125 practices, with about 750 clinicians from 38 states and 6 Canadian provinces. Eighty-nine percent of PROS clinicians are pediatricians, 10% are nurse practitioners, and 1% are physician assistants. Eighty-five percent of ASPN clinicians are family physicians, 7% are nurse practitioners, and 8% are physician assistants. The ASPN also collaborated with 2 regional networks to expand the number of participating family physicians. The characteristics of the Wisconsin Research Network and the Minnesota Academy of Family Physicians Research Network are similar to those of ASPN and contributed 38 and 24 participating clinicians, respectively.

Clinicians who had previously completed PROS or ASPN research studies and were not participating in other major studies were recruited by network coordinators and staff. After agreeing to participate, all clinicians completed the practice questionnaire assessing practice structure, attitudes, and training related to psychosocial problems. After completion of the practice questionnaire, practices received training materials for the study including a videotape and written instructions.

Three hundred ninety-five pediatric and family practice clinicians in 44 states, Puerto Rico, and 4 Canadian provinces participated in data collection for the Child Behavior Study (prinicpal investiagtor, K.J.K.). Prior research from both ASPN and PROS confirms the comparability of patients, clinicians, and practices participating in primary care networks studies with those identified in national samples.^34-37 A survey conducted as part of the Child Behavior Study³⁸ showed no difference in demographic factors, practice characteristics, or attitudes toward psychosocial problems among participating pediatricians and a random sample of primary care pediatricians from the American Academy of Pediatrics. Pediatricians from the American Academy of Pediatrics, however, had minimally higher rates of patients with either private insurance or no insurance.

All children between the ages of 4 and 15 years and their parents or guardians who visited the clinician's office for health care during the practice's enrollment period were approached in the waiting room for written consent to participate in the study. Procedures and consent forms were approved by institutional review boards affiliated with PROS, ASPN, and the University of Pittsburgh, Pittsburgh, Pa. Each clinician enrolled a consecutive sample of about 60 children aged 4 to 15 years presenting for nonemergent visits in the presence of a parent or caregiver.

All study materials were mailed to the research networks for inspection, verification, and coding. Data entry was contracted out to an independent organization.

Data on each child's age, sex, racial/ethnic background, health status, history of own and parental use of mental health services, parental marital status, parental education, and use of Medicaid insurance were recorded on the clinician visit questionnaire. The highest level of education for either parent was used as an estimate of SES. Parents with a high school education or less were coded as low SES; parents with some college, a college degree, or a technical school degree were coded as middle SES; and parents with a graduate or professional degree were coded as high SES.

Psychosocial functioning was assessed using the PSC (Figure 1). The PSC consists of 35 items that are rated as never, sometimes, or often present (scored 0, 1, and 2, respectively). Item scores are summed and the total score is recoded into a dichotomous variable indicating psychosocial impairment. For children aged 6 through 16 years, the cutoff score is 28 or higher. For 4- and 5-year-old children, the PSC cutoff is 24 or higher.^20,39

Twenty-two thousand fifty-nine preschool-aged and school-aged children met the initial criteria for inclusion and were enrolled in the study. No differences in age or sex were detected in a comparison of participating children with nonparticipating children. Children in the western United States were slightly more likely to participate. Of participating children, 665 (3%) had PSCs that were missing 5 or more items and 339 (1.5%) had other missing data, leaving a final sample of 21,065 children.

As shown in Table 1, of the final sample of children, 5573 (14%) were of preschool age, 10,477 (50%) were male, 3719 (18%) were from minority backgrounds, and 5992 (28%) were from single-parent households (separated, divorced, or deceased spouse). Of the total sample, 4543 children (22%) had at least 1 parent with a graduate or professional degree; 11,677 (55%) had at least 1 parent with some college, a college degree, or a technical school degree; and 4845 (23%) had parents with a high school diploma or less. Three thousand seven hundred forty-nine (18%) were covered by Medicaid insurance. Eight hundred children (4%) were reported by their parents to have fair or poor health. Three thousand seven hundred fifty-eight (18%) had a history of using mental health services.

As presented in Table 2, the PSC identified 2077 (13%) of the school-aged children as having psychosocial dysfunction, a prevalence rate that is consistent with the rates of 12% to 14% found in previous PSC studies in middle- or mixed-income samples. Among preschoolers, the prevalence of psychosocial dysfunction was 10% (580/5573), the mean of previous reported prevalence rates (7% and 14%) in 2 small samples that used the PSC with 4- and 5-year-olds.^31,32 Analysis of PSC case rates by region showed that the rate of psychosocial dysfunction was 10% in the northeastern practices, 13% in southern and western practices, 15% in midwestern practices, and 16% in Canadian practices (P<.001). Table 2 also presents the results of previous studies of the PSC in pediatric outpatient settings^19,21-24 in comparison with those of the current study.

As in previous studies, rates of psychosocial dysfunction were about 2 times higher for low SES children (parents have a high school education or less) than for high SES children (at least 1 parent has a graduate or professional degree) both for school-aged children (20% vs 9%) and for preschoolers (15% vs 6%). Results were almost as consistent with regard to the risk factor of coming from a single-parent family. In all previous studies with the PSC, children from single-parent families were more likely to be coded as dysfunctional than children from 2-parent families, although the increase in risk ranged from 10% to 5-fold. In the current study, for both school-aged and preschool-aged children, the increase in risk was just more than double (22% vs 10% and 17% vs 8%, respectively).

As indicated in Table 3, both poverty and single-parent status were significantly associated with higher mean PSC scores. The mean PSC score of children from single-parent households was 4 points higher (18.1 [SD = 11.3])than the mean PSC score of children from 2-parent families (13.9 [SD = 9.2]; t = −25.8, df = 9315.8,P<.001). Low-income children also averaged 4 points higher on the PSC (17.2 [SD = 11.2]) compared with children of middle- (13.2 [SD = 9.0]) or higher-income status (14.9 [SD = 9.7]; F = 195.66, df =221,062, P<.001).

Previous research had demonstrated consistent significant associations between a history of mental health treatment and higher scores on the PSC. The current study not only confirmed this finding for school-aged children, but suggested that it was even stronger than had previously been reported, with a 4- to 5-fold increase in risk rather than the 2- to 3-fold increase that had been noted in the smaller previous studies (37% vs 7%). Similarly, for preschoolers, a history of mental health services nearly quadrupled the likelihood of psychosocial impairment as measured by the PSC (36% vs 8%).

Although previous studies had shown mixed results in terms of sex differences, the current study demonstrated that boys were significantly more likely to have higher PSC scores than girls for both school-aged (17% vs 9%) and preschool-aged children (12% vs 9%). As indicated in Table 3, the mean PSC score of school-aged boys was more than 3 points higher than the mean PSC score of school-aged girls (17.3 [SD = 11.2] compared with 13.8 [SD = 9.8], respectively). The difference between boys and girls seemed to be somewhat less pronounced in the preschool years, where the mean PSC score for boys was about 112 points higher than the mean for girls (14.5 [SD = 8.0] compared with 12.8 [SD = 7.7]; t = −8.0,df = 5571.0, P<.001).

The feasibility of using the PSC was assessed by comparing the percentage of forms that were returned with at least 30 of the 35 items completed (no more than 5 missing items) across sociodemographic and risk groups. The overall rate of completed forms was well within an acceptable range for all groups assessed, with at least 94% of the parents in each group fully completing the PSC form (missing data on no more than 5% of the forms). In the sample as a whole, 97% of children screened had completed PSCs. No significant differences in completion rates were found by medical specialty (either family or pediatric practice), geographic region, or managed vs non–managed care plan membership. Parents' completion rates were slightly lower in urban locales (95% vs 98% [rural] and 98% [suburban];P<.001), for low SES parents (95% vs 98%[middle SES parents] and 98% [high SES parents]; P<.001), among those parents receiving Medicaid (94% vs 98% of those not receiving Medicaid; P<.001), and among minority parents (94% vs 98% among non-minority parents; P<.001).

Parents of more than 21,000 children aged 4 to 15 years seen in 2 large primary care networks involving 395 primary care clinicians in 44 states, Puerto Rico, and 4 Canadian provinces voluntarily filled out the PSC. Unlike previous studies using the PSC, screening was successfully implemented by clinic personnel without the aid of research assistants or other personnel trained specifically for study purposes. Ninety-seven percent of participating parents completed the PSC. The successful implementation of psychosocial screening using already existing personnel and the high rate of screening completion indicates that the PSC offered a feasible method of screening 4- to 15-year-old children attending both pediatric and family medicine practices nationwide.

The prevalence of psychosocial dysfunction in school-aged children in this nationally representative sample was 13%, a rate that is virtually identical to rates of impairment reported in previous studies using the PSC in outpatient mixed SES samples.¹⁹ The rate of impairment among preschool-aged children (10%) was also comparable to rates in previous studies of smaller samples. There has been some variability in the prevalence of preschool psychosocial dysfunction as measured by the PSC.^20,39 A recent study of a low-income, predominantly Mexican American population³⁹ showed a rate of 7% among 4- and 5-year-olds, compared with 14% among a suburban, predominantly middle-income sample and 11% in the current sample. This variability may be accounted for by several factors. The lower prevalence rate found in the small sample of Mexican American preschoolers may have been a result of underreporting by a population of parents whose native language and cultural expectations were different. Previous PSC studies of preschoolers have used very small samples of about 100 subjects, which may have contributed to artificially high or low prevalence rates. In accordance with previous studies using the PSC, poverty and coming from a single-parent family doubled the risk of psychosocial dysfunction. Children with a history of mental health treatment were 4 to 5 times more likely to have high PSC scores.

This was the first study of the PSC with school-aged children to document the significantly higher rate of dysfunction for boys, a finding that was also significant for preschoolers. In both age groups, boys were about 112 times more likely to be coded as cases on the PSC. These results are consistent with epidemiological findings showing a higher prevalence of parent-identified behavioral problems and a higher rate of mental health referral in male children and adolescents than in female children and adolescents.⁷ Previous studies also suggest that girls are more likely than boys to have internalizing difficulties (ie, anxiety and depression), which may not be as readily identified and/or reported by parents or teachers.⁷

The PSC has performed as predicted in 8 published primary care studies including the current sample of more than 21,000 children. Results of these studies have consistently demonstrated that about 1 of every 8 children seen in primary care have significant levels of psychosocial problems.

Given the high rates of unrecognized, untreated psychosocial problems in children, the PSC could play an important role in comprehensive care and health supervision. If the PSC scores are high, the physician may spend a few minutes with the parents going over the items they checked off to affirm validity and assess severity. This information may help the physician facilitate a discussion of the child's major areas of functioning (family, school, activities, and mood states), notable stressors (maternal depression or parental discord), and risk factors. The physician may then determine the need for pediatric follow-up to reassess impairment, parental guidance, or mental health referral.

Facing a rising prevalence of emotional and behavioral problems and limitations on reimbursement within the managed health care system, pediatricians may wish to screen and spend available time on those children with serious impairment. While the PSC offers an approach to identifying the growing number of children with psychosocial dysfunction, trials of screening and intervention will be necessary to assure improved outcomes for children with emotional and behavioral disorders as well as to assess the cost offset of enhanced care.

Accepted for publication August 18, 1998.

This study was supported by grant MH50629 from the National Institute of Mental Health, Rockville, Md (Dr Kelleher); grant MCJ-177022 from the Bureau of Maternal and Child Health, Health Resources and Services Administration, Washington, DC; and the Staunton Farm Foundation, Pittsburgh, Pa.

We acknowledge the contributions of the PROS network of the American Academy of Pediatrics, Elk Grove Village, Ill; ASPN, Denver, Colo; the Wisconsin Research Network, Madison; and the Minnesota Academy of Family Physicians Research Network, St Paul.

Corresponding author: Michael S. Jellinek, MD, Massachusetts General Hospital, Child Psychiatry Service, ACC 725, Boston, MA 02114 (e-mail: mjellinek@partners.org).

Editor's Note: I get depressed every time I'm reminded that more than 1 in every 10 children suffer from a psychosocial problem.—Catherine D. DeAngelis, MD