The Impact of Schoolwide Positive Behavioral Interventions and Supports on Bullying and Peer Rejection: A Randomized Controlled Effectiveness Trial

Objective To build on prior research documenting the impact of School-wide Positive Behavioral Interventions and Supports (SWPBIS) on school climate and discipline problems to examine the extent to which it affects bullying and peer rejection during the transition into early adolescence.

Design Three-level models were fit using hierarchical linear modeling to determine the effect of SWPBIS on children's involvement in bullying.

Setting Thirty-seven Maryland public elementary schools.

Participants Data involved 12 344 children (52.9% male, 45.1% African American, 46.1% white) followed up longitudinally across 4 school years.

Intervention A randomized controlled effectiveness trial of SWPBIS.

Outcome Measures Reports from teachers on bully-related behaviors were assessed through the Teacher Observation of Classroom Adaptation–Checklist.

Results Analyses indicated that children in schools that implemented SWPBIS displayed lower rates of teacher-reported bullying and peer rejection than those in schools without SWPBIS. A significant interaction also emerged between grade level of first exposure to SWPBIS and intervention status, suggesting that the effects of SWPBIS on rejection were strongest among children who were first exposed to SWPBIS at a younger age.

Conclusions The results indicated that SWPBIS has a significant effect on teachers' reports of children's involvement in bullying as victims and perpetrators. The findings were considered in light of other outcomes for students, staff, and the school environment, and they suggest that SWPBIS may help address the increasing national concerns related to school bullying by improving school climate.

There has been increasing national concern regarding bullying,¹ which is broadly defined as intentional and repeated acts that occur through direct verbal, direct physical, and relational forms that typically happen when there is a power difference.² The negative effects of bullying include academic, interpersonal, physical health, and mental health problems.^3-8 Despite these growing concerns, there are relatively few school-based programs that have been shown to be effective at preventing bullying behaviors.^9,10 Many states and schools have adopted zero-tolerance policies (eg, automatic expulsion) to address bullying; however, such policies have not been shown to be effective.^11,12 An alternative to zero-tolerance policies is positive schoolwide prevention efforts that involve all school staff and are implemented across all school settings.^5,6 Our study examines the extent to which one such widely used positively oriented universal behavioral prevention model called School-wide Positive Behavioral Interventions and Supports (SWPBIS)¹³ affects teachers' reports of bullying and peer victimization using data from a randomized controlled effectiveness trial.

Consistent with the social-ecological framework,⁵ there is a movement toward the adoption of universal schoolwide programs to prevent bullying and promote a positive school climate.^14,15 These efforts typically establish a common set of behavioral expectations across all school contexts and involve all staff in prevention activities. However, the findings from research investigating the effects of antibullying programs have been mixed.^9,14,16-18 Recent research on SWPBIS¹³ suggests that it may help prevent bullying.¹⁹

SWPBIS^13,20 is a noncurricular, universal prevention model that draws on behavioral, social learning, and organizational principles. The model aims to alter the school environment by creating improved systems (eg, discipline and data management) and procedures (eg, office referral, behavioral reinforcement) that promote positive changes in staff and student behaviors. A SWPBIS team coordinates the program and establishes 3 to 5 positively stated schoolwide expectations regarding student behavior (eg, “be respectful, responsible, and ready to learn”) that are posted across settings, taught to all students and staff, and reinforced through praise and tangible rewards (eg, tickets). SWPBIS is implemented in all classroom and nonclassroom contexts. Data are collected on student behaviors including problem behaviors such as bullying and used by school staff to increase supervision and monitor the impacts of the universal program or guide the use of more intensive prevention efforts. It follows the 3-tiered public health approach to prevention,^21,22 which aims to prevent disruptive behavior by layering onto the universal SWPBIS model more targeted (selective) and intensive (indicated) programs and services to meet the needs of students who do not respond adequately to the universal system of positive behavior support. To date, most schools have focused on implementing the universal, schoolwide elements of SWPBIS.¹³ Two group randomized controlled trials were recently conducted on the universal SWPBIS model in elementary schools. They documented significant impacts on teachers' use of effective classroom management strategies, student and staff perceptions of school climate and safety, discipline problems, and academic achievement, as well as children's aggressive/disruptive behavior problems, concentration problems, emotion regulation, and prosocial behavior^23-26(also C.P.B.; T.E.W.; and P.J.L.; unpublished data, June 2011).

Our study extends this work by examining the effect of the universal SWPBIS model on bullying and peer rejection. While there has been little systematic research on bullying-related outcomes, SWPBIS includes several core elements found to be effective in a recent meta-analysis of bullying prevention programs,²⁷ suggesting that it too might affect bullying behavior. Although the theory of change process has not been explicitly examined, SWPBIS emphasizes schoolwide behavioral expectations (eg, respecting others), which likely address bullying-related behaviors. SWPBIS teaches behavioral expectations through direct instruction, positive reinforcement, and consistent consequences, promoting acceptable social and classroom behaviors. This in turn is theorized to reduce the likelihood of engaging in and rewarding bullying behavior. Furthermore, the emphasis on using data (eg, office discipline referrals, suspensions) to guide adult supervision to hot spots within the school as well as the training that staff receive on how to consistently manage behavior problems across school settings¹⁹ are hypothesized to increase the likelihood that adults will intervene more consistently when they witness bullying. When all 3 tiers are implemented, students at increased risk for involvement in bullying receive more targeted and indicated preventive interventions. Together, these core elements are hypothesized to decrease rates of bullying.¹⁹

Determining the impact of SWPBIS on bullying also has important public health significance because more than 14 000 schools across the country are implementing SWPBIS and several state departments of education recommend its use.²⁸ The large-scale dissemination of this particular prevention model highlights the significance of research examining its impact on bullying-related behaviors.

Data for our study came from a group randomized controlled effectiveness trial²⁹ of the universal SWPBIS model conducted in 37 Maryland public elementary schools to determine the impact of the model on discipline problems and the school environment. Only public elementary schools were eligible for inclusion, and all schools approached about participation agreed to enroll. An open-cohort design was used, allowing new students to enroll at each data collection; however, resources were not available to follow up students who left the participating schools. The schools were matched on select baseline demographics (eg, school enrollment, suspensions), with 21 schools being randomized by the research team to the intervention condition and 16 schools being assigned to the comparison condition, which refrained from implementing SWPBIS for 4 years. Annual assessments of SWPBIS implementation quality were conducted in all 37 schools by trained assessors, who were unaware of the schools' implementation status, using the validated School-wide Evaluation Tool.³⁰ The assessments indicated that all schools with SWPBIS reached and maintained high-fidelity implementation by the end of the trial.^24-26 The project was approved by the institutional review board at the Johns Hopkins Bloomberg School of Public Health. Passive parental consent procedures were used for student participants.

Each of the 21 schools assigned to receive SWPBIS training formed internal SWPBIS teams comprising 5 to 6 members (teachers, administrators) who attended an initial 2-day summer training led by 1 of the developers of SWPBIS. To maintain consistently high levels of implementation fidelity, the SWPBIS school teams attended annual 2-day summer booster training events. Consistent with the effectiveness trial design,³¹ all initial training and booster training events were coordinated and led by the Maryland Positive Behavioral Interventions and Supports State Leadership Team and were also attended by other SWPBIS teams from across the state.²⁸ All SWPBIS schools received on-site support and technical assistance from a trained behavior support coach (eg, school psychologist, counselor) for the duration of the trial. Additional professional development and technical assistance were provided to the behavior support coaches through state-coordinated training events conducted 4 times each year.³²

The sample included 37 elementary schools, the size of which was determined through a power analysis. A total of 5 data points (fall and spring semesters of year 1, and spring semester of years 2, 3, and 4) were collected during the course of 4 school years on 12 334 students who were in kindergarten, first grade, and second grade when the study was initiated (Figure 1).

Reports from teachers on bullying-related behaviors were assessed through the Teacher Observation of Classroom Adaptation–Checklist (TOCA-C),^33,34 a research-based measure of student behavior problems that has been well validated. The bullying subscale included 4 items: (1) teases classmates, (2) yells at others, (3) harms others, and (4) fights (α = .87). Teachers rated the 4 items on a Likert scale ranging from 1 (never) to 6 (almost always). The 4 items were averaged to create composite scores. Reports from teachers on rejection were assessed through 3 TOCA-C items: (1) is rejected, (2) does not have many friends, and (3) is not liked by classmates (α = .84), which were also averaged.

The trial was conducted from March 2002 through July 2007. Teacher TOCA-C packets were mailed to the schools and distributed by administrators, school psychologists, or administrative assistants. The packets contained a checklist in reference to each student in the classroom, and each TOCA-C had a unique identifier for each student in the class, thereby allowing individual student data to be tracked by researchers during the course of the project. Teachers completed a TOCA-C in reference to each student in the class and returned them to the research team through the mail. The teacher return rate for the TOCA-Cs was high (96.2%).

Three-level hierarchical linear modeling analyses were conducted using HLM 6.1³⁵ software to examine the effects of SWPBIS on reports of rejection and bullying during the study. Based on our prior research,²⁵ the following school-level variables were included as covariates in all models: (1) student mobility, (2) enrollment, (3) student to teacher ratio, and (4) faculty turnover rate. We also adjusted for the following individual-level characteristics, which previous research suggested may be associated with bullying and peer victimization: (1) sex, (2) grade cohort (ie, the student's grade when the study began), (3) special education status, (4) free and reduced-price meals status, and (5) ethnicity (ie, coded 1 for African American vs 0 for all others). As a post hoc exploration, select interactions were conducted among 3 demographic variables (sex, special education status, and grade cohort) and intervention status to determine if they were significant intervention effect modifiers.

Although the participation rate was consistently high, we examined the patterns of missing data but did not find evidence that the level of missingness was problematic.^36,37 For example, baseline scores on concentration problems were unrelated to subsequent missingness on this measure (adjusted odds ratio = 1.00, 95% CI = 0.96-1.04). Baseline scores on the disruptive behaviors subscale were significantly associated with increased odds of subsequent missingness (adjusted odds ratio = 1.05, 95% CI = 1.03-1.07); yet this difference was small and likely has limited clinical significance. Neither sex nor intervention status had a significant effect on subsequent missingness on teacher ratings of behavior problems. As a result, our analyses assumed data were missing at random based on the assumption that the reason for missingness was not related to the missing value itself or was deemed random after controlling for the variables that were observed.^38,39 The HLM 6.1 software adjusts parameter estimates for attrition using full-information maximum-likelihood estimation, a widely recognized and appropriate means of handling missing data⁴⁰ under the assumption that data are missing at random.⁴¹ Specifically, individuals can have missing data across any of the times and still be included in the analyses; therefore, hierarchical linear modeling is robust to this level of missing data with repeated measures.^41,42

The demographic characteristics of the sample are provided in Table 1. The sample of 12 344 children was 52.9% male, 45.1% African American, and 46.1% white. Approximately 49% of the children received free or reduced-price meals and 12.9% received special education services.

With regard to the 3-level hierarchical linear modeling results, the findings for the perpetration of bullying are reported in Table 2 and those for rejection are in Table 3. The slope terms (Figure 2) indicate that children in both conditions generally experienced an increased risk for involvement in bullying and peer rejection during the 4-year trial. However, the hierarchical linear modeling results indicated that children in the SWPBIS schools displayed significantly less bullying behavior (γ = −0.02, t = −2.60, P < .05, SE = 0.01) and experienced lower levels of rejection (γ = −0.03, t = −2.32, P < .05, SE = 0.016) over time vs children in the comparison schools (Table 1 and Table 2). We also examined for possible cross-level interaction effects of age, sex, and special education status on the impact of SWPBIS on bullying and rejection; however, only 1 interaction effect was significant (Table 2 and Table 3). The 1 significant cross-level interaction effect indicated that children in higher grades in comparison schools showed greater increases in rejection relative to their age-mates in SWPBIS schools (Figure 2C).

Given the increasing concerns about bullying⁴³ and the relative paucity of effective prevention programs,^9,10 there is a great need for further research on the impact of commonly used prevention programs on bullying. The current study used data from a randomized controlled trial of the widely disseminated SWPBIS model with the goal of exploring the impact of the program on teachers' ratings of children's perpetration of bullying behaviors and experience of peer rejection. We examined the effect of this program during late elementary school when the rates of bullying tend to increase.^43-45 Effective prevention efforts targeting this age group have the potential to attenuate the typical spike in bullying during middle school.

Consistent with prior developmental research, the data from our study indicated a significant increase in the risk for bullying and peer rejection as the children grew older. During middle school, there is less adult supervision and an increase in salience of peer relationships and social status, which in turn likely contribute to the peak in bullying.⁴⁶ However, the increases in bullying and rejection were attenuated in the schools implementing SWPBIS, indicating that in the SWPBIS environment, the typical escalation of bullying and rejection found as youth approach middle school was lessened.

There was only 1 significant cross-level interaction, which emerged for rejection; therefore, we are cautious in our interpretation of this finding. However, these data further highlight the developmental significance of being exposed to SWPBIS at a younger age because those children were least likely to experience rejection by peers. As a result of exposure to SWPBIS in elementary school, we anticipate that these children will make the transition to adolescence with a reduced risk for involvement in bullying. However, further longitudinal data are needed to confirm this hypothesis. While we were somewhat surprised that none of the other potential effect modifiers, such as sex, special education receipt, or ethnicity, proved to be significant, this finding is promising because it suggests that the main effects are rather robust for all children within schools.

Although it is difficult to discern what elements or aspects of SWPBIS accounted for the reduced risk for involvement in bullying, there are specific activities implemented through SWPBIS that likely reduce bullying, such as promoting a positive school environment based on respect, positive reinforcement of desired behaviors coupled with consistent discipline, and consequences for inappropriate behaviors.¹⁹ Furthermore, the improved organizational climate^19,25 and overall reductions in student discipline problems^23,26 observed in schools with the SWPBIS model may contribute to a more positive school environment, which also has been linked with reductions in bullying.⁴³ Additional research is needed to explore the potential mediators of SWPBIS's effects on bullying and the possibility that SWPBIS may be more effective at stemming some forms of bullying (eg, more overt forms).

There were some limitations to our study. The measures of bullying and peer rejection were rather circumscribed; future studies should incorporate more comprehensive measures of bullying and rejection that include different forms of bullying and victimization. Reports from teachers were used; however, future studies would benefit from using youth self-reports, peer reports, and/or outside observations of bullying behaviors to reduce possible biases in using 1 informant.⁴³ The intervention schools had only received training on the universal SWPBIS model; therefore, we anticipate even larger effects when the more intensive selective and indicated preventive interventions are layered onto the universal SWPBIS effort.¹³ For example, children who are at risk for involvement in bullying could receive more targeted programs, such as social skills training, whereas those who are showing early symptoms of involvement in bullying would receive more intensive counseling and therapeutic services. Furthermore, the universal SWPBIS model is not directed specifically at bullying prevention; therefore, the effects of the universal program would also likely be stronger if specific activities related to bullying had been incorporated into the training.¹⁹ Additional efforts to incorporate parents would also be beneficial; for example, schools should increase communication with parents regarding the reporting of bullying to the school⁴⁷ and effective strategies for supporting bullied youth.⁴⁸ Such efforts may more directly address issues related to bullying over and above the more general climate and behavior-enhancing universal program elements. The effect sizes were relatively small and although this is often the case in longitudinal efficacy and effectiveness studies,^31,49 the practical significance of these findings should be considered through a cost-benefit analysis. Although the schools all volunteered to participate in the trial, which limits the generalizability, a recent study used matching methods and found that the schools enrolled in the trial were similar with regard to academic achievement and levels of discipline problems to other schools in the state, suggesting a potential for the generalizability of the findings to other Maryland schools.⁵⁰ Related research indicates that schools with greater needs (eg, higher suspensions, poorer academic performance) are more likely to volunteer for SWPBIS training and eventually adopt the model.²⁸

Despite these limitations, the effects of SWPBIS on bullying are encouraging and consistent with policymakers' and researchers' emphasis on school climate and culture as potential targets for bullying prevention efforts as an alternative to zero-tolerance policies.^15,51 These findings suggest that a universal SWPBIS model is a promising approach for preventing bullying. Although the rates of bullying tend to be the highest in middle school, when SWPBIS is implemented in elementary school, it may help children better prepare for the transition into adolescence. Specifically, SWPBIS programming may suppress the increasing rates of bullying and rejection that typically occur during early adolescence. Given the extensive national network of SWPBIS schools, the model may also serve as a potential strategy through which other targeted and indicated bullying prevention approaches could be disseminated.

Correspondence: Catherine P. Bradshaw, PhD, Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, 624 N Broadway, Room 839, Baltimore, MD 21205 (cbradsha@jhsph.edu).

Accepted for Publication: July 28, 2011.

Author Contributions: Dr Bradshaw had full access to all data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Waapsdorp, Bradshaw, and Leaf. Critical revision of the manuscript for important intellectual content: Waapsdorp, Bradshaw, and Leaf. Obtained funding: Waapsdorp, Bradshaw, and Leaf.

Funding/Support: This study was supported by grants from the Centers for Disease Control and Prevention (R49/CCR318627, 1U49CE 000728, and K01CE001333-01), the National Institute of Mental Health (1R01MH67948-1A), and the Institute of Education Sciences (R305A090307).

Disclaimer: The opinions expressed are those of the authors, not of the funding agencies, and such endorsements should not be inferred.

Additional Contributions: We thank the Maryland Positive Behavioral Interventions and Supports State Leadership Team for its support of this project.