TDP indicates Teen Driving Plan; tODA, Teen On-road Driving Assessment.aIndicates withdrew before taking the primary outcome assessment at 24 weeks; an as-treated analysis was not indicated.bIndicates did not take the tODA due to noncompliance with the administrative protocol (ie, lost the hard-copy version of the learner’s permit, already received an intermediate license, or reasons were not documented).cIndicates sick/injured on day of the scheduled tODA, cancellation due to a death in the family, or reasons were not documented.
We compared termination rates among teenagers randomized to receive the Teen Driving Plan intervention (TDP group) (n = 86) and the usual practice (control group) (n = 65) using the log-rank test. The x-axis represents the modules assessed in the 24-week Teen On-road Driving Assessment, and the y-axis shows the cumulative termination rate at each module. Modules are described in the Outcome Assessments subsection of the Methods section. Censored observations are plotted with circles.
Mirman JH, Curry AE, Winston FK, Wang W, Elliott MR, Schultheis MT, Fisher Thiel MC, Durbin DR. Effect of the Teen Driving Plan on the Driving Performance of Teenagers Before LicensureA Randomized Clinical Trial. JAMA Pediatr. 2014;168(8):764-771. doi:10.1001/jamapediatrics.2014.252
Copyright 2014 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.
Many studies have failed to show an effect of parent-supervised practice driving on the driving performance of teenagers; nevertheless, most Graduated Driver Licensing programs have provisions that require supervised practice.
To determine whether a web-based intervention, the Teen Driving Plan (TDP), can improve the driving performance of teenagers before licensure as measured by the Teen On-road Driving Assessment (tODA).
Design, Setting, and Participants
Randomized, single-blind, clinical trial among 217 dyads (1 parent: 1 teenaged learner’s-permit holder) to test TDP effectiveness on increasing the quantity and diversity of supervised practice and improving the teenagers’ prelicensed driving performance. The study was conducted from December 2011 through January 2013 in Southeastern Pennsylvania.
Dyads were randomized (3:2) to receive the TDP or the Pennsylvania driver’s manual (control group). The TDP is a psychoeducational intervention designed to increase the quantity and diversity of parent-supervised practice. Materials are grouped by the following driving environments: empty parking lots, suburban residential streets, intermediate (1- or 2-lane) roads, highways, rural roads with curves and elevation changes, and commercial districts.
Main Outcomes and Measures
The main outcomes were self-reported practice driving across 6 environments and 2 conditions and driving performance as measured by the teenagers’ completion of the standardized and validated tODA 24 weeks after enrollment. Certified professional driving evaluators blinded to randomization status terminated the tODA if they determined that the teenager could not safely complete it. We examined mean differences in the quantity of supervised practice, differences in the overall proportion of teenagers in each group that had assessments terminated for unsafe driving, and the point of termination during the assessment.
The TDP dyads reported more practice in 5 of the 6 environments and at night and in bad weather compared with the control dyads. Overall, 5 of 86 TDP teenagers (6%) had the tODA terminated compared with 10 of 65 control teenagers (15%) (risk difference [TDP − control], −9% [95% CI, −21% to 2%]; P = .06). The hazard ratio for exposure to TDP was 0.35 (95% CI, 0.12-1.03; P = .05, log-rank test).
Conclusions and Relevance
Preliminary evidence suggests that the TDP improves supervised practice and the driving performance of prelicensed teenaged drivers. Future studies can explore how to revise the TDP to enhance the treatment effect and how best to disseminate the TDP without compromising implementation fidelity.
clinicaltrials.gov Identifier: NCT01498575
Graduated Driver Licensing (GDL) provisions have been enacted throughout the United States and internationally. Although GDL has demonstrated effectiveness in reducing crashes and fatalities among 16-year-old drivers, principally by delaying the onset of independent licensure, recent studies point to the need to enhance driver safety and competence directly as evidenced by the increased risk for fatal crashes among 18-year-old drivers exposed to GDL.1- 5 During the learner phase, most states impose limited requirements for supervisors (eg, a licensed adult) and practice content (eg, the number of hours required). However, parent supervisors vary in their interest, ability, and approach to supervision.6- 10 Further, parents have poor knowledge of GDL requirements and have demonstrated weak uptake of available resources.11,12 The few studies that exist on parent-supervised practice illustrate its lack of sufficient quantity, diversity, challenge, instructional quality, and fidelity to real-world driving.10,13 Owing in part to poor optimization of the learner phase, US teenagers appear to accomplish most of their learning as early independent drivers during the intermediate phase of GDL, resulting in high crash rates during this time.14 Although not the sole reason for high crash rates among teenaged novice drivers, inexperience is a major contributing factor.14- 16
Recognizing that these experience deficits can be mitigated, computer- and simulator-based training programs have been developed. Results suggest success in improving specific driving skills (eg, scanning for hazards), but there is not yet evidence to support a direct effect on drivers’ safety as measured by reductions in crash risk.17- 21 Similarly, evaluations of driver training programs primarily examining the contribution of training toward vehicle handling and studies examining the quantity of practice (eg, number of supervised practice hours) have found inconsistent safety outcomes for teenaged drivers.11,14,22- 30
Many prior evaluations of driver training programs had methodological problems (eg, lack of random assignment), and the programs have been criticized for a lack of evidence base.19,31- 34 To address this gap, the Teen Driving Plan (TDP), a web-based program for parent supervisors and teenaged prelicensed drivers, has been developed and evaluated.10,35,36 We used a randomized clinical trial design to test the hypotheses that exposure to TDP would result in (1) greater quantity of supervised driving practice across a range of environments and conditions and (2) a greater proportion of prelicensed teenaged drivers that could complete a challenging on-road driving assessment administered 24 weeks after enrollment compared with a usual-practice condition.
The TDP program includes 53 brief videos for parent supervisors on structuring practice drives to focus on specific skills (eg, merging, scanning) and creating a positive learning environment, interactive planning and logging tools, and telephone calls to promote program engagement. Collectively, these intervention components are designed to increase parental engagement and effective supervision and the social support parents provide to teenagers related to practice driving. Program materials are grouped by the following driving environments: empty parking lots, suburban residential streets, intermediate (1- or 2-lane) roads, highways, rural roads with curves and elevation changes, and commercial districts. The overall goal of the program is to increase the quantity and diversity of parent-supervised practice driving to foster development of teenagers’ driving skills before licensure. Usability testing performed by the funder, State Farm Mutual Automobile Insurance Company (State Farm), informed the final design and functionality. A detailed description of the TDP can be found in Mirman et al.36
General (eg, log-ins) and specific (eg, watching videos) use was monitored with user-specific log-in credentials. If 2 log-ins (eg, twice by the teenager, twice by the supervisor, or once each by the teenager and the supervisor) did not occur every 3 weeks, a telephone call was placed to the participants by the TDP support team to provide technical support if needed (eg, a parent forgot his log-in credentials) and/or to use a short script to encourage use of the TDP.
The Institutional Review Board of The Children’s Hospital of Philadelphia approved this study. A 2-arm randomized clinical trial design assessed how exposure to the TDP compared with a usual-practice condition. Participants in the usual-practice (control) group received a copy of the Pennsylvania driver’s manual, also readily available at all state licensing centers and online. The manual was available to families in the TDP group, but it was not provided by study personnel. Participants in both groups could seek out and use other resources, including driver education instructors. Dyads were allocated to the TDP and control groups by simple block randomization (block size = 10) using a table of random numbers. A 3:2 ratio was used to allocate participants to the TDP and control groups, respectively. Participants were not blinded to assignment status, and the allocation sequence was not concealed from the research coordinator who enrolled the dyad.
Prelicensed teenaged drivers were required to be 16 or 17 years of age at the time of enrollment, to hold a Pennsylvania learner’s permit with no more than 5 hours of behind-the-wheel practice, to be fluent in written and spoken English, to have an Internet connection in the home, to have 1 available vehicle at their primary residence, and to have a parent or guardian at least 21 years of age serve as the primary practice supervisor. Pregnant teenagers, teenagers who anticipated needing a handicapped placard or license to drive, and teenagers who had received formal behind-the-wheel instruction previously were ineligible to participate.
From December 2011 through August 2012, participants were recruited from the following 5 primary sources: (1) local high schools; (2) primary care practices affiliated with The Children’s Hospital of Philadelphia; (3) local licensing centers; (4) community events; and (5) targeted advertisements in online venues such as Facebook and Google AdWords. Fliers, letters, and advertisements describing the study were distributed via these channels. Interested participants consented and underwent screening for eligibility by a member of the study team.
On enrollment, participants completed a sociodemographic survey and were provided log-in credentials, if randomized to the TDP. Welcome calls were made to all dyads 1 week after enrollment to ensure that participants received their enrollment packet (user’s manual and log-in information [TDP group] or the Pennsylvania driving manual [control group] and remuneration). For completion of all study activities, teenagers could receive as much as $100, and parents could receive as much as $80.
Parents and teenagers separately reported the amount of practice the teenager completed in 6 environments (empty parking lots, residential neighborhoods, 1- or 2-lane roads, busy commercial districts, highways, and rural roads) and in 2 conditions (after dark and during inclement weather) using a 6-point scale ranging from none (1 point) to more than 10 hours (6 points). Surveys were sent to families 24 weeks after enrollment and remained open for 35 days.
The specific roadways and scoring procedures used for the Teen On-road Driving Assessment (tODA) were based on the California Driver Performance Evaluation (CA DPE)37,38 and adapted for use in Southeastern Pennsylvania with the assistance of a consultant with experience developing the CA DPE.39 Teenagers were administered the tODA in real traffic by a professionally trained, certified driving evaluator blinded to treatment status in a dual-control vehicle at of 24 (±3) weeks after enrollment. The tODA is 30.6 km (19.0 miles) long and consists of 7 sequential modules in 2 sets. Set 1 includes the following 4 modules: parking lot, intermediate roads, suburban commercial district, and residential neighborhood. Set 2 includes the following 3 modules: urban commercial district, highway, and rural roads (low traffic volume, 2-lane roads with curves and elevation changes) with speed limits ranging from 40 to 88 km/h (25-55 mph). The set 1 modules have lower task demands and less exposure to dangerous situations, whereas the set 2 modules present greater task demands and safety risks owing to increases in the complexity of road geometry and traffic congestion by multiple types of road users and various travel speeds. An interim on-road assessment consisting only of the set 1 modules was conducted 12 (±3) weeks after enrollment to assess for unanticipated adverse treatment effects that would require termination of the trial.
Two driving evaluators, certified by the Association for Driver Rehabilitation Specialists, were trained on tODA administration and scoring procedures. One evaluator was present in the front passenger seat of a dual-control vehicle during every assessment. Interrater reliability assessments completed on a separate sample of 19 teenagers demonstrated substantial interrater reliability (κ = 0.85 [95% CI, 0.84-0.87]).39
First, we examined the effect of the TDP on the quantity of supervised practice in each environment. Second, we examined the effect of the TDP on teenagers’ ability to complete the tODA as measured by the proportion of terminations and time to termination, expressed as the number of modules completed. Criteria for termination included: (1) a driver action or inaction requiring evaluator intervention to prevent a collision; (2) a driving task requiring assistance from the evaluator to be performed safely; (3) violation of a traffic law; (4) evasive action needed by another vehicle or a pedestrian to avoid a collision; or (5) a subjective assessment by the evaluator that the teenager could not continue safely.
To determine the sample size we used the failure rate (45%) of novice first-time applicants who had taken the CA DPE previously.37 A sample size of 100 teenagers in each group provided 80% power to detect an absolute difference of 20% in the proportion of terminations between the groups, using a 2-sided χ2 test and a type I error probability of 0.05. We anticipated an attrition rate of 20% and therefore planned to enroll 250 dyads. Limitations in scheduling resulted in a total enrollment of 217 dyads.
To assess the success of randomization and the potential for bias due to selective attrition, we compared the overall distribution of several sociodemographic characteristics in the TDP and control groups (and those who took the tODA and those who did not) using the Wilcoxon rank sum test for continuous variables and the Pearson χ2 test and Fisher exact test for categorical variables. Primary analyses were performed on the TDP and control groups as originally randomized. We compared the mean amount of practice reported by dyads (parent and teen responses were averaged) in each group using an independent-samples t test. Logistic regression and a 2-sided Fisher exact test were performed to test for differences in the proportion of the TDP and control groups having their assessment terminated, and estimated odds ratios and 95% confidence intervals were calculated from contingency tables. We calculated the number needed to treat by taking the inverse of the absolute risk reduction.
For analyses that examined the specific assessment module at which termination occurred, we used a discrete time-to-event (survival) analysis.40 We created Kaplan-Meier failure curves to depict graphically the probability that each successive assessment module was terminated and compared these distributions using the log-rank test. To quantify any differences in the risk for termination during the course of the route, we used a discrete Cox proportional hazards model to estimate unadjusted hazard ratios, using an exact method for handling tied event times. We assessed the proportional hazards assumption by considering an interaction between treatment assignment and time to termination.
Eight teenagers (6 in the TDP and 2 in the control groups) did not drive the highway module secondary to participant request at the onset of the assessment due to a lack of highway driving experience and were subsequently treated as censored at the highway driving module. A sensitivity analysis was conducted by imputing termination status for the drivers with censored data using supplementary data collected on the tODA. Specifically, at the conclusion of the drive, the evaluators rated the teenagers on a scale of 1 to 10, with a 10 being comparable to a safe, skilled, licensed adult driver. For the purposes of this study, this rating was used to impute the termination status for teenagers who did not drive the highway module. All statistical analyses were conducted using commercially available software (SAS, version 9.3 [SAS Institute, Inc] and R, version 2.15.2 [http://cran.r-project.org/bin/windows/base/old/2.15.2/]).
Of the 1207 dyads assessed for eligibility, 512 (42.4%) were eligible and agreed to participate; of the remaining 695, 85.2% were ineligible, and 14.8% actively or passively refused. The first 217 eligible dyads were scheduled to take the tODA, and the subsequent 295 dyads were assigned to a group that completed other study-related surveys (used to address other aims of the overall project) but did not take the tODA and are not the subject of the present analysis. Of the 217 teenagers assigned to take the tODA, 128 (59.0%) were randomized to receive the TDP and 89 (41.0%) to the control group (Figure 1). Participant attrition resulted in 86 teenagers in the TDP (67.2%) and 65 in the control (73.0%) groups completing the 24-week tODA.
Distributions of sociodemographic characteristics at enrollment were similar in both groups, and we found no evidence of selective attrition (Table 1). Table 2 summarizes TDP use metrics and provides descriptive statistics for the number of telephone calls made to the TDP dyads. Sixty-nine dyads (80.2%) received at least 1 telephone call; the median number of calls per dyad was 1.
The TDP dyads reported more practice in each environment and condition with the exception of highway practice compared with the control dyads (Table 3). The greatest amount of practice for TDP and control teenagers occurred on intermediate (1- or 2-lane) roads, in residential neighborhoods, and in commercial districts.
The interim assessment did not provide evidence of adverse events and did not justify an early termination of the study. The TDP teenagers took a mean (SD) of 55.6 (6.3) minutes to complete the primary outcome assessment compared with 53.5 (11.2) minutes for the control teenagers. Five of 86 TDP teenagers (6%) had assessments terminated compared with 10 of 65 control teenagers (15%) (risk difference [TDP − control], −9% [95% CI, −21% to 2%]; P = .06). The Kaplan-Meier survival plot comparing termination rates by treatment arm across assessment modules shows a higher termination rate among controls (P = .05). The TDP reduced the relative risk of teenagers being terminated for unsafe driving performance by 65% compared with the control group (hazard ratio, 0.35 [95% CI, 0.12-1.03]; P = .05) (Figure 2). We found no strong evidence that the proportional hazards assumption failed (ie, that the treatment effect differed by module) (P = .10).
We conducted a sensitivity analysis to predict the number of teenagers with censored data who would have failed the highway module had it been administered. In the overall sample, all teenagers with a driver rating of 1 failed the highway module. Those with a 2 had a 50% chance and those with a 3 or 4 had a 10% chance of failing the highway module. Based on the driver ratings among censored teenagers, the most likely outcome would have been that no teenager in the control group would have failed and 1 teenager in the TDP group would have failed. Using these imputed data, the overall estimate of TDP effectiveness changed slightly (hazard ratio, 0.42 [95% CI, 0.15-1.16]; P = .08).
Despite its ubiquity, the formal learner period of GDL has not been optimized to prepare teenagers to drive safely during early independent licensure. Recognizing this problem, many states offer materials and programming for parent supervisors; but to date, evidence-based materials and rigorous evaluations have been notably lacking. This study demonstrates that supervised practice can be increased using an evidence-based behavioral intervention. The largest increases were observed in commercial districts and residential neighborhoods, at night, and in bad weather. Further, the results suggest that using TDP might contribute to the safer driving performance of teenagers before licensure. We estimate that for every 11 teenagers who use TDP, 1 additional teenager would be prevented from failing the tODA for safety reasons.
Although this study used a strong design, including random assignment, an evaluation of driving performance blinded to group assignment, and few refusals of eligible participants, moderate attrition occurred. However, we found no evidence of selective attrition, suggesting that the benefits of randomization were preserved. The specific characteristics of our study sample and the voluntary nature of participation may limit the generalizability of the results but do not affect the internal validity of the findings.
In addition, 8 teenagers (6 in the TDP and 2 in the control groups) were not administered the highway module and were censored; however, they passed both adjacent modules. A sensitivity analysis yielded a smaller effect size. Taken together, the results from the study are promising, but replication studies with larger sample sizes are needed, especially for the driving assessment. We based our a priori sample size estimate on the CA DPE failure rates of first-time test takers, which are based on the accumulation of points from errors.41 Our use of a safety-based metric for failure provided a more direct measure of the TDP’s effect. Examining group differences in the frequency of specific errors is not possible because the opportunity to commit errors is related to time to termination, such that the most poorly performing drivers (a greater percentage of whom were in the control group) had fewer opportunities to commit errors.
Crashes, especially severe ones, are rare events requiring large sample sizes that are often infeasible in the context of complex behavioral interventions. Because crashes are caused by multiple interactive factors, how much any single behavioral intervention can appreciably affect population-level crash rates remains unclear. Rather, a comprehensive approach is needed in which evidence-based resources are properly attuned to the sociocultural and developmental needs of teenagers and families at all phases of GDL. For example, the TDP is designed to increase teenagers’ quantity and diversity of practice driving, thereby providing them with a more diverse and challenging behind-the-wheel experience before they become licensed drivers. The TDP can be complemented with other behavioral interventions administered at subsequent phases of GDL and with interventions targeting other domains, such as psychosocial and other risk factors.42 Health care providers can continue to highlight the importance of parental involvement at all stages of GDL.
The TDP was not designed to be made passively available via the Internet; ideally, it would be implemented within the context of a structured program (eg, health class, driver’s education/training program). Future analyses will include an in-depth examination of how the TDP was used, the mechanisms by which the TDP is effective, including studying the effect of the TDP on parental engagement and social support, and a determination of whether the effect of the TDP will be maintained over time. Future studies can explore how to revise the TDP to enhance the treatment effect and how to best disseminate the TDP without compromising implementation fidelity.
Preliminary evidence suggests that the TDP improves supervised practice and the driving performance of teenagers before licensure. The TDP can be complemented with other behavioral interventions administered at subsequent phases of GDL and with interventions targeting other domains, such as psychosocial and other risk factors.
Accepted for Publication: February 10, 2014.
Corresponding Author: Jessica H. Mirman, PhD, Center for Injury Research and Prevention, The Children’s Hospital of Philadelphia, 3535 Market St, Ste 1150, Philadelphia, PA 19104 (firstname.lastname@example.org).
Published Online: June 23, 2014. doi:10.1001/jamapediatrics.2014.252.
Author Contributions: Drs Mirman and Durbin had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Mirman, Curry, Winston, Elliott, Schultheis, Fisher Thiel, Durbin.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Mirman.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Curry, Wang, Elliot.
Obtained funding: Mirman, Winston, Durbin.
Administrative, technical, or material support: Mirman, Winston, Fisher Thiel, Durbin.
Study supervision: Mirman, Winston, Fisher Thiel, Durbin.
Conflict of Interest Disclosures: Dr Curry has received an additional grant from State Farm Mutual Automobile Insurance Company (State Farm), the company that funded this study. At the time of drafting of the manuscript, the funder has posted an Internet intervention, a modified version of the TDP, on its corporate website. Drs Elliot and Durbin have received consulting fees from State Farm for a different project. No other disclosures were reported.
Funding/Support: This study was supported by State Farm. The funder provided financial and in-kind support for the development of the TDP.
Role of the Sponsor: State Farm reviewed this manuscript and supported the decision to submit the manuscript for publication but did not have a role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or manuscript preparation.
Disclaimer: The findings and conclusions are those of the authors and do not necessarily represent the views of State Farm.
Additional Contributions: The creative and graphical design treatments were developed by State Farm and implemented by CDM SMITH, who developed the software. Yi-Ching Lee, PhD (Center for Injury Research and Prevention, The Children’s Hospital of Pennsylvania), Lela Jacobsohn, PhD (Center for Injury Research and Prevention, The Children’s Hospital of Pennsylvania), Ken Ginsburg, MD (Center for Injury Research and Prevention and the Craid-Dalsimer Division of Adolescent Medicine, Department of Pediatrics, The Children’s Hospital of Pennsylvania), Mary Jeffery (American Driving School), Ian Jeffery (American Driving School), and Gerald Murphy (Defensive Driving Academy) contributed to the development of the TDP. Dustin Albert, PhD (Duke University), provided input on analytic approaches to the data. Melissa Pfeiffer, MPH (Center for Injury Research and Prevention, The Children’s Hospital of Pennsylvania), provided data management and critical review of the manuscript. Julia Lockyer (Center for Injury Research and Prevention, The Children’s Hospital of Pennsylvania) contributed administrative support. Christine Norris (Center for Injury Research and Prevention, The Children’s Hospital of Pennsylvania) provided review and feedback. We thank the network of primary care clinicians and their patients and families for contributing to this project and clinical research facilitated through the Pediatric Research Consortium at The Children’s Hospital of Philadelphia. These contributions were supported by the study funding source. We also thank the many people who contributed to the design of the TDP and provided administrative support for this study and earlier studies that informed the development of the intervention.