Validation of Brief Screening Tools to Identify Impaired Driving Among Older Adults in Australia

Key Points Question Are brief off-road screening measures sufficiently sensitive and specific to identify older drivers who will not pass an on-road driving test in Australia? Findings This prognostic study including 560 drivers aged 63 years and older found that off-road screening tests could identify older drivers who would not pass an on-road driving test with 77% sensitivity and 82% specificity. A combination of measures drawing from multiple skill domains provided the best prediction. Meaning These findings suggest that brief off-road screening tests could be a cost-effective, objective tool to screen older drivers to determine who might be an unsafe driver and to indicate referral for an on-road driving test.

Participants at the Canberra site took part in a detailed cognitive assessment battery conducted in person by a trained research assistant. The standardized cognitive tests included Mini-Mental Status Exam (MMSE), Victoria Stroop Test (Part A: dots, Part B: non-colour words; Part C: colour-words), California Verbal Learning Test (CVLT) immediate recall and delayed recall, Controlled Oral Word Association Test (COWAT), Boston Naming Test 15item, Benton Visual Retention Test Copy, Trail Making Test Part B, Wechsler Digit Span Backwards, and the Game of Dice Test. Participants also completed in survey format, a validated memory concerns questionnaire (Memory Complaints Questionnaire MAC-Q) and questions regarding difficulty with instrumental activities of daily living (shopping, meal preparation, using a map, making telephone calls, or taking medications), including whether those difficulties were due to memory issues or physical health issues, or both.
An algorithm was used to classify participants as meeting International Working Group (IWG) General criteria for MCI. These criteria were: absence of dementia, presence of subjective cognitive decline, presence of objective cognitive impairment on testing, and minimal impairment of IADLs. MCI etiology was not considered because this was beyond the scope of the present study, and the target population of older drivers presenting to primary care physicians for Fitness to Drive assessment is typically etiologically heterogeneous. The IWG criteria (rather than other more recent diagnostic frameworks such as DSM-5 mild neurocognitive disorder) was selected because MCI is currently the most commonly used diagnostic definition among clinicians for a pre-clinical dementia stage (1, 2).
The algorithm evaluated participant testing and survey data against each of the general MCI criteria using validated cut-off scores as presented in eTable 2. This approach has been previously validated against expert diagnosis in a sample of 1644 Australian adults aged 72-76 years(3). Participants' performance on each of the neurocognitive domains is presented in eTable 3. In general, participants in the Cognitively Impaired sub-group as a whole (which included participants meeting criteria for either MCI or dementia, and those referred to the Driver Assessment and Rehabilitation Clinic without MCI or dementia), demonstrated cognitive performance approximately 0.5 standard deviations below that of the Comparison Group.

Section 2. On-Road Driving Test Method
The route was pre-determined and incorporated situations drivers typically encounter during suburban driving. All assessments were conducted during daylight, non-peak traffic hours. Although the driving context, traffic density and roads are different between the two cities, the standardized routes were carefully mapped to be of similar duration (45-50 minutes at both sites) and distance (19-20 km at both sites), and to include similar components. At each site, route components included: traffic light controlled intersections, non-traffic controlled intersections (i.e., stop signs, give way signs), roundabouts, straight driving along single carriage as well as dual carriage roads, curved driving along single and dual carriage roads, highway driving (80-100km/hr zones), residential area driving (50-60km/hr), active school zones (Canberra only), pedestrian crossings, chicanes, one-way roads (Brisbane only), parking, 20 meter reverse, three-point turn and pull-in pull-out maneuver. At both sites, the driving instructor provided navigation instructions for 80% of the driving route. The remaining 20% of the drive was completed under self-navigation conditions where participants were instructed to drive to a pre-determined destination.
The scoring protocol was adapted from the methods typically used by driver trained Occupational Therapists (OT) in Australia when conducting on-road assessments. Seated in the rear passenger seat, the OT scored the participants' driving performance in the areas of general observation (scanning and attention), blind spot checks, lane positioning, braking/acceleration (appropriate speed and braking), gap selection (gap selected when entering traffic or the gap between the driver and other vehicles) and approach to hazards (appropriate planning and preparation).(4) Indication/signaling (appropriate use of directional indicator) was also assessed where appropriate. The final driver safety rating was standardized by ensuring OTs at both sites used a 1-10 scale at each site. Prior studies have validated this rating scale against other scoring methods (5), and compared performance on the scale against both self-reported crashes as well as state records of motor-vehicle crashes (6, 7). In this scale, a score between 1 and 3 was incurred when a driver demonstrated multiple serious driving errors which reflected loss of the skill level required to complete the driving task safely in simple and complex traffic. Typically, in these cases, the DI was required to intervene on multiple occasions to prevent an accident or dangerous situation and, if undertaking a local licensing test, the driver's performance would result in a fail and possible loss of license. A score of 4 or 5 indicate poor driving and observation skills, while a score between 6 to 8 indicated average driving skills with some bad habits, and a score of 9 to 10 indicated excellent driving and observational skills. Drivers deemed as unsafe were counselled regarding their performance on the day and advised to follow up with their general practitioner. Inter-rater reliability of test scores between the OT and DI (using the same scale) was high (intra-class correlation = 0.94 (95% CI: 0.93-0.95), n = 548). The mean safety rating at the two sites were not statistically different (Canberra: Mean=5.95 (SD=1.57); Brisbane: Mean=5.91 (SD=2.07), Mean Difference = 0.032(-0.28,0.34), t(463.9)=0.20, p=0.84) and a small inter-site reliability test conducted at the Brisbane site confirmed the two OTs had comparable ratings of participant performance on the same route (intra-class correlation = 0.90 (95%CI:0.50,0.98), n=8 (OT1=5.25(1.28); OT2=5.63(1.19), Mean Difference = -0.38 (95%CI:-0.99, 0.25), t(7)=-1.42, p=0.20).