Key PointsQuestion
Does interviewing have added value compared with questionnaires in the assessment of functional outcomes in clinical research on traumatic brain injury (TBI)?
Findings
In this cohort study of 3691 individuals with TBI, there was good agreement between interview and questionnaire methods of assessing TBI outcomes, and correlations with baseline factors and patient-reported outcomes were similar between the methods. The core outcome information from interviews and questionnaires was found to be substantially similar.
Meaning
These findings suggest that in appropriate settings, questionnaires can be used to assess global functional outcomes after TBI and may offer substantial practical advantages compared with interviews.
Importance
An interview is considered the gold standard method of assessing global functional outcomes in clinical trials among patients with acute traumatic brain injury (TBI). However, several multicenter clinical trials have used questionnaires completed by a patient or caregiver to assess the primary end point.
Objective
To examine agreement between interview and questionnaire formats for assessing TBI outcomes and to consider whether an interview has advantages.
Design, Setting, and Participants
This cohort study used data from patients enrolled in the Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) project from December 2014 to December 2017. Data were analyzed from December 2020 to April 2021. Included patients were aged 16 years or older with TBI and a clinical indication for computed tomography imaging. Outcome assessments were completed using both an interview and a questionnaire at follow-up 3 and 6 months after injury.
Exposures
Traumatic brain injury of all severities.
Main Outcomes and Measures
Ratings on the Glasgow Outcome Scale–Extended (GOSE) administered as a structured interview rated by an investigator and as a questionnaire completed by patients or caregivers and scored centrally were compared, and the strength of agreement was evaluated using weighted κ statistics. Secondary outcomes included comparison of different sections of the GOSE assessments and the association of GOSE ratings with baseline factors and patient-reported mental health, health-related quality of life, and TBI symptoms.
Results
Among the 3691 eligible individuals in the CENTER-TBI study, both GOSE assessment formats (interview and questionnaire) were completed by 994 individuals (26.9%) at 3 months after TBI (654 [65.8%] male; median age, 53 years [IQR, 33-66 years]) and 628 (17.0%) at 6 months (409 [65.1%] male; median age, 51 years [IQR, 31-64 years]). Outcomes of the 2 assessment methods agreed well at both 3 months (weighted κ, 0.77; 95% CI, 0.73-0.80) and 6 months (weighted κ, 0.82; 95% CI, 0.78-0.86). Furthermore, item-level agreement between the 2 methods was good for sections regarding independence in everyday activities (κ, 0.70-0.79 across both time points) and moderate for sections regarding subjective aspects of functioning such as relationships and symptoms (κ, 0.41-0.51 across both time points). Compared with questionnaires, interviews recorded more problems with work (294 [30.5%] vs 233 [24.2%] at 3 months and 161 [26.8%] vs 136 [22.7%] at 6 months), fewer limitations in social and leisure activities (330 [33.8%] vs 431 [44.1%] at 3 months and 179 [29.7%] vs 219 [36.4%] at 6 months), and more symptoms (524 [53.6%] vs 324 [33.1%] at 3 months and 291 [48.4%] vs 179 [29.8%] at 6 months). Interviewers sometimes assigned an overall rating based on judgment rather than interview scoring rules, particularly for patients with potentially unfavorable TBI outcomes. However, for both formats, correlations with baseline factors (ρ, −0.13 to 0.42) and patient-reported outcomes (ρ, 0.29 to 0.65) were similar in strength.
Conclusions and Relevance
In this cohort study, GOSE ratings obtained by questionnaire and interview methods were in good agreement. The similarity of associations of the ratings obtained by both GOSE methods with baseline factors and other TBI outcome measures suggests that despite some apparent differences, the core information collected by both interviews and questionnaires was similar. The findings support the use of questionnaires in studies in which this form of contact may offer substantial practical advantages compared with interviews.
A popular way of assessing outcomes for clinical trials in acute traumatic brain injury (TBI) is through a clinician rating scale, particularly the Glasgow Outcome Scale (GOS) or Glasgow Outcome Scale–Extended (GOSE).1 A structured interview has become a standard method for obtaining ratings on the GOSE and is a core recommended outcome in the Common Data Elements for TBI.2
A questionnaire version of the GOSE, completed by the patient or a caregiver, has been used as an end point in several multicenter trials of acute TBI.3-6 The questionnaire format avoids investigator bias in studies such as surgical trials where masking is impractical. Questionnaires offer pragmatic advantages in overall costs and can make large-scale clinical trials of TBI feasible if industry sponsorship is lacking.7 Although there have been concerns about low follow-up rates,8 6-month GOSE outcomes were obtained for 97% of patients enrolled in the Eurotherm3235 Trial.3 In practice, studies have typically followed up nonresponders by telephone interview or another type of contact that can be organized centrally. These studies thus ultimately combined ratings derived from questionnaires and interviews in their primary end point.
Work9 to date comparing GOSE interviews and questionnaires has been small in scale and did not indicate whether there were differences in the information collected or whether an interview offered added value compared with a questionnaire. Interviewing may be expected to be superior because it allows for flexible questioning in borderline cases, the reliability of respondents can be evaluated, and when inconsistencies arise, a judgment can be made concerning the overall rating. Areas likely to need judgment include the influence of preexisting disability or extracranial injury.10 If interviews are superior, the ratings should have better validity, for example, by identifying dependency more precisely or by discounting preexisting disability. Interview ratings should be more correlated with measures of injury severity than questionnaires completed by patients and caregivers. The latter, in contrast, might be expected to be more subjective and more correlated with patient-reported outcomes.
The Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) project11 used a flexible data-collection approach to maximize follow-up. The GOSE was administered as either an interview or a questionnaire. To allow comparison of methods, the study design encouraged CENTER-TBI investigators to collect both versions of the GOSE when possible. In addition, patient-reported outcomes were used to assess health-related quality of life, mental health, and TBI symptoms. The information that was available to investigators at the time of the interview could include completed questionnaires. Thus, the comparison made in the current study concerns whether interviewing added value to the GOSE assessment and increased validity. We compared agreement of the assessments in 3 areas: (1) overall ratings, (2) individual sections of the GOSE, and (3) correlations with baseline factors and patient-reported outcomes. We also studied the use of judgment by interviewers in assigning an overall rating.
This cohort study used data from patients enrolled in the CENTER-TBI project from December 2014 to December 2017. Data were analyzed from December 2020 to April 2021. Ethical approval was obtained for each project site according to national and local procedures. A detailed ethics statement is given on the project website.12 This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.
The CENTER-TBI project included 4509 patients from 65 sites across 18 countries.13 Inclusion criteria were a diagnosis of TBI and clinical indication for a computed tomography scan, being seen in a hospital within 24 hours of the injury, and availability of informed consent (written consent was obtained at the earliest opportunity, but some patients may have been enrolled initially with oral consent). Patients were excluded if they had a severe preexisting neurological disorder that would confound outcome assessments. Additional inclusion criteria for the current analyses were an age of 16 years or older, survival at 6 months after injury, a complete and scorable GOSE interview and questionnaire for the participant at 3 months or 6 months after injury, and completion of the GOSE interview and questionnaire within 3 weeks of one another.
Demographic information was recorded at the time of recruitment along with information on the cause of injury and preexisting systemic disease based on the American Society of Anesthesiologists Physical Status Classification System.14 Injury severity was assessed using early computed tomography imaging,15 the Abbreviated Injury Scale score (scores range from 1 to 6, with higher scores indicating more severe injury) and Injury Severity Score (scores range from 1 to 75, with higher scores indicating more severe injury),16 and a baseline Glasgow Coma Scale score (scores range from 3 to 15, with higher scores indicating less severe injury).17,18
The GOSE interview19 was administered at each site either in person or by telephone. Investigators attended training and were provided a study manual including advice on supplementary questions for borderline situations, problem cases, and scoring.10 Interviewers were instructed to include disability associated with all aspects of the injury, including extracranial injury, in the rating. The weighted κ statistic (κw) for test-retest agreement is 0.92.20
The GOSE questionnaire9 consists of 14 questions in 7 sections that parallel the interview (eAppendix in the Supplement 1). Questions are designed to be appropriate for an adult patient or caregiver. The Flesch readability score for the text is 72, and the Flesch-Kincaid Grade Level is 6.5. The response choices include the option to indicate that limitations are present but are not attributable to head injury; responses using this option are not included in the scoring. Because it is not practical to assess responsiveness using a questionnaire format, the categories of “lower severe disability” and “vegetative state” are collapsed. The κw for test-retest agreement is 0.98.9
Health-Related Quality of Life
The 36-Item Short Form Health Survey, version 2 (SF-36v2)21 is a patient-reported outcome that has been used for many health conditions. The instrument has 8 subscales and 2 summary scores, the Mental Component Summary and the Physical Component Summary. Scores are transformed to T-scores (mean [SD], 50 [10]), with higher scores indicating better outcomes.
The Quality of Life after Brain Injury Scale22 is a TBI-specific measure of health-related quality of life comprising 37 items in 6 domains relevant for brain injury. Scores range from 0 to 100, with higher scores indicating better health-related quality of life.
The Patient Health Questionnaire–9 is a self-report instrument of 9 items assessing depression severity.23 Scores range from 0 to 27, with higher scores indicating greater depression. The Generalized Anxiety Disorder–7 is a 7-item self-report instrument for the severity of anxiety symptoms.24 Scores range from 0 to 21, with higher scores indicating greater anxiety.
The Rivermead Post Concussion Symptoms Questionnaire is a self-report instrument consisting of 16 symptoms typical after mild or moderate TBI.25 Scores range from 0 to 64, with higher scores indicating a greater burden of symptoms. Comparisons with this questionnaire were restricted to patients with a baseline Glasgow Coma Scale score of 9 to 15 (ie, mild or moderate injury), consistent with the context of use of this instrument. When translations were not available from the publishers, all instruments used were translated into local languages using a process of linguistic validation.26
Data Collection Procedure
Patients were enrolled from December 2014 to December 2017. Follow-up was scheduled at 3 and 6 months. The 3-month assessment was conducted either in person or by a postal questionnaire and telephone interview. The 6-month assessment was planned as an in-person meeting that included the GOSE interview; questionnaires could be completed at the time of follow-up or returned by post. To allow sites the flexibility to maximize follow-up, the use of both interviews and questionnaires was not mandated, but investigators were encouraged to collect both versions if possible.
Data were downloaded on November 22, 2019, from the Neurobot database, version 2.1 (International Neuroinformatics Coordinating Facility). Analyses were conducted from December 2020 to April 2021 using IBM SPSS, version 25 (IBM). Demographic and clinical characteristics were described using frequencies and percentages for categorical variables and medians and IQRs for continuous data.
Agreement Between Instruments
Preinjury and postinjury items in each of the 7 subsections of the GOSE were used to code whether a problem or limitation was recorded that had not been present before the injury. The strength of agreement for these 2 × 2 comparisons was evaluated using the κ statistic27 (≤0.20, poor; 0.21-0.40, fair; 0.41-0.60, moderate; 0.61-0.80, good; and 0.81-1.00, very good).28 Differences in limitations recorded in each section were evaluated by the McNemar test. Agreement between overall ratings was assessed using κw; quadratic weights penalize extreme disagreements between ratings more heavily than slight disagreements.29 The Wilcoxon signed rank test was used to test for differences between GOSE scores from the 2 formats, with r as the measure of effect size. We also compared the questionnaire and interview formats when ratings were dichotomized between upper severe disability and lower moderate disability, a common cut point for unfavorable vs favorable outcomes.
To provide an indication of the use of personal judgment in assigning overall ratings for the interviews, we identified departure from interview scoring rules. We scored the interviews centrally according to the standard procedure for the assessment and calculated the difference between interviewer ratings and central scoring. Variation in these differences across GOSE outcome categories (assigned by central scoring) was assessed using χ2 tests.
Spearman correlations were calculated between the GOSE and baseline factors typically included in prognostic models (ie, age, Glasgow Coma Scale score, pupillary reactivity, Injury Severity Score, Abbreviated Injury Scale score, and extracranial injury) and patient-reported outcome measures (ie, measures of health-related quality of life, psychological status, and TBI symptoms). A percentile bootstrap method30 was used to ascertain whether pairs of correlations were significantly different. Because multiple comparisons were made, statistical tests were considered significant only if 2-tailed P < .01.
Of 3691 individuals aged 16 years or older who were alive and eligible for follow-up 6 months after injury (eFigure in the Supplement 1), 994 (26.9%) completed both assessments at 3 months (654 [65.8%] male; median age, 53 years [IQR, 33-66 years]) and 628 (17.0%) completed both assessments at 6 months (409 [65.1%] male; median age, 51 years [IQR, 31-64 years]). The demographic and clinical characteristics of the individuals are summarized in Table 1.
Agreement Between GOSE Scores
Details of GOSE outcome assessments are given in Table 2. Agreement between formats was good at 3 months (κw, 0.77; 95% CI, 0.73-0.80) and very good at 6 months (κw, 0.82, 95% CI, 0.78-0.86). There was exact agreement for 539 three-month ratings (53.8%) and 376 six-month ratings (61.6%) (eTable 1 in the Supplement 1). An additional 306 three-month ratings (30.8%) and 179 six-month ratings (28.5%) disagreed by only 1 GOSE category. Large discrepancies (ie, ≥3 GOSE categories) were uncommon and included 58 ratings (5.8%) in the 3-month sample and 25 (4.0%) in the 6-month sample. At the 3-month follow-up, the interview rating was greater than that of the questionnaire for 195 patients (19.6%) and less than that of the questionnaire for 264 (26.6%), whereas at the 6-month follow-up, the interview rating was greater for 86 patients (13.7%) and lesser for 155 patients (24.7%). Wilcoxon signed rank tests comparing formats indicated that median GOSE scores were not significantly different at 3 months’ follow-up (Z, −1.45; r, 0.05; P = .15), whereas the difference was significant at 6 months’ follow-up (Z, −3.88; r, −0.15, P < .001).
Additional comparisons were made to examine subgroups divided by preexisting functional limitations, concomitant extracranial injury, and level of education (eTable 2 in the Supplement 1). There was at least good agreement (ie, κw, ≥0.70) between GOSE assessments for the subgroup comparisons; there was greater disagreement for patients with preexisting systemic illness than for those without.
Scores dichotomized between upper severe disability and lower moderate disability were in good agreement at 3 months (κ, 0.65; 95% CI, 0.59-0.71) and 6 months (κ, 0.75; 95% CI, 0.68-0.82). At 3 months, fewer outcomes were classified as unfavorable by the interview than by the questionnaire (30 [3.0%] vs 79 [7.9%]; P < .001), whereas at 6 months, this difference was no longer significant (14 [2.2%] vs 26 [4.1%]; P = .08) (eTable 3 in the Supplement 1).
A difference of 1 or more categories between the interviews scored centrally and the ratings assigned by interviewers was present in 257 three-month scores (26.2%) and 143 six-month scores (23.7%). There was a significant association between discrepancies and GOSE category at both 3 months (χ2, 226; P < .001) and 6 months (χ2, 213; P < .001) (eTable 4 in the Supplement 1). Interviewers tended to rate the patients classified by central scoring as having unfavorable outcomes as less disabled (particularly in GOSE category 4). Interviewers also tended to rate the patients at the upper end of the scale as more disabled (particularly GOSE category 8).
Ratings for Sections of the GOSE
Levels of agreement in individual sections of the interview and questionnaire are shown in Table 3 (frequency counts are shown in eTable 5 in the Supplement 1). Sections regarding independence at home, shopping, and travel had good levels of agreement (κ, 0.70-0.79 across both time points), as did sections regarding work and participation in social and leisure activities (κ, 0.60-0.74 across both time points). Personal relationships and symptoms that interfered with daily life had moderate levels of agreement (κ, 0.41-0.51 across both time points).
The percentages of patients who had post-TBI limitations on each section of the GOSE are shown in the Figure. Limitations were most common for TBI-related symptoms (interview: 524 [53.6%] at 3 months and 291 [48.4%] at 6 months; questionnaire: 324 [33.1%] at 3 months and 179 [29.8%] at 6 months), social and leisure activities (interview: 330 [33.8%] at 3 months and 179 [29.7%] at 6 months; questionnaire: 431 [44.1%] at 3 months and 219 [36.4%] at 6 months), and work (interview: 294 [30.5%] at 3 months and 161 [26.8%] at 6 months; questionnaire: 233 [24.2%] at 3 months and 136 [22.7%] at 6 months) and least common for independence inside the home (interview: 179 [18.3%] at 3 months and 99 [16.4%] at 6 months; questionnaire: 173 [17.7%] at 3 months and 87 [14.4%] at 6 months), for shopping (interview: 158 [16.1%] at 3 months and 78 [13.0%] at 6 months; questionnaire: 146 [14.9%] at 3 months and 68 [11.3%] at 6 months), for travel (interview: 160 [16.3%] at 3 months and 72 [11.9%] at 6 months; questionnaire: 164 [16.8%] at 3 months and 78 [12.9%] at 6 months) and for relationships (interview: 117 [12.0%] at 3 months and 82 [13.6%] at 6 months; questionnaire: 128 [13.2%] at 3 months and 104 [17.2%] at 6 months). Interviewers recorded significantly more problems with work (294 [30.5%] vs 233 [24.2%]; P < .001) and fewer problems with social and leisure activities (330 [33.8%] vs 431 [44.1%]; P < .001) at 3 months than were reported on the questionnaires; these differences remained at 6 months but were reduced (work: 161 [26.8%] vs 136 [22.7%]; P = .002; social and leisure activities: 179 [29.7%] vs 219 [36.4%]; P < .001). At both time points, interviewers recorded the presence of symptoms that interfered with daily life more often than was reported on the questionnaires (524 [53.6%] vs 324 [33.1%] at 3 months [P < .001] and 291 [48.4%] vs 179 [29.8%] at 6 months [P < .001]).
Because there were differences in recorded symptoms, we analyzed the data including all reported symptoms on the questionnaire (ie, not only the symptoms that were judged by the respondent to interfere with daily life). Agreement between the interview and questionnaire for symptoms increased to a κ of 0.56 (95% CI, 0.51-0.61) at 3 months and 0.50 (95% CI, 0.43-0.57) at 6 months. With use of this scoring, a similar proportion of patients had symptoms on the interview (524 [53.6%]) and on the questionnaire (501 [51.2%]) at 3 months (P = .13), whereas a smaller proportion of patients had symptoms on the interview (291 [48.4%]) than on the questionnaire (351 [58.4%]) at 6 months (P < .001).
Correlations between variables are shown in Table 4. Correlations of the GOSE interview and questionnaire outcomes with baseline variables were strongest for the Glasgow Coma Scale score (interview: ρ, 0.42; questionnaire: ρ, 0.44) and the Injury Severity Score (interview: ρ, −0.43; questionnaire: ρ, −0.47) and weakest for pupil reactivity (interview: ρ, −0.18; questionnaire: ρ, −0.16) and age (interview: ρ, −0.12; questionnaire: ρ, −0.06). Correlations between the GOSE and other outcomes were generally stronger than those between the GOSE and baseline factors. The strongest correlations were between the GOSE and the SF-36v2 role-physical subscale (interview: ρ, 0.63; questionnaire: ρ, 0.65), the Physical Component Summary (interview: ρ, 0.56; questionnaire: ρ, 0.55), and the Quality of Life after Brain Injury Scale’s daily life and autonomy subscale (interview: ρ, 0.58; questionnaire: ρ, 0.59). Associations with baseline factors (ρ, –0.13 to 0.42) and patient-reported outcomes (ρ, 0.29 to 0.65) were similar between the interview and the questionnaire, and none of the differences reached a significance level of P < .01.
Overall, the GOSE scores from interviews and questionnaires were in good agreement. The GOSE consists of a hierarchy of broad categories, and thus, many individuals are unambiguously assessed as being in a particular category. Cases assessed near borderlines were associated with more uncertainty.10 Disagreement between the interview and questionnaire scores by 1 category was common and suggests that borderlines may be an important factor in differences between the 2 approaches. Some cases of TBI represent a challenge for global outcome assessment and may lead to large discrepancies. These challenging cases include those in individuals with preexisting limitations, which can mask any post-TBI changes.10 Consistent with this, more disagreement between the questionnaire and interview occurred in the context of preexisting systemic disease in the present study. However, large discrepancies between the 2 formats were found in only 5.8% of cases at 3 months and 4.0% at 6 months.
Guidelines for interviewing allow the assessor to use judgment to move the rating to a higher or lower category than indicated by the responses recorded.9 We found that interviewers were using such discretion, and this was particularly prominent for individuals who potentially had outcomes regarded as unfavorable. Consistent with this, when outcomes were dichotomized at the cut point between upper severe disability and lower moderate disability, the interviews at 3 months showed fewer ratings of an unfavorable outcome than did the questionnaires. A judgment to assign a higher category may have been made because later parts of the interview are inconsistent with dependency (eg, the person is back at work). However, it may also indicate bias on the part of interviewers toward assigning particular outcomes.
Levels of agreement on individual sections of the GOSE were highest for objective aspects of functioning such as independence in activities of daily living and lowest for subjective aspects such as TBI symptoms and personal relationships. Judgments about ability to return to participation in work and social and leisure activities can be hypothetical when the individual is still recovering, and the differences between interview and questionnaire outcomes seemed to decrease as the time since the TBI increased. These may be areas where the interview allowed finer judgment, particularly in the first few months after injury.
Of note, we found that symptoms that interfere with daily life were less likely to be recorded on the questionnaire than by interviewers. The findings suggest that respondents may find it difficult to judge the association between TBI-related symptoms and daily functioning and may even be unaware of symptoms that are relevant.31 These observations are consistent with literature on mild TBI,32,33 in which symptom reports can vary substantially depending on the method of data collection. Some researchers have found that self-report questionnaires yield more reports of symptoms than do interviews,32 whereas other researchers have found the opposite.33
Concerns have been raised about the use of patient reports in TBI studies, particularly that patient reports may suggest an overly optimistic perspective of recovery owing to lack of awareness.34 However, we did not find this to be true overall. To clarify issues of informant reliability, it would be useful to conduct further research in which self-awareness was examined directly.
The associations found between the GOSE ratings and other factors were as expected from previous research.35-37 A key novel finding of this study was the similarity in the strength of the correlations between each of the 2 assessment methods and other variables. The expectation that stronger correlations would be found between interview ratings and baseline factors and between questionnaire scores and patient-reported outcomes was not shown. The concordance in the associations implies that the core information collected from interviews and questionnaires concerning global outcomes after TBI was similar. This conclusion is consistent with the good alignment that has been reported between prognostic models based on questionnaire or interview outcomes.38
This study found that there was not a definitive advantage of interviewing. In a single-center study with a limited number of data collectors, one may expect interviews to be superior to questionnaires for reasons already stated. However, in a large-scale study such as the CENTER-TBI project, the involvement of multiple interviewers may introduce additional variability in the assessment. Thus, the potential advantage given by interviews may be cancelled by interrater differences. To achieve benefit from interviewing, multicenter studies may need to address interrater differences systematically, for example, by regular and repeated training and by undertaking central monitoring of individual assessments.39,40
Investigators may wish to use the 2 formats (questionnaires and interviews) as alternative methods of data collection. The results of the current study support combining ratings from these separate sources with the caveat that some additional variation may be introduced. The study’s findings indicate that amendments to scoring may help to align the methods further. Investigators using questionnaires may also consider using interviews to obtain additional information concerning specific individuals. For example, individuals with a need for assistance in only 1 area (eg, home, shopping, or travel) would potentially be on the borderline for independence and might be followed up by interview.
The GOSE questionnaire was originally designed for postal administration and is readily adaptable for use as an online instrument or a smartphone app. The GOSE interview has been applied in conditions other than TBI, including stroke, cardiac arrest, and multiple trauma.41-43 The instrument could potentially be appropriately reworded to assess the long-term neurological consequences of other conditions that have an acute onset and before-and-after states, including sepsis and other illnesses that may manifest long-term neurological symptoms. Validation would be necessary and could open the way for large-scale data collection for a variety of conditions for which chronic neurological outcomes are underresearched.
This study has limitations. The comparison of interviews with questionnaires was a planned analysis of the CENTER-TBI project but was not an experimental design. Information about baseline factors or scores on other outcome assessments was not masked to investigators. Furthermore, systematic comparisons between different modes of data collection (ie, telephone vs in-person interviews or patients vs other informants) were not possible because of either limited numbers of cases or confounding with outcome distributions. Prospective studies are needed to compare modes of data collection. The study also excluded patients with severe preexisting neurological conditions, and this limits the generalizability of the findings to these patients; interviews may be more able to disentangle the association between complex preinjury conditions and TBI outcomes.
In this cohort study, GOSE ratings of outcomes for TBI that were obtained from questionnaires and interviews had good overall agreement. We found some disagreement between GOSE categories as well as differences in the percentages of post-TBI problems recorded by each method. We also found that interviewers used judgment in their overall ratings. However, any differences in the ratings did not translate into differences in the validity of the assessments. In this large-scale, multicenter study, interviews did not seem to offer substantial advantages compared with central scoring of information collected directly from patients and caregivers using a questionnaire. These findings support the use of questionnaires in studies in which this form of contact may offer substantial practical advantages compared with interviews.
Accepted for Publication: September 13, 2021.
Published: November 11, 2021. doi:10.1001/jamanetworkopen.2021.34121
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2021 Horton L et al. JAMA Network Open.
Corresponding Author: Lindsay Wilson, PhD, Division of Psychology, University of Stirling, Stirling FK9 4LA, United Kingdom (l.wilson@stir.ac.uk).
Author Contributions: Drs Horton and Wilson had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Horton, Menon, Maas, Wilson.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Horton, Wilson.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Horton, Wilson.
Obtained funding: Menon, Maas.
Administrative, technical, or material support: Rhodes, Menon, Maas.
Supervision: Rhodes, Menon, Maas, Wilson.
Conflict of Interest Disclosures: Dr Menon reported receiving grants from the European Union 7th Framework Programme and the National Institute for Health Research (UK) during the conduct of the study and receiving grants from Lantmannen AB, grants and personal fees from GlaxoSmithKline and PresSura Neuro Ltd, and personal fees from Calico outside the submitted work. Dr Maas reported receiving grants from the European Union 7th Framework Programme and Hannelore Kohl Stiftung to Antwerp University Hospital during the conduct of the study; grants from Integra LifeSciences and NeuroTrauma Sciences to Antwerp University Hospital outside the submitted work; and personal fees from PresSura Neuro Ltd outside the submitted work. Dr Wilson reported receiving grants from the European Union 7th Framework Programme to the University of Stirling during the conduct of the study and receiving personal fees from Roche Pharma, Vasopharm, and Novartis outside the submitted work. No other disclosures were reported.
Funding/Support: This study was funded by EC grant 602150 from the European Union 7th Framework Programme.
Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Group Members: The Collaborative European NeuroTrauma Effectiveness Research in TBI (CENTER-TBI) participants and investigators are listed in Supplement 2.
Additional Information: Data used in the preparation of this article were obtained in the context of CENTER-TBI, a large collaborative project with the support of the European Union 7th Framework Programme. The article is based on work submitted by Dr Horton for the degree of PhD at the University of Stirling, Stirling, UK.
1.Horton
L, Rhodes
J, Wilson
L. Randomized controlled trials in adult traumatic brain injury: a systematic review on the use and reporting of clinical outcome assessments.
J Neurotrauma. 2018;35(17):2005-2014. doi:
10.1089/neu.2018.5648
PubMedGoogle ScholarCrossref 5.Mendelow
AD, Gregson
BA, Rowan
EN,
et al; STITCH(Trauma) Investigators. Early surgery versus initial conservative treatment in patients with traumatic intracerebral hemorrhage (STITCH[Trauma]): the first randomized trial.
J Neurotrauma. 2015;32(17):1312-1323. doi:
10.1089/neu.2014.3644
PubMedGoogle ScholarCrossref 6.Mendelow
AD, Gregson
BA, Fernandes
HM,
et al; STICH investigators. Early surgery versus initial conservative treatment in patients with spontaneous supratentorial intracerebral haematomas in the International Surgical Trial in Intracerebral Haemorrhage (STICH): a randomised trial.
Lancet. 2005;365(9457):387-397. doi:
10.1016/S0140-6736(05)70233-6
PubMedGoogle ScholarCrossref 7.Edwards
P, Arango
M, Balica
L,
et al; CRASH trial collaborators. Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury-outcomes at 6 months.
Lancet. 2005;365(9475):1957-1959. doi:
10.1016/S0140-6736(05)66552-X
PubMedGoogle Scholar 11.Maas
AI, Menon
DK, Steyerberg
EW,
et al; CENTER-TBI Participants and Investigators. Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI): a prospective longitudinal observational study.
Neurosurgery. 2015;76(1):67-80. doi:
10.1227/NEU.0000000000000575
PubMedGoogle ScholarCrossref 13.Steyerberg
EW, Wiegers
E, Sewalt
C,
et al; CENTER-TBI Participants and Investigators. Case-mix, care pathways, and outcomes in patients with traumatic brain injury in CENTER-TBI: a European prospective, multicentre, longitudinal, cohort study.
Lancet Neurol. 2019;18(10):923-934. doi:
10.1016/S1474-4422(19)30232-7
PubMedGoogle ScholarCrossref 14.Dripps
RD. New classification of physical status.
Anesthesiology. 1963;24:111.
Google Scholar 15.Vande Vyvere
T, Wilms
G, Claes
L,
et al; Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) Investigators and Participants. Central versus local radiological reading of acute computed tomography characteristics in multi-center traumatic brain injury research.
J Neurotrauma. 2019;36(7):1080-1092. doi:
10.1089/neu.2018.6061
PubMedGoogle ScholarCrossref 21.Ware
JE, Kosinski
M, Bjorner
JB, Turner-Bowker
DM, Gandek
B, Maruish
ME. User’s Manual for the SF-36v2 Health Survey. 2nd ed. QualityMetric Inc; 2007.
22.von Steinbüchel
N, Wilson
L, Gibbons
H,
et al; QOLIBRI Task Force. Quality of Life after Brain Injury (QOLIBRI): scale validity and correlates of quality of life.
J Neurotrauma. 2010;27(7):1157-1165. doi:
10.1089/neu.2009.1077
PubMedGoogle ScholarCrossref 25.King
NS, Crawford
S, Wenden
FJ, Moss
NEG, Wade
DT. The Rivermead Post Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced after head injury and its reliability.
J Neurol. 1995;242(9):587-592. doi:
10.1007/BF00868811
PubMedGoogle ScholarCrossref 26.Acquadro
C, Conway
K, Girourdet
C, Mear
I. Linguistic Validation Manual for Health Outcome Assessments. MAPI Institute; 2012.
28.Altman
DG. Practical Statistics for Medical Research. London: Chapman & Hall; 1991.
36.Nelson
LD, Ranson
J, Ferguson
AR,
et al. Validating multidimensional outcome assessment using the TBI Common Data Elements: an analysis of the TRACK-TBI Pilot sample.
J Neurotrauma. 2017. doi:
10.1089/neu.2017.5139
PubMedGoogle Scholar 37.Kreitzer
NP, Hart
K, Lindsell
CJ,
et al. A comparison of satisfaction with life and the Glasgow Outcome Scale-Extended after traumatic brain injury: an analysis of the TRACK-TBI pilot study.
J Head Trauma Rehabil. 2019;34(3):E10-E17. doi:
10.1097/HTR.0000000000000457PubMedGoogle Scholar 38.Roozenbeek
B, Lingsma
HF, Lecky
FE,
et al; International Mission on Prognosis Analysis of Clinical Trials in Traumatic Brain Injury (IMPACT) Study Group; Corticosteroid Randomisation After Significant Head Injury (CRASH) Trial Collaborators; Trauma Audit and Research Network (TARN). Prediction of outcome after moderate and severe traumatic brain injury: external validation of the International Mission on Prognosis and Analysis of Clinical Trials (IMPACT) and Corticoid Randomisation After Significant Head injury (CRASH) prognostic models.
Crit Care Med. 2012;40(5):1609-1617. doi:
10.1097/CCM.0b013e31824519ce
PubMedGoogle ScholarCrossref 39.Wilson
JTL, Slieker
FJA, Legrand
V, Murray
G, Stocchetti
N, Maas
AIR. Observer variation in the assessment of outcome in traumatic brain injury: experience from a multicenter, international randomized clinical trial.
Neurosurgery. 2007;61(1):123-128. doi:
10.1227/01.neu.0000279732.21145.9e
PubMedGoogle ScholarCrossref 40.Boase
K, Machamer
J, Temkin
NR,
et al; TRACK-TBI Investigators. Central curation of Glasgow Outcome Scale- Extended (GOSE) data: lessons learned from TRACK-TBI.
J Neurotrauma. 2021;38(17):2419-2434. doi:
10.1089/neu.2020.7528
PubMedGoogle ScholarCrossref 41.Langham
J, Reeves
BC, Lindsay
KW,
et al; Steering Group for National Study of Subarachnoid Haemorrhage. Variation in outcome after subarachnoid hemorrhage: a study of neurosurgical units in UK and Ireland.
Stroke. 2009;40(1):111-118. doi:
10.1161/STROKEAHA.108.517805PubMedGoogle ScholarCrossref