The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial Designs | Adolescent Medicine | JAMA Network Open | JAMA Network
[Skip to Navigation]
Sign In
Views 1,087
Citations 0
Invited Commentary
Infectious Diseases
April 12, 2019

The Critical Role of Statistical Analyses in Maximizing Power Gains From Covariate-Adaptive Trial Designs

Author Affiliations
  • 1Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
JAMA Netw Open. 2019;2(4):e190789. doi:10.1001/jamanetworkopen.2019.0789

The proposed randomized clinical trial design by Stankiewicz Karita et al1 addresses the important question of whether the human papillomavirus (HPV) vaccine Gardasil-9 reduces recurrence of anal and vulvar high-grade squamous intraepithelial lesions (HSIL). The target population is unvaccinated adults with a history of anal or vulvar HSIL. The trial is generally well designed to achieve the study objective.

We first describe a potential improvement to the trial design, which involves the primary statistical analysis. The protocol’s primary analysis involves using a Cox proportional hazards model with time to recurrence as the outcome. No baseline variables are adjusted for in the primary analysis.

We recommend that the primary analysis be changed to adjust for the 3 baseline variables (called covariates) used in the covariate-adaptive randomization procedure. Such adjustment is generally recommended in randomized trials that use stratified randomization or covariate-adaptive randomization.2-4 One of the main reasons for such adjustment is to improve power. The authors are correct that using covariate-adaptive randomization can lead to greater power. However, the power gains are only realized if the reduced estimator variance due to covariate-adaptive randomization is accounted for in the analysis. Covariate adjustment can be used to obtain an accurate estimate of this variance, which could be overestimated (leading to power loss) if the covariates were ignored in the analysis. In brief, not adjusting for the covariates would effectively throw away any power gain from covariate-adaptive randomization; it could also result in confidence intervals that are wider than necessary. There are multiple statistical methods available to adjust for baseline covariates in randomized trials with time-to-event outcomes.5-7 The method of Díaz et al5 avoids making the proportional hazards assumption by focusing on estimation of the restricted mean survival time, ie, the average survival time truncated at a final point such as 36 months. All of the aforementioned methods also adjust for missing data due to loss to follow-up.

It is possible to include other baseline variables, in addition to the ones used in the covariate-adaptive randomization, in the primary analysis. This may lead to improved precision and power, as long as the new variables explain some of the residual variation that is left after adjustment for the variables in the covariate-adaptive randomization. Previous data sets and clinical knowledge could be used to identify which (if any) variables are likely to add substantial value. An important consideration is to avoid using too many baseline variables because this may lead to poor estimator performance; this is of special concern at smaller sample sizes such as when subpopulations are analyzed. All aspects of the primary analysis need to be prespecified in the study protocol, including the set of variables to be adjusted for and the statistical method to be used.

The trial goals include estimation of treatment effects not only in the combined population but also in the HIV-positive and HIV-negative subpopulations. We recommend that analyses of these subpopulations be conducted using the same approach we describe except with analyses done separately for each subpopulation. The variables adjusted for could be the same as for the combined population (as long as there are not too many variables compared with the sample sizes of the subpopulations), except excluding the indicator of baseline HIV status (as the analyses would be done separately using data from HIV-positive and HIV-negative participants, respectively).

It may be advantageous to have the trial be event driven, ie, to let follow-up continue until the total number of events (pooled across study groups) exceeds a predefined threshold (chosen to ensure the desired power at the minimum, clinically meaningful treatment effect size) or a maximum time is reached. This can help to ensure a correctly powered trial, as long as the required number of events occurs before the maximum time is reached.

A final recommendation is that the intention-to-treat analysis set be defined as all randomized participants (or all randomized participants who receive ≥1 dose of treatment or placebo). Currently, the intention-to-treat analysis set is defined as all randomized participants for whom events accrue starting at day 30 after the first vaccine dose (section 16.2 of the protocol, version 7.0.) This could lead to bias if participants drop out during the first 30 days after the first treatment or placebo dose owing to, eg, adverse effects caused by treatment.

Back to top
Article Information

Published: April 12, 2019. doi:10.1001/jamanetworkopen.2019.0789

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2019 Rosenblum M et al. JAMA Network Open.

Corresponding Author: Michael Rosenblum, PhD, Department of Biostatistics, Johns Hopkins Bloomberg School of Public Health, 615 N Wolfe St, Baltimore, MD 21205 (

Conflict of Interest Disclosures: None reported.

Stankiewicz Karita  HC, Hauge  K, Magaret  A,  et al.  Effect of human papillomavirus vaccine to interrupt recurrence of vulvar and anal neoplasia (VIVA): a trial protocol.  JAMA Netw Open. 2019;2(4):e190819. doi:10.1001/jamanetworkopen.2019.0819Google Scholar
European Medicines Agency. Guideline on adjustment for baseline covariates in clinical trials. Published February 26, 2015. Accessed February 25, 2019.
International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. E9: Statistical principles for clinical trials. Published February 5, 1998. Accessed February 25, 2019.
Kahan  BC, Morris  TP.  Improper analysis of trials randomised using stratified blocks or minimisation.  Stat Med. 2012;31(4):328-340. doi:10.1002/sim.4431PubMedGoogle ScholarCrossref
Díaz  I, Colantuoni  E, Hanley  DF, Rosenblum  M. Improved precision in the analysis of randomized trials with survival outcomes, without assuming proportional hazards [published online February 28, 2018]. Lifetime Data Anal. doi:10.1007/s10985-018-9428-5PubMed
Lu  X, Tsiatis  AA.  Semiparametric estimation of treatment effect with time-lagged response in the presence of informative censoring.  Lifetime Data Anal. 2011;17(4):566-593. doi:10.1007/s10985-011-9199-8PubMedGoogle ScholarCrossref
Lu  X, Tsiatis  AA.  Improving the efficiency of the log-rank test using auxiliary covariates.  Biometrika. 2008;95(3):679-694. doi:10.1093/biomet/asn003Google ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words