Marciniak TA, Cherepanov V, Golukhova E, Kim MH, Serebruany V. Drug Discontinuation and Follow-up Rates in Oral Antithrombotic Trials. JAMA Intern Med. 2016;176(2):257–259. doi:10.1001/jamainternmed.2015.6769
Missing data are common, challenging the validity of trial results.1 However, it is unclear how to characterize the extent of missing data. The CONSORT statement2 specifies reporting number lost to follow-up but does not define it operationally. The US Food and Drug Administration (FDA) recently published a review3 providing the follow-up completeness by a specific methodology for major oral antithrombotic trials. In this report, we compare the FDA follow-up rates with the published rates. We also analyze drug discontinuation rates as a possible contributory cause of incomplete follow-up and compare them with the outcomes because excessive incomplete follow-up may cause the end point rate difference, rather than representing true drug effect.
One of us (T.A.M.), while working at the FDA, evaluated incomplete follow-up as follows: First, identify the earliest last follow-up date defined by the study documentation. Then, using all available information (eg, visits, telephone calls, hospitalizations), determine for each patient the last contact date at which end points (other than vital status alone) were ascertained. Count the patient as having incomplete follow-up if the latter date was prior to the earliest last follow-up date and the patient was not known to be dead. For the publication we used one of the terms “lost to follow-up,” “unable to contact,” “withdrew consent,” “unknown status,” or similar (as reported). For all denominators we used patients randomized. We also extracted drug discontinuation rates from the FDA review and calculated primary end point differences between arms and incomplete follow-up rates. Institutional review board approval was not required for this study.
The FDA review calculated incomplete follow-up for 23 of the 25 trials discussed. We excluded 2 of the 23 trials because their publications did not report follow-up. The Table summarizes the remaining 21 trials having both publication and FDA-calculated rates. The trials randomized 270 089 patients observed for a median (range) duration of 20 (8-43) months with last enrollment dates spanning 1995 to 2011.
The mean published rate of loss to follow-up was 0.4% (median, 0.3%; range, 0.005%-2%). The published rates were consistently lower than the FDA-calculated incomplete follow-up rates: mean, 12% (median, 13%; range, 2%-23%). There was no correlation between the published and FDA-calculated rates, as shown by the scatterplot in the Figure and a linear regression analysis (R, 0.07; P = .76). With the inclusion of all missing follow-up categories, the published rates (mean, 2.7%; median, 0.9%) remained substantially lower than the FDA-calculated rates.
Whereas the published rate of loss to follow-up is usually less than the end point rate difference, the FDA-calculated rates of loss to follow-up were substantially greater than the latter differences. The mean end point rate difference was 1.3% (median, 1.0%; range, 0.2%-3.0%).
The mean drug discontinuation rate was 24.9% (median, 23.7%; range, 7.2%-39.4%). These rates were not correlated with either publication (R, 0.28; P = .22) or FDA-calculated follow-up rates (R, 0.25; P = .27).
In this study, published rates of loss to follow-up were very low whereas the FDA-calculated incomplete follow-up rates are typically double-digit percentages uncorrelated with the published rates. The published rates consistently seem to be inadequate representations of the completeness and quality of follow-up. The extent to which the FDA-calculated rates exceed the end point rate differences implies that the end point differences may be due to differential follow-up rather than drug effect.
We recommend that incomplete follow-up rates, like P values, should be considered critical estimates of the reliability of trial results. The FDA has done so in its approval deliberations, for example, for the ATLAS trial. While its publication reported a rate of loss to follow-up of 0.3%, the FDA-calculated incomplete follow-up rate was 20%. An FDA advisory committee recommended against approving rivaroxaban for acute coronary syndromes based on the latter rate,4 and the FDA did not approve rivaroxaban for that indication.
The high rates of therapy discontinuation offer one explanation for why incomplete follow-up is common because these patients no longer need to return to the study sites to pick up drug supplies.
Prior studies have reported similar controversy regarding published rates of loss to follow-up.5,6 Our unique contribution is the comparison with the FDA independent assessment of follow-up completeness. It is clear that capturing and reporting of follow-up must be improved for better confidence in the validity of trial results. We suggest the FDA-calculated follow-up assessment methodology as a start.
Published Online: January 11, 2016. doi:10.1001/jamainternmed.2015.6769.
Conflict of Interest Disclosures: None reported.
Corresponding Author: Victor Serebruany, MD, PhD, Johns Hopkins University, Osler Medical Center, 7600 Osler Dr, Ste 307, Towson, MD 21204 (firstname.lastname@example.org).
Disclaimer: Whereas some of the work was performed while one of us was an FDA Team Leader, the opinions expressed are our own and should not be construed as official FDA policy.
Additional Information: Dr Marciniak is former Medical Team Leader, Cardio-renal Division, FDA.