[Skip to Navigation]
Sign In
Editor's Note
May 2017

How We Evaluate Biomarker Studies

JAMA Cardiol. 2017;2(5):524. doi:10.1001/jamacardio.2017.0291

The abundance of data sets with stored blood or tissue samples, combined with a societal interest in personalized medicine, have led to a proliferation of biomarker studies. However, among the many markers evaluated, few reach clinical practice and guidelines. In fact, highly cited or “first-in-print” studies frequently overestimate biomarker performance compared with subsequent meta-analyses.1 False-positive or overstated associations can misdirect science and potentially clinical care. Below are some of the questions we at JAMA Cardiology ask when we evaluate biomarker studies and their value to our readership.

Is There a Clear Objective?

Biomarker studies can aim to elucidate new disease mechanisms, aid diagnosis, improve risk prediction, personalize treatment, and establish surrogacy for clinical end points and outcomes. Each of these requires different study designs and hypothesis testing.

Is the Data Source Appropriate?

A representative at-risk population for the disease or outcome of interest is critical. Biomarker performance in convenience samples or highly specialized cohorts may not satisfactorily replicate or be useful in general practice. Biomarker evaluation for treatment benefit (prognostication) is best performed in the context of randomized trials of the therapy of interest, rather than in observational cohorts that may be susceptible to confounding by treatment selection.

Are the Statistics Sound?

An adequate number of outcomes are required to assess incremental benefit of biomarker-based approaches. Multiple testing (biomarkers, end points, subgroups, cutoff values, and models) should be transparent and ideally prespecified. Correction for multiple testing is important to minimize false-positives, acknowledging some risk for type II error. Model overfitting can lead to spurious associations rather than true biological signals. Association, discrimination, prediction, and accuracy represent different aspects of biomarker performance.2

Is There Validation?

Replication of results in a separate, representative at-risk population is ideal.

The study by Willeit and colleagues3 nicely illustrates some of these practices and others detailed in professional society recommendations.4 The authors used separate derivation and validation cohorts (but the same biomarker assay) and included 2 well-established, high dynamic-range markers in their models (N-terminal pro-B-type natriuretic peptide and high-sensitivity cardiac troponin T). After rigorous adjustment for covariates and multiple comparisons, only soluble vascular cell adhesion molecule 1 was associated with incident atrial fibrillation in derivation and validation. Notably, associations of markers of systemic inflammation, which have previously been associated with atrial fibrillation, were negative, which is an important message for the field. As early identification and targeted screening of atrial fibrillation continues to emerge as a public health priority, carefully conducted biomarker studies have the potential to aid us in achieving this goal.

Back to top
Article Information

Conflict of Interest Disclosures: Both authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr Sabatine has received grants and/or personal fees from Abbott Laboratories, Amgen, AstraZeneca, Critical Diagnostics, Daiichi-Sankyo, Eisai, GlaxoSmithKline, Intarcia, Merck, Roche Diagnostics, Takeda, Gilead, CVS Caremark, Poxel, Alnylam, Novartis, MedImmune, Ionis, Janssen Research Development, Genzyme, Cubist, Esperion, Medicines Company, MyoKardia, and Zeus Scientific. Dr Turakhia has received grants and/or personal fees from Medtronic, AstraZeneca, the Veterans Health Administration, AliveCor, St Jude Medical, Boehringer Ingelheim, Precision Health Economics, and Metrica Health.

Disclaimer: The content and opinions expressed are solely the responsibility of the authors and do not necessarily represent the views or policies of the Department of Veterans Affairs.

Ioannidis  JPA, Panagiotou  OA.  Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent meta-analyses.  JAMA. 2011;305(21):2200-2210.PubMedGoogle ScholarCrossref
Cook  NR.  Use and misuse of the receiver operating characteristic curve in risk prediction.  Circulation. 2007;115(7):928-935.PubMedGoogle ScholarCrossref
Willeit  K, Pechlaner  R, Willeit  P,  et al.  Association between vascular cell adhesion molecule 1 and atrial fibrillation [published online March 29, 2017].  JAMA Cardiol. doi:10.1001/jamacardio.2017.0064Google Scholar
Hlatky  MA, Greenland  P, Arnett  DK,  et al; American Heart Association Expert Panel on Subclinical Atherosclerotic Diseases and Emerging Risk Factors and the Stroke Council.  Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association [published correction appears in Circulation. 2009;119(25):e606].  Circulation. 2009;119(17):2408-2416.PubMedGoogle ScholarCrossref