Accuracy and Methodologic Challenges of Volatile Organic Compound–Based Exhaled Breath Tests for Cancer Diagnosis: A Systematic Review and Meta-analysis

Key Points

Question  What are the diagnostic accuracy and methodologic challenges of volatile organic compound (VOC)–based exhaled breath tests for the detection of cancer?

Findings  This systematic review and meta-analysis identified 63 relevant publications from which pooled analysis of outcomes found a sensitivity of 79% and specificity of 89% for the detection of cancer using VOC breath tests. Breath collection method, patient physiologic condition, test environment, and method of analysis may influence VOC test results.

Meaning  VOC breath tests may have potential for noninvasive cancer diagnosis, but methodologic standardization may be required.

Importance  The detection and quantification of volatile organic compounds (VOCs) within exhaled breath have evolved gradually for the diagnosis of cancer. The overall diagnostic accuracy of proposed tests remains unknown.

Objectives  To determine the diagnostic accuracy of VOC breath tests for the detection of cancer and to review sources of methodologic variability.

Data Sources  An electronic search (title and abstract) was performed using the Embase and MEDLINE databases (January 1, 2000, to May 28, 2017) through the OVID platform. The search terms cancer, neoplasm, malignancy, volatile organic compound, VOC, breath, and exhaled were used in combination with the Boolean operators AND and OR. A separate MEDLINE search that used the search terms breath AND methodology was also performed for studies that reported factors that influenced the concentration of VOCs within exhaled breath in humans.

Study Selection  The search was limited to human studies published in the English language. Trials that analyzed named endogenous VOCs within exhaled breath to diagnose or assess cancer were included in this review.

Data Extraction and Synthesis  Systematic review and pooled analysis were conducted in accordance with the recommendations of the Cochrane Library and Meta-analysis of Observational Studies in Epidemiology guidelines. Bivariate meta-analyses were performed to generate pooled point estimates of the hierarchal summary receiver operating characteristic curve of breath VOC analysis. Included studies were assessed according to the Standards for Reporting of Diagnostic Accuracy Studies checklist and Quality Assessment of Diagnostic Accuracy Studies 2 tool.

Main Outcomes and Measures  The principal outcome measure was pooled diagnostic accuracy of published VOC breath tests for cancer.

Results  The review identified 63 relevant publications and 3554 patients. All reports constituted phase 1 biomarker studies. Pooled analysis of findings found a mean (SE) area under the receiver operating characteristic analysis curve of 0.94 (0.01), sensitivity of 79% (95% CI, 77%-81%), and specificity of 89% (95% CI, 88%-90%). Factors that may influence variability in test results included breath collection method, patient physiologic condition, test environment, and method of analysis.

Conclusions and Relevance  The findings of our review suggest that standardization of breath collection methods and masked validation of breath test accuracy for cancer diagnosis is needed among the intended population in multicenter clinical trials. We propose a framework to guide the conduct of future breath tests in cancer studies.

The use of exhaled breath analysis for disease diagnosis and therapeutic monitoring provides an attractive option for patients and practitioners because it is noninvasive with the potential to be conducted at the point of care. This approach is based on the detection and quantification of volatile organic compounds (VOCs) in exhaled breath. A VOC is a carbon-containing compound that is sufficiently volatile to be detectable in the gas phase at room temperature. The US Environmental Protection Agency sets a vapor pressure lower than 0.1 mm Hg as discriminant for volatility.¹ Established uses of VOC measurements include the assessment of environmental contamination, flavor and fragrance industry, and counterterrorism. Examples of the analysis of exhaled VOCs within clinical practice include breathalyzer devices for ethanol detection, carbon 13 urea breath testing for Helicobacter pylori, exhaled nitric oxide in asthma, and hydrogen-methane testing for small-bowel bacteria overgrowth.^2-5 Breath testing has the potential to address a central diagnostic dilemma in cancer. The presence of red flag symptoms, which are often the initiating factor for investigation of underlying malignancy, typically occur late in the disease process and are associated with poor prognosis. Within a population there can also be a high prevalence of nonspecific symptoms relative to the prevalence of cancer, but these symptoms may be the manifestation of an early-stage cancer. There is therefore an unmet need for a noninvasive test to triage patients who may benefit from subsequent invasive investigations or advanced imaging.

In 1971, the double Nobel Laureate Linus Pauling reported the quantification of 250 volatiles in a breath sample using gas chromatography (GC).⁶ Gas chromatography followed the invention of partition chromatography by Martin and Synge (Nobel Prize in chemistry, 1952), whereas mass spectrometry (MS) was first introduced by Aston (Nobel Prize in chemistry, 1922).⁷ The GC-MS combination offers the possibility to separate and identify the individual constituents of a gaseous sample, but to be quantitative, the technique requires calibration; therefore, target compounds need to be commercially available in pure form or they must be synthesized. Direct injection analytical techniques developed for atmospheric chemistry, such as proton transfer reaction (PTR)–MS and selected ion flow tube (SIFT) MS, have been in use since the 1990s for VOC analysis in exhaled breath. With these techniques, MS detection is performed directly on the ionized sample and therefore allows quantification with no prior calibration. In ion mobility spectrometry, the ionized sample interacts with a buffer gas and one or more electric fields to achieve a separation of the analytes. Ion mobility spectrometry can work as a standalone instrument or in combination with mass analyzers. The silicon chip MS has been developed for the selective detection and quantification of carbonyl compounds (aldehydes and ketones).⁸ Recent advances include the application of 2-dimensional (2-D) GC (GC-GC) and 2-D MS (MS-MS) in combination with fast MS detectors, such as time of flight, which enhance resolution for more precise identification and quantification of VOCs. These advances resulted in the detection of VOCs in exhaled breath at parts per billion or parts per trillion levels and have encouraged scientists and practitioners to investigate breath tests for disease diagnosis and therapeutic monitoring.

The specific aims of this review were to (1) study diagnostic accuracy of VOC-based exhaled breath tests for cancer, (2) examine methodologic challenges and mitigating strategies for the use of breath testing in cancer diagnosis, and (3) create a framework for conducting and reporting future studies on the role of VOCs in cancer diagnosis. To achieve these aims, we performed a systematic appraisal of the published evidence regarding the use of exhaled VOCs for cancer diagnosis. The review included only studies that identified named VOCs altered within the exhaled breath of patients with cancer. Studies that involved sensors and pattern recognition techniques that did not report on specific VOCs were excluded. We also conducted a narrative review of methodologic factors that are associated with variation in exhaled VOCs. These findings were used in a framework on which to base future trials.

This systematic review and pooled analysis were conducted in accordance with the recommendations of the Cochrane Library and Meta-analysis of Observational Studies in Epidemiology guidelines.⁹ An electronic search (title and abstract) was performed using the Embase and MEDLINE databases (January 1, 2000, to May 28, 2017) through the OVID platform. The search terms cancer, neoplasm, malignancy, volatile organic compound, VOC, breath, and exhaled were used in combination with the Boolean operators AND and OR. The search was limited to human studies published in the English language. Conference abstracts were excluded. Full details of the search strategy are provided in eTable 1 in the Supplement.

Two reviewers (G.B.H. and P.R.B.) screened the titles and abstracts of studies identified through the electronic search, and the full texts of potentially relevant articles were retrieved to determine eligibility for inclusion in the review. Studies were included if they met the following criteria: (1) trials that analyzed endogenous VOCs within exhaled breath to diagnose or assess cancer and (2) studies that included VOC identification. Studies were excluded if (1) an exogenous substrate (intravenous, oral, or inhaled) was administered before exhaled breath sampling, (2) sensor-based pattern recognition technology was used in the absence of VOC identification, and (3) VOCs were analyzed not in exhaled breath but in breath condensate or other biofluid, including urine, serum, feces, and gastric content. The reference lists of included studies were hand searched to identify additional relevant studies.

For each study, 2 reviewers (P.R.B. and S.R.M.) independently extracted data related to discriminative VOCs, analytical platform, the biomarker discovery phase of the study, and sensitivity, specificity, and the area under the receiver operating characteristic (ROC) curves derived from diagnostic models. The VOCs were later grouped according to their chemical class. Areas of disagreement were resolved by a third reviewer (G.B.H.).

In accordance with the validated methods of Harbord et al,¹⁰ bivariate meta-analyses were performed to generate pooled point estimates of the hierarchal summary ROC curve of breath VOC analysis. The software used for this analysis was the custom-designed statistical package MIDAS.¹¹ An area under the hierarchal summary ROC curve was obtained directly from the MIDAS output.

The quality of reporting in each of the diagnostic studies identified was examined using the Standards for Reporting of Diagnostic Accuracy Studies (STARD) checklist.¹² STARD reflects the completeness and transparency of reporting within diagnostic accuracy studies. The checklist contains 30 items based on evidence that linked these items to bias, variability, and limitations of the applicability of results to other settings.

We also used a modified Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) tool to assess the quality of the included studies.¹³ QUADAS-2 consists of 4 domains, including patient selection, index test, reference standard, and flow of patients through the study. In this review, the reference standard was considered to be histologic or radiologic confirmation of cancer. The QUADAS-2 tool was specifically modified to have greater relevance to phase 1 biomarker discovery studies; specific changes included importance given to the following: the inclusion of benign conditions (positive controls), internal and/or external validation of results, assessment before therapeutic intervention, and suitable reproducibility and sensitivity of the chosen index test (full details of the modified QUADAS-2 tool are presented in eTables and 3 and eFigure 1 in the Supplement). The modified tool was piloted independently by 3 reviewers (G.B.H., P.R.B., and S.R.M.) on 4 studies and was subsequently used independently by 2 assessors (P.R.B. and S.R.M.) to rate all included studies.

A MEDLINE search (all fields) using the keywords breath AND methodology was performed. Title and abstracts of identified articles were screened by a single reviewer (P.R.B.) for studies that reported factors that influenced the concentration of VOCs within exhaled breath in humans (healthy or with disease). Additional relevant studies were identified through hand searching of the reference lists of identified articles. Exclusion criteria were as follows: studies that reported only the effects of known disease processes, reports not published in the English language, conference abstracts, animal or in vitro experiments, and studies that reported the outcomes of mathematical models without presentation of clinical data.

Study findings were categorized by 2 reviewers (P.R.B. and G.B.H.) according to principal factors that influence exhaled VOC levels. Factors were grouped under the following domains: (1) patient-related physiologic factors, (2) sampling methods, (3) environmental considerations, and (4) instrument-specific sources of variability. Factors that influence VOC levels within each domain were described, and the consequences of the presence of such factors were examined. Mitigating strategies to overcome challenges in the methods of breath collection and analysis were considered.

The literature search yielded 63 publications^14-76 that met the inclusion criteria, with 1 study¹⁷ providing data for 4 cancer sites (eFigure 2 in the Supplement). All reports constituted phase 1 biomarker discovery studies. The number of patients with cancer included in those studies ranged from 5 to 220. Within included studies, lung (n = 39), breast (n = 7), gastroesophageal (n = 6), colorectal (n = 4), oral cavity (n = 2), head and neck (n = 2), mesothelioma (n = 1), thyroid (n = 1), liver (n = 1), ovarian (n = 1), prostate (n = 1), and laryngeal (n = 1) cancers were studied (Table and eTable 4 in the Supplement). Most included studies compared patients with cancer (of often mixed histologic subtype) with a healthy control population and/or patients with benign conditions that affected the same organ (Table). Studies tended to include patients with early and advanced tumor stages, although tumor stage was not reported in 21 studies. The most commonly used analytical platform was GC-MS (n = 49 studies). A total of 253 VOCs were reported in association with cancer diagnosis in different tumor sites. For the most part, these VOCs were hydrocarbons (aromatic and aliphatic) and oxygenated compounds (namely, aldehydes, alcohols, phenols, carboxylic acids, ethers, and furans) and less frequently nitrogen-, sulfur- and halogen-containing compounds. The most common VOCs associated with cancer were 2-butanone (n = 14 studies), 1-propanol (n = 8), nonanal (n = 8), isoprene (n = 8), ethylbenzene (n = 8), 4-methyl octane (n = 8), 3-hydroxy-2-butanone (n = 8), acetone (n = 7), toluene (n = 7), ethanol (n = 7), pentanal (n = 7), heptanal (n = 7), and pentane (n = 7). Full details of cancer VOCs, including their chemical classes, are provided in eTables 4 and 5 and eFigure 3 in the Supplement.

The sensitivity of breath testing for cancer diagnosis ranged from 28% to 100%, whereas the specificity ranged from 61% to 100%. Twenty-eight publications that evaluated all types of cancer were included in the pooled analysis. Summary ROC analysis confirmed the ability of VOC analysis of exhaled breath in distinguishing patients with cancer from individuals without cancer, with a mean (SE) area under the curve of 0.94 (0.01) (Figure 1). Pooled sensitivity was 79% (95% CI, 77%-81%), and pooled specificity was 89% (95% CI, 88%-90%) (Figure 2 and Figure 3).

The STARD score for reported cancer studies ranged from 5 to 22, with a mean (SD) of 13.7 (4.3). The details of STARD for each study are provided in eTable 6 in the Supplement. Assessment of biases and applicability of outcomes using QUADAS-2 are detailed in eTable 3 and eFigure 1 in the Supplement. Major sources of bias were (1) the absence of a positive control group, (2) the absence of a validation set for the index test, and (3) failing to report the reference standard test or acknowledge the interval between index and reference tests. There were no significant applicability concerns for patient selection, index test, and reference standard.

A total of 574 abstracts were screened, and a total of 87 relevant articles were identified. Complete references for these 87 studies are provided in eTable 7 in the Supplement. Patient-related factors were described in 55 studies, sampling methods in 29, environmental considerations in 24, and instrument source of variability in 9 (eTable 7 in the Supplement). Most studies used MS-based techniques, principally PTR-MS (n = 25), SIFT-MS (n = 24), and GC-MS (n = 27). Abundant compounds were the most commonly examined VOCs: acetone (n = 52 studies), isoprene (n = 48), ethanol (n = 22), methanol (n = 20), and ammonia (n = 18). Therefore, it remains unclear to what extent sampling and patient-specific factors affect trace VOCs associated with cancer.

The association of sampling and patient-specific factors with the variability of VOC analysis in exhaled breath has attracted the greatest interest in methodologic development. The influence of environmental factors, specifically inhaled ambient air, was frequently examined within methodologic studies. Another crucial topic was the assessment of the ability of analytical instruments to support breath testing beyond research laboratories within clinical practice and multicenter clinical trials. Mass spectrometry has been the standard method for the detection of VOCs within exhaled breath, although a range of analytical techniques have been recently used. Based on the literature, when a specific method is optimized for the analysis of chosen compounds, optimum sensitivity and instrument-specific reproducibility can be achieved. However, intrainstrument and interinstrument variability influences the reproducibility of the results. For instance, intrapatient variability was 1% to 19% using SIFT-MS and 5% to 29% using PTR-MS. The coefficient of variance for the reproducibility of PTR-MS measurements of a standard mixture of 7 VOCs was reported to be 10% to 19%.⁷⁷

A framework (Box) was created based on factors that influenced the results of breath sampling and analysis in the methods review and from the lessons learned from the assessment of the quality of reporting within diagnostic studies using the STARD checklist and QUADAS-2. We have had experience with the value and influence of the factors included in the framework during our own research in the past decade.

Box Section Ref ID

The analysis of VOCs within exhaled breath provides a promising approach to the diagnosis of multiple tumor types. Phase 1 biomarker studies demonstrated the ability of breath VOC analysis for cancer diagnosis with a mean (SE) area under the ROC curve of 0.94 (0.01), a pooled sensitivity of 79% (95% CI, 77%-81%), and a pooled specificity of 89% (95% CI, 88%-90%). The noninvasive nature of breath testing offers an additional advantage that it enhances patient acceptability. Nevertheless, for this promising approach to reach its potential in clinical practice, several milestones must be achieved.

First, the standardization of breath collection and analysis is crucial. Our methodologic review revealed that the results of breath testing depend on several factors related to the method of sample collection, patients’ physiologic condition, and test environment. Lessons learned from the study of exhaled nitric oxide as a biomarker for pulmonary inflammation should inform current developments. A defining moment in the effort to develop nitric oxide as a clinically applicable breath biomarker was the recognition that multiple respiratory factors influenced its concentration within breath, leading to the publication of international recommendations for its standardized measurement (American Thoracic Society and European Respiratory Society, 2005),⁷⁸ which have in turn supported its adoption as a diagnostic method. The framework proposed herein is an initial step toward achieving this vision.

Second, the instrumentation used for breath analysis requires careful consideration. Mass spectrometry used in VOC analysis is a standard technique that is widely used in pharmacology, toxicology, and atmospheric chemistry. The novelty of using MS-based technologies in VOC analysis should therefore not be misperceived as the instrumentation being in the development stage. However, to obtain accurate results in clinical practice and gain the confidence of practitioners and cancer scientists, the reliability of instruments and reproducibility of results should be tested, optimized, and reported for exhaled VOC analysis in clinical studies.

Third, external validation experiments against positive control groups and multicenter studies among the target population within the environment where the breath test will be ultimately used are essential steps to confirm internal and external validity and inform clinical applications. Established test thresholds for separating patients with cancers of different stage and tumor subtypes from controls is needed before embarking on masked validation studies. Accurate clinical reporting based on STARD and CONSORT guidance is an important consideration, as shown in the assessment of the quality of reporting of studies included in our systematic review.

Our review focused on the analytical techniques capable of VOC identification. A total of 253 VOCs were reported, and studies typically used multiple VOCs to construct diagnostic models for different cancer sites. Most publications used MS-based technology, which suits the discovery phase of disease-profiling studies. The refinement of diagnostic models so that they use a reduced number of VOCs is desirable and will stimulate the development of bespoke technologies and point-of-care diagnostics directed toward target volatile biomarkers. However, there has been a paucity of studies designed to investigate the molecular mechanisms and exhalation kinetics underpinning the production of volatile biomarkers in cancer. Such studies are needed to further understand the factors that influence test performance, refine the diagnostic model, and increase confidence in breath analysis for cancer diagnosis.

There are several potential locations of breath tests in the patient care pathway. First, for patients who present with nonspecific common symptoms that could be an early indication of cancer, an exhaled breath test could act as a triage investigation to direct patients to have invasive or more specialized investigations. Tissue diagnosis will nevertheless remain an essential requirement before starting cancer therapy or surgical resection of affected organs. As shown in this systematic review, discriminative VOCs in the diagnostic models vary among tumor sites; thus, an exhaled breath test from a single breath collection may be suitable for use as a triage tool to decide which patients require referral for suspected cancers. The use of a breath test as a triage investigation could increase the proportion of appropriate referrals from primary care and improve the application national guidelines. For example, if the general practitioner was assessing a patient with gastrointestinal symptoms that do not meet the guidelines for prompt referral, they would not need to watch and wait to see if symptoms worsen but could offer the test immediately. The general practitioner would order a breath test in the same way as routine blood tests, with a single breath collection to assess for individual or multiple cancers. A nurse can perform the test and send breath samples to a regional laboratory for analysis. A positive result would warrant immediate referral. A negative test result would permit the general practitioner to reassure the patient and offer retesting if symptoms persist.

Second, if breath tests have acceptable diagnostic accuracy, they may have a role in screening programs in which the noninvasive nature of the test and its acceptability by patients and practitioners would increase the uptake of screening. Large, multicenter clinical studies among target populations with appropriate sample sizes would be crucial before considering such an approach.

Third, breath tests may have a role in monitoring the response to cancer therapy. There is a potential for using breath tests to detect disease recurrence, as we have found in our investigations in detecting colorectal recurrence after surgical resection.⁷⁹ Current tests for monitoring precision treatment response may be inadequate because invasive serial tissue sampling is not generally acceptable to patients and radiologic assessment is insensitive to biological evolution; therefore, breath testing may provide a solution. However, most studies identified by this systematic review included patients with early and advanced cancer. Although it is widely hypothesized that breath testing may fulfill a role in the detection of early cancers, less than half of identified studies (n = 22) explored the effect of stage on cancer breath test performance and these studies often had inconsistent findings.^{21,23,25,27,32,36,37,39,43,44,48,53,58,60-62,66,68,71,73-75} Heterogeneity in reporting meant that it was not possible to undertake further subgroup analysis of such studies herein. Future studies should seek to determine the precise role of tumor stage and other factors, including the association of histologic and molecular subtype with the diagnostic accuracy of proposed breath tests.¹⁴

Implementation of breath analysis on a wider scale in clinical practice requires careful consideration. The first option is a central or regional laboratory model in which breath samples are collected in general practice or hospital clinics and sent to a laboratory for analysis in a similar way to blood investigations. The design of breath collection devices and the methods for transporting breath samples are crucial to the success of such an approach. This laboratory-based model permits the application of quality assurance methods to guarantee instrument reliability and optimize breath analysis and thus ensure robust results.

The second option for clinical implementation is the development of point-of-care devices to have the results available to practitioners when seeing patients for immediate action. Human factors and ergonomic principles should specify the design requirement for point-of-care breath analyzers. There is a need for those devices to be able to use different VOC diagnostic models to detect various cancers because a device for each cancer site is not a practical option. Cost-effectiveness studies are required to examine both options. Test accuracy and the location within the patient care diagnostic pathway will inform economic studies, which in turn will determine the optimum approach. Although there is a need for the accuracy of breath tests to be confirmed in large-scale, multicenter clinical trials among the intended population before the introduction into clinical practice, models for implementation should be considered at an early stage to direct future research and clinical studies.

This systematic review does not include studies that used sensor and pattern recognition technologies that did not report volatile biomarkers. A finding as well as a limitation of our review is the lack of external validation studies. There was also a paucity of studies that addressed health economics in different clinical settings and disease conditions.

Standardization of breath collection methods and masked validation of breath test accuracy for cancer diagnosis is needed among the intended population in multicenter clinical trials. The proposed framework could guide the conduction of future breath tests in cancer studies.

Accepted for Publication: May 8, 2018.

Published Online: August 16, 2018. doi:10.1001/jamaoncol.2018.2815

Correction: This article was corrected on May 23, 2019, to fix errors in Figure 3.

Corresponding Author: George B. Hanna, FRCS, PhD, Department of Surgery and Cancer, Imperial College London, 10th Floor Queen Elizabeth the Queen Mother Wing, South Wharf Road, London W2 1NY, United Kingdom (g.hanna@imperial.ac.uk).

Author Contributions: Drs Hanna and Boshier had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Hanna, Boshier.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: Hanna, Boshier.

Statistical analysis: Hanna, Markar.

Administrative, technical, or material support: Hanna, Boshier, Romano.

Supervision: Hanna.

Conflict of Interest Disclosures: None reported.