False-Positive Results in Rapid Antigen Tests for SARS-CoV-2

Joshua S. Gans; Avi Goldfarb; Ajay K. Agrawal; Sonia Sennik; Janice Stein; Laura Rosella

doi:10.1001/jama.2021.24355

Comment

Kretschmer A, Kossow A, Grüne B, Schildgen O, Mathes T, Schildgen V. False positive rapid antigen tests for SARS-CoV-2 in the real-world and their economic burden. J Infect. Published online August 17, 2021. doi:10.1016/j.jinf.2021.08.020 PubMed Google Scholar

Ogawa T, Fukumori T, Nishihara Y, et al. Another false-positive problem for a SARS-CoV-2 antigen test in Japan. J Clin Virol. 2020;131:104612. doi:10.1016/j.jcv.2020.104612 PubMed Google Scholar

Kanji JN, Proctor DT, Stokes W, et al. Multicenter postimplementation assessment of the positive predictive value of SARS-CoV-2 antigen-based point-of-care tests used for screening of asymptomatic continuing care staff. J Clin Microbiol. 2021;59(11):e0141121. doi:10.1128/JCM.01411-21 PubMed Google Scholar

Select Your Interests

Save Preferences

Others Also Liked

11 Comments for this article

EXPAND ALL

January 7, 2022

Disagree

Al Musella, DPM | Musella Foundation For Brain Tumor Research & Information, Inc

The conclusion of this Research Letter is that there aren’t many false positives.

The relevant numbers are 462 rapid screens with false-positive results, or 42% of those with reference standard PCR information. Why bother with a test that is not so different from flipping a coin?

The research effort did not test for false negatives so we do not know if that vast majority of negatives are true or false.

CONFLICT OF INTEREST: None Reported

January 8, 2022

Statistics 101

Kurtis Elward, Clinical Professor | Family Medicine of Belvedere

I agree with the previous comment about the error in interpretation.

The question that the authors targeted - "false positives" - does indeed indicate that the overall administrative burden of false positives will be low. For lab directors this will be reassuring. For people interested in controlling the disease spread it will not be very valuable.

This is because the authors did not report the false negatives, which were not able to be considered due to the circumstances in which the study was done. Nonetheless, rapid tests are known for high false negative rates, perhaps the more concerning limitation, given that they are used to "clear" persons for return to work, school, or clinical practice.

Without looking at false negatives, the value of the test as a screening mechanism is unknown. It would have been helpful if the Editors made sure this caveat was put into the Discussion, the most disappointing feature of this article.

CONFLICT OF INTEREST: None Reported

January 9, 2022

Consider the Gold Standard

Peter English, MB, ChB, MPH | Public Health Physician, Consultant in Communicable Disease Control

It is incorrect to refer to all positive rapid antigen test screens in which a PCR test, taken at about the same time, was negative, as "false positive" results.

The PCR test is, in itself, far from 100% sensitive - probably 90% at best in the real world (depending on timing and sample-collection technique).

This means that, even if 100% of rapid antigen test results were true positive results, a proportion (up to 10%) would, in such an analysis, be incorrectly described as "false positive" results.

CONFLICT OF INTEREST: None Reported

January 11, 2022

Positive Predictive Value depends upon prevalence of disease in community

Brian Doyle | Emergency Physician, FACEM FACEP

There were only 0.15% positive results in this sample. Therefore it should come as no surprise that there was a high proportion of false positive tests.

To put a bit more simply, if there is no community prevalence of disease, than all positive tests are false positives.

However, if there is a high burden of disease (i.e. COVID outbreak) than it is much more likely that a positive test represents a true positive.

The other limitation is the use of PCR as the gold standard. It is also possible that the rapid test was actually correct (positive) but the PCR a false negative.

Regardless, the major issue with the rapid antigen tests is not the false positive rate, but rather the false negatives (i.e. relatively poor sensitivity)

CONFLICT OF INTEREST: None Reported

January 18, 2022

Quality Control?

Kenneth Gorelick, MD | Zymo Consulting Group

This report suggests that a product from one of the most respected manufacturers of diagnostic tests was the source of so many false-positive results. This indicates a failure in either quality control or product design (less likely as all false-positive results were attributed to a single batch of product).

It would be good to hear a response from Abbott, the manufacturer of the test, or the regulatory authorities (HealthCanada or FDA). These products are released under emergency use authorizations (EUAs) that are less stringent than actual approvals. This problem should be fixed or the EUA revoked. The pandemic is far too serious to allow misleading or faulty tests to be distributed.

CONFLICT OF INTEREST: I consult for the biopharma industry, although I have never consulted in the field of SARS-CoV-2 testing

January 20, 2022

Role of Rapid Antigen Detection Test (RADT) for Detection of SARS Cov-2 Variants

Atul Garg, MD, DNB | Associate Professor, Virology section, Department of Microbiology, Sanjay Gandhi Post Graduate Institute of Medical Sciences, Lucknow, India.

This study reports that 1103 health care workers were RADT positive and that corresponding RTPCR was positive only in 641 (58%) cases. The authors assume that all RADT-positive/RTPCR-negative samples are false positives (42%).

An important technical point missing from the discussion is that RADTs are mainly targeted at detecting the viral nucleocapsid protein coded by the N gene which in the Delta/ Omicron variant appears less divergent than the S gene expressing spike protein. As a majority of Delta variant (B.1.617.2) mutations occur in S gene and other open reading frames (ORF), there are high chances that detection of these variants is missed by RTPCR targeting S/ORF genes, making RTPCR and less accurate reference standard. Preliminary results of the rapid assessment conducted by UK’s NHS and Foundation for Innovative New Diagnostics (FIND) suggest that the accuracy of RADTs has not been impacted by the emergence of the Delta or Omicron variant.

Variants keep on emerging during an pandemic especially in immunocompromised patients and those variants not fit for long term survival are eliminated by process of natural selection. Since all RADT-positive / RTPCR-negative samples were reported from 2 centers, it would be interesting to study the S and N gene mutations by analyzing the whole genome sequencing data of SARS CoV-2 strains from these geographical areas during the study period.

CONFLICT OF INTEREST: None Reported

January 21, 2022

PCR Test Characteristics

George Alakpa, PhD, Dsc, MPH | Private Research

At what number of cycles (C_t) was the PCR reference standard considered positive or negative? That definition could affect it accuracy (sensitivity, specificity, predictive values). Also, was PCR confirmation done at a different test facility?

CONFLICT OF INTEREST: None Reported

January 22, 2022

Cohort Status

David States, MD, PhD | Angstrom Bio, Inc

The study reports that among 903,408 biweekly rapid antigen tests performed over a 39-week period between January 11, 2021 and October 13, 2021, 1,322 were positive. If 58% were confirmed there were 767 screen-detected cases of infection (6.6% of the study cohort). During this period Canada reported roughly 1.7 million confirmed cases of COVID in a population of 38 million (4.5% of the population).

Was the use of a rapid antigen test screening program actually associated with an increase in COVID infections among employees of participating employers (perhaps due to false confidence in receiving a negative test)?
/> How many of the documented cases of COVID among employees were detected in the screening program, i.e. what was the false negative rate for screening?

CONFLICT OF INTEREST: Chief Medical and Chief Science Officer for Angstrom Bio, Inc, a company engaged in COVID testing

January 24, 2022

Test Accuracy at Low Temperatures

Brian Tefft | Private Researcher

In a study published in the Journal of Clinical Virology, Haage et al. reported that several commercially available rapid antigen tests for SARS-CoV-2 suffer reduced sensitivity when stored or used at temperatures above those recommended by the manufacturer, and most relevant to the current study, they also found that two--including Abbott's Panbio--were found to suffer reduced specificity when stored and used at temperatures colder than recommended by the manufacturer (1).

Given that most of the false positives in the current study were from 2 workplaces using the same batch of Abbott Panbio tests, it is possible the false positives observed were attributable to manufacturing issues, as suggested by the authors. However, the results reported by Haage et al. remind us that it is important to ensure that tests are stored and used within the temperature range specified by the manufacturer. This might be a greater concern when the tests are administered outside of clinical settings, eg if members of the general public store and use their test in their car in the winter.

Reference

1. Haage, V. et al. (2021). Impaired performance of SARS-CoV-2 antigen-detecting rapid diagnostic tests at elevated and low temperatures. J. Clin Virol, 138, 104796. https://doi.org/10.1016/j.jcv.2021.104796

CONFLICT OF INTEREST: None Reported

January 27, 2022

Too Quick to Blame Manufacturer?

William Shang, MD | Cornell University

Licensed laboratories test validate new batches or lots prior to bringing them into service. The authors missed the opportunity to mention how this standard practice could have prevented this supposed occurrence of false positives.

Also, most lab directors will consider other factors such as transit conditions, on-site storage as well as the competency of those administering and interpreting the lab results. Did they report to Abbott and allow the manufacturer to see if other sites/labs reported anomalies with this batch?

Many opportunities for quality improvement exist and don't solely reside with the manufacturer.

CONFLICT OF INTEREST: None Reported

October 21, 2023

Explaining False Positives

Ralf Kaiser, MD, MHBA | Hospital of Püttlingen, Saar (Germany)

The discussion reads "false-positives are possible due to the timing of the test (ie, too early or too late in the infectious stage) or quality issues in how the self-test was completed." That might explain false-negatives, but I don´t see how this is related to false-positives.

CONFLICT OF INTEREST: None Reported

Research Letter

January 7, 2022

False-Positive Results in Rapid Antigen Tests for SARS-CoV-2

Joshua S. Gans, PhD¹; Avi Goldfarb, PhD¹; Ajay K. Agrawal, PhD¹; et al Sonia Sennik, MBA²; Janice Stein, PhD¹; Laura Rosella, PhD¹

Author Affiliations Article Information

¹University of Toronto, Toronto, Ontario, Canada
²Creative Destruction Lab, Toronto, Ontario, Canada

JAMA. 2022;327(5):485-486. doi:10.1001/jama.2021.24355

Concerns have been raised whether rapid antigen tests for SARS-CoV-2 can result in false-positive test results¹^,2 and undermine pandemic management for COVID-19. This study investigated the incidence of false-positive results in a large sample of rapid antigen tests used to serially screen asymptomatic workers throughout Canada.

Methods

Rapid antigen tests for SARS-CoV-2 were implemented as an extra layer of protection to control transmission in workplaces throughout Canada by the Creative Destruction Lab Rapid Screening Consortium (CDL RSC). Asymptomatic employees were screened twice weekly. Workplace participation was voluntary. From January 11 to October 13, 2021, tests were conducted by employees, with some workplaces providing at-home screening and others on-site screening programs. Over this period, Canada experienced 2 significant Delta variant–driven waves from March to June and August to October. Screening results were recorded, including a deidentified record identifier, the place of employment, the test, and (optionally) the lot number. If a test result was positive, the patient was immediately referred for a confirmatory polymerase chain reaction (PCR) test to be completed within 24 hours. Initial data validation was completed at the point of collection. All data collected before June 26 and presumptive positive screen results and PCR test results reported before September 15 were externally verified through an audit process by participant organizations. False-positive results were matched to lot number and test manufacturer. A false-positive result was defined as a positive screen on a rapid antigen test and a subsequent negative confirmatory PCR.

The data from the CDL RSC were collected to inform the operational requirements of deploying rapid antigen screens in workplaces. All participants provided written consent to participate in the screening program and to share their deidentified data with the CDL RSC, including for publication, and with public health authorities. This study was approved by the University of Toronto Research Ethics Board.

Results

There were 903 408 rapid antigen tests conducted for 537 workplaces, with 1322 positive results (0.15%), of which 1103 had PCR information. Approximately two-thirds of screens were trackable with a lot number. The number of false-positive results was 462 (0.05% of screens and 42% of positive test results with PCR information). Of these, 278 false-positive results (60%) occurred in 2 workplaces 675 km apart run by different companies between September 25 and October 8, 2021. All of the false-positive test results from these 2 workplaces were drawn from a single batch of Abbott’s Panbio COVID-19 Ag Rapid Test Device.

Discussion

The overall rate of false-positive results among the total rapid antigen test screens for SARS-CoV-2 was very low, consistent with other, smaller studies.³ The cluster of false-positive results from 1 batch was likely the result of manufacturing issues rather than implementation. These results inform the discussion of whether rapid antigen tests will result in too many false-positives that could overwhelm PCR testing capacity in other settings.¹^,2 Also, the results demonstrate the importance of having a comprehensive data system to quickly identify potential issues. With the ability to identify batch issues within 24 hours, workers could return to work, problematic test batches could be discarded, and the public health authorities and manufacturer could be informed. Aside from issues with the batch, false-positives are possible due to the timing of the test (ie, too early or too late in the infectious stage) or quality issues in how the self-test was completed.

Limitations of the study include the convenience sample of workplaces and that reporting of PCR confirmatory results and identification of lot number was not compulsory. In addition, these results reflect the epidemiology experienced in Canada and may not generalize to other countries experiencing different COVID-19 incidence.

Section Editors: Jody W. Zylke, MD, Deputy Editor; Kristin Walter, MD, Associate Editor.

Article Information

Accepted for Publication: December 20, 2021.

Published Online: January 7, 2022. doi:10.1001/jama.2021.24355

Corresponding Author: Joshua S. Gans, PhD, Rotman School of Management, University of Toronto, 105 St George St, Toronto, ON M5S3E6, Canada (joshua.gans@utoronto.ca).

Author Contributions: Drs Goldfarb and Rosella had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Goldfarb, Agrawal, Sennik, Stein, Rosella.

Acquisition, analysis, or interpretation of data: Gans, Goldfarb, Agrawal, Sennik, Rosella.

Drafting of the manuscript: Gans, Goldfarb.

Critical revision of the manuscript for important intellectual content: Goldfarb, Agrawal, Sennik, Stein, Rosella.

Statistical analysis: Goldfarb, Rosella.

Obtained funding: Agrawal, Sennik, Stein.

Administrative, technical, or material support: Gans, Goldfarb, Agrawal, Sennik.

Supervision: Agrawal, Sennik.

Conflict of Interest Disclosures: All authors are members of the Steering Committee of the Creative Destruction Lab Rapid Screening Consortium (CDL RSC; a nonprofit organization in Canada). Dr Agrawal reported serving on the boards of Genpact and Sanctuary.

Funding/Support: The CDL RSC was founded with financial support from 12 corporations: Air Canada, CPP Investments, Genpact, Loblaw Companies Limited, Magna, MDA, Maple Leafs Sports & Entertainment Partnership, Nutrien, Rogers, Scotiabank, Shoppers Drug Mart, and Suncor and received funding from the Safe Restart Agreement from the Government of Canada (Health Canada).

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: We thank Kyle Robinson, PhD, Devon Elke, HBSc, and Evgenia Gatov, MPH (all paid employees of CDL RSC, Toronto, Ontario, Canada), for assistance in collating the data.

References

Ogawa T, Fukumori T, Nishihara Y, et al. Another false-positive problem for a SARS-CoV-2 antigen test in Japan. J Clin Virol. 2020;131:104612. doi:10.1016/j.jcv.2020.104612 PubMed Google Scholar

False-Positive Results in Rapid Antigen Tests for SARS-CoV-2

Featured Articles

USPSTF Recommendation Statements

Blogs

See More About

Select Your Interests

Select Your Interests

Others Also Liked

Citation

Manage citations:

False-Positive Results in Rapid Antigen Tests for SARS-CoV-2