[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Viewpoint
July 26, 2016

Stealth Research and TheranosReflections and Update 1 Year Later

Author Affiliations
  • 1Stanford Prevention Research Center, Department of Medicine, Stanford University School of Medicine, Stanford, California
  • 2Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California
JAMA. 2016;316(4):389-390. doi:10.1001/jama.2016.6986

I first noticed the name Theranos while walking by its facilities in Palo Alto, California, in 2014. That name was so intriguing that I searched Google to find it and learned that it supposedly combined the Greek words “therapy” and “diagnosis.” The basic vision was impressive: massive, inexpensive diagnostic testing using a new technology that could run hundreds of tests concurrently at very low cost and using tiny amounts of blood without a needlestick. The idea crossed my mind to consider collaborating with such avant-garde entrepreneurs in research studies.

A search of the scientific literature to see how this technology works revealed a surprise: no articles could be located. This led to a Viewpoint in JAMA in February 2015 in which Theranos was used as an example to question the appropriateness of “stealth research” in biomedicine1 (ie, research not communicated in the peer-reviewed literature), and concerns were raised about the rationale for promoting massive diagnostic testing. At that time Theranos had teamed with Walgreens and had plans for partnering with Safeway for offering blood testing in their stores. However, diagnostic testing performed while buying candy or groceries made me uneasy.

After publication of the Viewpoint, some media and scientists sensitized to the need for transparency, rigorous peer-review, and reproducibility echoed these concerns, and some laboratory scientists also questioned what Theranos was trying to accomplish.2 However, the company continued to thrive. Its chief executive officer, Elizabeth Holmes, received many accolades including an honorary PhD from Pepperdine University; the US vice president visited the company’s facilities; and most mass media continued to praise the biotech start-up with the highest valuation. In July, the US Food and Drug Administration (FDA) approved one Theranos test (herpes simplex virus antibody),3 and more positive media attention followed.

Meanwhile, the new Theranos general counsel reached out to me and suggested a meeting with Ms Holmes to coauthor with her an editorial in some major medical journal supporting the view that FDA clearance offered the highest possible level of evidence for any diagnostic blood test technology. The willingness to pursue FDA review is important (and something that other companies should also do) but insufficient. For any technology, and even more so for one aiming to affect health care at large, scientific peer-reviewed articles are a requirement. Therefore, I declined to write that article but agreed to meet Ms Holmes to exchange views.

Theranos did not confirm my tentative meeting with Ms Holmes. Instead I was contacted by a Wall Street Journal investigator to talk about the company. On October 16, 2015, the Wall Street Journal published the results of its investigation, and it raised many important questions about Theranos.4 It claimed that Theranos was no longer using its own technology (except for the one test that was FDA approved), and there were stories of patients receiving incorrect test results, a sensational suicide story, lack of trust, concerns about lack of biomedical expertise of the Theranos advisory board, and more.4 The Theranos communications team predictably responded and the company enriched the advisory board with biomedical experts. Ms Holmes spoke to the media multiple times claiming that any fearless innovator meets with resistance before winning the world over. Nevertheless, what was needed was published, peer-reviewed scientific evidence to evaluate the new technology.5

The next 6 months were far more difficult for Theranos. In a spiraling sequence, multiple authorities questioned the work of the company. On October 28, 2015, the FDA reported that it had found major shortcomings and ordered Theranos to stop using its new device (called Edison) except for the one approved test. More concerns were reported by the Centers for Medicare & Medicaid Services (CMS) noting multiple deficiencies with both testing practices and results, including failed control checks, concerns about the qualification of personnel, the chain of communicating results, sample storage, and more. In late January 2016, CMS claimed that testing at the Newark, California, Theranos laboratory location (based on the November 2015 assessments) posed “immediate jeopardy to patient health and safety.”6 In the meantime, Walgreens placed on hold the Theranos Wellness Centers and stopped testing in California (the only site using Edison). Plans for collaboration with Safeway and Cleveland Clinic had already ended. In March and April 2016, CMS was apparently considering major sanctions against the company, including a 2-year ban for Ms Holmes to own or operate a blood laboratory unless regulatory requirements were promptly met.7

Some peer-reviewed data also appeared, but not from Theranos. A study by academic investigators8 compared Theranos testing against standard testing platforms for several commonly performed diagnostic tests and found Theranos lacking in almost all aspects. Theranos flagged tests outside their normal range 1.6 times more often than the 2 standard comparators; 15 of 22 laboratory measurements evaluated showed large interservice variation; the laboratory results varied substantially depending on what testing service had been used and at what time the sampling had been done; and there was a particularly worrisome divergence of results for lipid levels. As of the writing of this Viewpoint (May 6, 2016), Theranos was under investigation by federal prosecutors and by the Securities and Exchange Commission.6 Mass media speculated about whether there was evidence for Theranos misleading investors, customers, or both.

It would be ideal if the original success story were correct: a new company with new technology that disrupts the entire landscape of diagnostic testing. Hopefully, evidence that this disruptive technology works will be forthcoming in peer-reviewed publications. However, as evidence is delayed, a success story becomes less likely.

Perhaps more important, diagnostic testing does need to be disrupted, but not for the reasons originally proposed in exciting talks to general, nonscientific audiences. Too much diagnostic testing is performed for no reason, and this has profound consequences for overdiagnosis, inaccurate results that spiral down the path of more testing, and false test results that lead to incorrect therapeutic decisions and iatrogenic disease. The resulting cumulative burden of disease and cost is proportional to the amount of unnecessary testing done. The problem that some patients do not have tests performed when they genuinely need them is also real but probably of lesser magnitude. Currently, the costs of diagnostic testing are not well justified, much like the costs of therapeutic medicine, for which insurance companies and other intermediaries charge prices that are astronomical compared with the real cost involved. Better financing and organization of health care and, perhaps, reduction of the profit margin could markedly decrease testing cost, even if very old (but appropriately validated) diagnostic technologies are used.1

Finally, the most troubling aspect may be the core vision of Theranos and other companies that try to push the agenda of massive personalized diagnostic testing. The notion of patients and healthy people being repeatedly tested in supermarkets and pharmacies, or eventually in cafeterias or at home, sounds revolutionary, but little is known about the consequences. Even if the tests were accurate, when they are performed in massive scale and multiple times, the possibility of causing substantial harm from widespread testing is very real, as errors accumulate with multiple testing. Repeated testing of an individual is potentially a dangerous self-harm practice, and these individuals are destined to have some incorrect laboratory results and eventually experience harm, such as, for example, the anxiety of being labeled with a serious condition or adverse effects from increased testing and procedures to evaluate false-positive test results. Moreover, if the diagnostic testing process becomes dissociated from physicians, self-testing and self-interpretation could cause even more problems than they aim to solve. Physicians use diagnostics less than optimally, but it is not clear that healthy people or patients can be trained to use diagnostics more wisely. These issues remain unresolved. Hopefully, the name Theranos does stand for well thought-out and useful therapy and diagnosis and does not represent the harms suggested by another similar Greek word, thanatos (death). Regardless, discovering and validating new diagnostic technologies should not necessarily be equated with more testing. Less may be better.

Back to top
Article Information

Corresponding Author: John P. A. Ioannidis, MD, DSc, 1265 Welch Rd, Medical School Office Bldg X306, Stanford, CA 94305 (jioannid@stanford.edu).

Published Online: May 18, 2016. doi:10.1001/jama.2016.6986.

Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.

References
1.
Ioannidis  JP.  Stealth research: is biomedical innovation happening outside the peer-reviewed literature? JAMA. 2015;313(7):663-664.
PubMedArticle
2.
Diamandis  EP.  Theranos phenomenon: promises and fallacies. Clin Chem Lab Med. 2015;53(7):989-993.
PubMedArticle
3.
US Food and Drug Administration. 510(k) Substantial equivalence determination decision summary. http://www.accessdata.fda.gov/cdrh_docs/reviews/k143236.pdf. Accessed May 6, 2016.
4.
Carreyrou  J. Hot startup Theranos has struggled with its blood-test technology. Wall Street Journal. October 16, 2015. http://www.wsj.com/articles/theranos-has-struggled-with-blood-tests-1444881901. Accessed May 6, 2016.
5.
Ioannidis  JP. For health tech to thrive, don’t keep the evidence under wraps. New Scientist. October 2015. https://www.newscientist.com/article/mg22830450-300-for-health-tech-to-thrive-dont-keep-the-evidence-under-wraps/. Accessed May 6, 2016.
6.
Bilton  N. The secret culprit in the Theranos mess. Vanity Fair. May 2, 2016. http://www.vanityfair.com/news/2016/05/theranos-silicon-valley-media. Accessed May 6, 2016.
7.
Abelson  R, Pollack  A. Theranos under fire as US threatens crippling sanctions. New York Times. April 13, 2016. http://www.nytimes.com/2016/04/14/business/theranos-elizabeth-holmes-proposed-ban.html?_r=0. Accessed May 6, 2016.
8.
Kidd  BA, Hoffman  G, Zimmerman  N,  et al.  Evaluation of direct-to-consumer low-volume lab tests in healthy adults. J Clin Invest. 2016;126(5):1734-1744.
PubMedArticle
×