Dirty Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) Trial data: Corning-Besselaar Inc (Princeton, New Jersey), the contract research organization that ran the STEPS trial, sent Parke-Davis a series of letters in the spring of 1996 detailing the poor quality of much of the STEPS data. In this memo,17 Corning-Besselaar Inc describes several factors contributing to poor data, all of which were the result of deliberate decisions related to study conduct.
The 1996 Neurontin Situation Analysis. Neurontin Situation Analyses recapped the marketing strategies used for the previous year and introduced future strategies. This document20 shows that the Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) trial was intended to encourage investigator-physicians to increase their prescribed doses of Neurontin. The document also measures the success of Neurontin in terms of increases in prescriptions, dosage levels, and market share.
Memo from the director of customer marketing following up on the Neurontin Plus Plan, a comprehensive plan developed in the middle of 1995 to increase sales of Neurontin. This memo22 described several marketing programs that were available as part of efforts to expand the Neurontin prescribing base to community neurologists, including the Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) trial investigators' meetings.
A 1995 Selected Physician Titration Analysis, showing the dramatic increase in prescriptions by the Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) investigators compared with a control group of neurologists and child neurologists.31 The change in prescriptions of other anticonvulsants demonstrates no clear change among the 2 groups. N/CHN indicates neurologist/child neurologist. MKT was not defined.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Krumholz SD, Egilman DS, Ross JS. Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) Trial: A Narrative Account of a Gabapentin Seeding Trial. Arch Intern Med. 2011;171(12):1100–1107. doi:10.1001/archinternmed.2011.241
Seeding trials, clinical studies conducted by pharmaceutical companies for marketing purposes, have rarely been described in detail.
We examined all documents relating to the clinical trial Study of Neurontin: Titrate to Effect, Profile of Safety (STEPS) produced during the Neurontin marketing, sales practices, and product liability litigation, including company internal and external correspondence, reports, and presentations, as well as depositions elicited in legal proceedings of Harden Manufacturing vs Pfizer and Franklin vs Warner-Lambert, most which were created between 1990 and 2009. Using a systematic search strategy, we identified and reviewed all documents related to the STEPS trial in order to identify key themes related to the trial's conduct and determine the extent of marketing involvement in its planning and implementation.
Documents demonstrated that STEPS was a seeding trial posing as a legitimate scientific study. Documents consistently described the trial itself, not trial results, to be a marketing tactic in the company's marketing plans. Documents demonstrated that at least 2 external sources questioned the validity of the study before execution, and that data quality during the study was often compromised. Furthermore, documents described company analyses examining the impact of participating as a STEPS investigator on rates and dosages of gabapentin prescribing, finding a positive association. None of these findings were reported in 2 published articles.
The STEPS trial was a seeding trial, used to promote gabapentin and increase prescribing among investigators, and marketing was extensively involved in its planning and implementation.
Pharmaceutical companies use a variety of techniques to promote their products, including “seeding trials.” Seeding trials are clinical trials, deceptively portrayed as patient studies, which are used to promote drugs recently approved or under review by the US Food and Drug Administration (FDA) by encouraging prescribers to use these medications under the guise of participating as an investigator in a clinical trial.1 In fact, marketing departments, rather than clinical research departments, are known to design and conduct these trials.2 Although seeding trials are not illegal, they are unethical. Their primary goal is to expose physicians to a new drug and have them interact with the pharmaceutical company sponsor and its sales representatives, in order to influence prescribing decisions, independent of any findings from the actual study. In addition, physician “investigators” are the actual trial subjects, and this information is neither disclosed to them nor the human participants. There are no current estimates of how frequently seeding trials are conducted, and most evidence of their planning and conduct has come from documents produced in tort litigation against pharmaceutical companies.3
A recent analysis of documents produced during litigation against Merck & Co Inc (Whitehouse Station, New Jersey) related to rofecoxib led to the first in-depth account of a seeding trial. Documentation described the marketing rationale behind the Assessment of Differences between Vioxx and Naproxen To Ascertain Gastrointestinal Tolerability and Effectiveness (ADVANTAGE) trial, as well as the marketing department's involvement in trial conception and implementation.2
In 2006, an analysis of a limited set of documents produced during litigation against Parke-Davis (formerly a division of Warner-Lambert Co, now a subsidiary of Pfizer Inc, New York, New York) overviewed Parke-Davis's promotion of gabapentin (Neurontin), an anticonvulsant. Marketing involvement in the Study of Neurontin: Titrate to Efficacy, Profile of Safety (STEPS) trial was briefly mentioned among gabapentin marketing techniques, but discussion was necessarily incomplete; the investigation was limited to only a small subset of documents and depositions (approximately 250 of over 300 000 produced in litigation).4 The purpose of this investigation is to more fully evaluate whether STEPS was a seeding trial, including discussion of its conception, design, implementation, and impact on gabapentin prescribing. The recent availability of the complete set of documents and depositions produced during litigation provided a unique opportunity to examine the STEPS trial in more detail.
We examined all documents produced during the Neurontin marketing, sales practices, and product liability litigation. In Harden Manufacturing vs Pfizer and Franklin vs Warner-Lambert, plaintiffs' attorneys aggregated all documents produced by the defendants into an integrated database. Documents included internal and external correspondences, internal planning documents and presentations, clinical research reports, and market research analyses, most of which were created between 1990 and 2009. As consultants to the plaintiffs, we had access to the entire document production and all depositions taken for the cases; 2 of us were paid consultants (S.D.K. and D.S.E.), and the third investigator (J.S.R.) was an unpaid consultant.
One of us (S.D.K.) conducted a primary review of all documents from the database, identifying those related to the STEPS trial using a systematic search strategy, searching “STEPS” in conjunction with the following key words: trial, marketing, promotion, seeding, advisory board, investigators, and the names of internal personnel and associated external investigators. Documents identified using these key search terms were reviewed, subsequently leading to retrieval of related or referred-to documents that included STEPS-relevant content. The reviewer used a Boolean search because of the high frequency of the word “steps” within document production.
The original keyword searches returned approximately 3000 documents. The primary reviewer (S.D.K.) read all documents and identified a subset of approximately 400 documents relevant to STEPS- and marketing-related issues. These documents consisted primarily of internal memos, marketing presentations, and correspondence between Parke-Davis employees, STEPS investigators, and employees of Corning-Besselaar Inc (Princeton, New Jersey), a contract research organization. The primary reviewer reread these documents to establish the chronology of the trials conduct and design and to identify broad themes reflecting the design and conduct of a seeding trial. In this iterative process, segments of text were organized according to their essential concepts,5,6 a method similar to that used recently with court documents to examine tobacco marketing, pharmaceutical marketing, and ghostwriting for scientific papers.2,4,7-9 Next, selected subsets of documents, those that were more critical to establishing themes and the meanings of which were more open to interpretation, were reviewed by the other 2 investigators to identify and further develop core themes. Finally, the primary reviewer again reviewed all of the documents for additional evidence to support or refute the core themes.
The STEPS trial was a phase 4 uncontrolled, unblinded trial sponsored by Parke-Davis. The STEPS trial's stated objective was to study efficacy, safety, tolerability, and quality of life among gabapentin users when titrating the drug to patient effect. Parke-Davis recruited 772 investigators to participate in STEPS, enrolling 2759 patients, a ratio of approximately 4 patients per investigator. Informed consent documents explained that patients were enrolling in a study “designed to assess the safety and tolerability of doses of Neurontin (gabapentin) from 900 to 3600 mg daily whose partial seizures are not completely controlled by other drugs,”10 without mention of marketing objectives. Patients were initially given Neurontin, 900 mg/d, during the first week and then were to have their doses titrated up to 1800 mg/d, 2400 mg/d, and ultimately to 3600 mg/d. Deviation from the rigid titration schedule led to exclusion from the study's primary analysis. Titration stopped if the patient developed dose-limiting adverse effects or if the physician judged that the patient had reached an efficacious dose.11 The study ultimately resulted in 2 published articles in the journals Epilepsia12 and Seizure,13 one of which described the efficacy analysis, the other of which described the safety and tolerability analyses. Both articles were generally supportive, describing gabapentin as effective, safe, and tolerable.
The STEPS trial used an uncontrolled and unblinded design to study gabapentin efficacy, safety, and tolerability, a questionable design, particularly for efficacy. In fact, 2 independent external sources questioned the STEPS trial's scientific validity before it was initiated. The Johns Hopkins University institutional review board (IRB) (Baltimore, Maryland) rejected the application for the STEPS trial, both initially and on appeal, stating “the board in its deliberation, voted to disapprove the protocol, since we believe that the entry criteria and outcome measures are too vague to allow any scientific conclusions to be reach [sic].”14 Parke-Davis also remarked in an internal memo that the FDA director of the Division of Drug Marketing Advertisements and Communications [DDMAC] believed that “the idea [of STEPS] was a good one from a marketing perspective, [but] she did not think the trial was needed to acquire the desired information on high dose use.”15
Furthermore, the STEPS trial used complicated inclusion and exclusion criteria for each analysis, limiting generalizability. For instance, for the tolerability analysis, there was a prespecified, rigid up-titration method, which led to the exclusion of 87.3% nonrandom study participants. Following the completion of STEPS, customer business units (CBUs), which were autonomous, regionally focused branches of Warner-Lambert that planned and implemented marketing strategies, conceded that the study design was not rigorous enough for dissemination. For instance, a West CBU planning document acknowledged that “though [STEPS] is likely to be published, [Territory Managers] will not be able to distribute results due to the open label format of the study.”16
In addition to STEPS’ design limitations, the trial itself was not conducted in a way that was conducive to ensuring good data quality. Parke-Davis recruited site investigators with little or no clinical trial experience, provided insufficient training, and did not audit study sites prior to the beginning of the trial, which led to poor trial data quality. An April 1996 memo from Corning Besselaar [the contract research organization used for the trial], noted that “the data clean-up process for STEPS has been a larger task than anticipated. The data was very dirty;” and added17 (Figure 1)
Investigators are inexperienced with conducting clinical trials, investigators do not have study coordinators, up-front training for completing Case Report Forms (CRFs) was minimal at the video-conferenced investigator meeting, and the CRF does not have annotated pages included for reference.
During statistical analysis, Parke-Davis also described data quality issues. For instance, site investigators were nonadherent to the seizure frequency assessment protocol, scheduling follow-up visits after more than 16 weeks, such that fewer than 25% of patients were assessed during follow-up weeks 13 through 16.18 In 1 case, 27 weeks elapsed between the baseline and final visit. Yet there were no mentions of data irregularity in either the internal research report or the published articles.
Pharmaceutical sales representatives were directly involved in collecting and recording individual subject trial data. At a December 1995 Northeast CBU anticonvulsant advisory board meeting, a co–lead investigator explained that there was a greater completion rate when Parke-Davis representatives filled out the study forms.19 Although this seemed to raise concerns among some meeting attendees, a Parke-Davis marketing manager reassured the audiences that “STEPS a Phase IV postmarketing study and does not follow the same rigorous protocols as phase III trials. We should stress with the representatives that they are not allowed to fill out paperwork [underline from original].”19 Nevertheless, the role some representatives played in data collection was not mentioned in the final published articles.
The STEPS trial was a key component of gabapentin marketing strategy. Multiple strategic plans cite the STEPS trial itself, as opposed to the anticipated trial findings, as a key marketing tool for the promotion of gabapentin. The 1995 Neurontin Situation Report20 identified STEPS as a key deliverable under the strategy “Solidify Neurontin's position with neurologists and select primary care physicians as the safe and easy add-on for refractory patients.” Furthermore, the purpose and anticipated impact of the STEPS trial are described20 (Figure 2) as
To give neurologists the opportunity to titrate to higher doses (>1800 mg) when needed. Some indicators of success include 20% increase in new patients' starts in March and a 3% market share in new prescriptions, percent increase in 400 mg strength. The next key steps are to watch the average size of a prescription increase and to enroll patients as quickly as possible.
The STEPS trial conduct facilitated Parke-Davis's efforts to reach community and office-based neurologists. A North Central CBU document states: “the rapid growth of Neurontin depends on the ability to influence the large population of community neurologists that see the majority of nonrefractory seizure patients. The STEPS trial . . . was a strong start to this Customer Business Unit (CBU) priority.”21 A similar push took place in the Northeast CBU, where one marketing memo stated “STEPS is the best tool we have for Neurontin and we should be using it wherever we can”22 (Figure 3). These memos were written as the trial was ongoing.
For STEPS to reach as many community neurologists as possible, site investigators were widely recruited across the country. The company sent recruitment letters to approximately 5000 potential investigators.23 Ultimately, 1542 “potential investigators” attended an introductory study briefing simulcast to 9 different regional centers,24,25 where, along with study information, promotional information about gabapentin was also presented.21,26 To accommodate the involvement of such a large number of site investigators, sites were generally limited to a maximum of 10 patients each and averaged 4 recruited patients per site. However, more influential site investigators were offered the opportunity to recruit more patients.27
Patient recruitment for the STEPS trial was also used as an opportunity to provide marketing information about gabapentin to physicians. Company representatives were encouraged to ask site investigators to institute “Shadow Days,” during which patients with epilepsy would make up the bulk of the clinic day's schedule, permitting representatives to be present and encourage patient enrollment while simultaneously promoting gabapentin. The company also suggested offering promotional rewards to achieve enrollment goals. For example, company sales representatives rewarded some investigators for achieving specific recruitment milestones; physicians were given a free lunch after recruiting 3 patients and a free dinner after 7 patients.28 Patients were not informed of these “promotional reward” programs.
Patient recruitment for the STEPS trial was used not only as an opportunity to promote gabapentin, but also to block competing medications, particularly lamotrigine (Lamictal; GlaxoSmithKline, Brentford, England). For instance, one employee explained that “at the very least, we should be looking to place as many managed-care patients as feasible in [STEPS] to prevent Lamictal starts.”29 A 1996 “Neurontin Situation Analysis” similarly suggested that an added benefit of the STEPS trial is that it “effectively blocked physicians from actively participating in the Lamictal/Alert trial.”30
The purpose of STEPS was to increase the prescription rates of physician-investigators. Parke-Davis monitored investigators' prescribing of gabapentin during and after the completion of the STEPS trial through analyses completed by its Precision Marketing department and by the IMS Health Promotrak division. Neither the investigators nor the patients were informed that these analyses would be conducted as part of the study. One analysis compared prescribing before and after attendance at the STEPS introductory investigators briefing, finding a 38% increase in prescriptions after event attendance.31 The same study also found a 10% increase in the average prescribed dose.31 Subsequent analysis demonstrated that investigator attendance was associated with persistently increased high-dose prescribing over time, although the rate of increase decreased.32 Other analyses compared prescribing among STEPS investigators with a control group of neurologists and child neurologists, finding similarly increased prescribing among STEPS investigators31,33,34 (Figure 4). The success of the STEPS trial in increasing prescriptions was reported within the 1996 “Neurontin Situation Analysis,” which described how investigators were “increasing their [Neurontin] shares . . . and their use of 400 mg capsules.”30 Of note, these analyses were conducted prior to the dissemination of any trial results.
The STEPS trial was a seeding trial used to promote gabapentin and increase prescribing among investigators despite its stated scientific objective to examine efficacy, safety, and tolerability of the drug. Although STEPS was conducted 15 years ago, the ethical issues illustrated by the trial's conduct, and the data gained from Parke-Davis's marketing analyses, have tremendous relevance in today's debates over the limits and consequences of pharmaceutical industry sponsorship of phase 4 postmarketing clinical trials. To our knowledge, our analysis is only the second comprehensive account of a seeding trial based on primary source documents and clearly demonstrates how a clinical trial was designed, conducted, and explicitly used to promote marketing objectives, not science, without providing full informed consent to the patients and physicians who participated in the study.
Seeding trials have been used in the pharmaceutical industry for at least 20 years.1 However, since they are designed to have an impact on sales, they may never be published and remain difficult to identify even when published. The only previous documentary account of a seeding trial, published in 2008,2 identified 3 main characteristics of seeding trials: marketing involvement in study conception and design, marketing involvement in data collection and analysis, and nondisclosure of the study's true purpose from institutional review boards, patients, and investigators. Similarly, Kessler et al1 described seeding trials, without the use of source documents, as lacking scientifically rigorous design or purpose and using the trial itself, not its findings, for wider marketing campaigns. STEPS differed from these previous examples because it was designed and conducted by a contract research organization, not a company's marketing department. However, STEPS was clearly intended to promote gabapentin. The study, independent of any results, was repeatedly described as a means to market gabapentin and increase prescribed dosages. Our findings both corroborate previous descriptions of seeding trials and provide new insights into their execution.
There has been little academic research on the effect of seeding trials on prescribing. Danish researchers examined the effect on prescribing of participation in a clinical trial as a site investigator,35 finding participation was associated with increased prescribing of the trial sponsor's drug. In both ADVANTAGE and STEPS, the sponsor companies internally measured prescribing among investigators and found positive effects.2 While these internal analyses are suggestive, caution is warranted because the data and reports were not peer reviewed and there were strong incentives to demonstrate that these seeding trials, as investments, were successful, potentially biasing the data. In addition, much of the STEPS prescription data was uncontrolled and limited to summary descriptions, not raw data. Rigorously conducted research examining the impact on prescribing of participating as a trial investigator, or attending clinical trial marketing events, is necessary.
There were several ethical breaches within the STEPS trial. Principally, STEPS and all seeding trials prevent patients from making informed consent decisions about participation because the true marketing objectives are not disclosed. Informed consent is an established cornerstone of research on human subjects, both internationally36 and in US law.37 During STEPS, among 2759 patients, 11 patients died, 73 experienced serious adverse events, and 997 experienced less serious adverse effects,11 suggesting that patients were at more than minimal risk. Second, investigators were also not fully informed, which is clearly unethical because these physicians were the intended study subjects. Third, the promotional rewards used within STEPS were also unethical. Conventional wisdom suggests that providing a small gift after data collection is acceptable but is unacceptable when given before,38 particularly if potentially coercive or presents undue influence.39
Seeding trials are not illegal and generally do not fall under the authority of the FDA, which has oversight only over clinical trials conducted as part of new drug applications or intended to support other label or advertising changes.40 However, the US clinical trial regulatory system, principally under the authority of the US Office for Human Research Protections, includes registration of clinical trials and protection of human research subjects that is dependent on individual IRBs. As such, IRBs likely have the strongest potential to prevent seeding trials, outside of appeals to professionalism and ethical practice. However, recent research on IRBs suggest problems with conflict of interest and lax regulation, among both commercial and academic IRBs.41 These findings were substantiated by the US Government Accountability Office, which demonstrated the inability of some commercial IRBs to protect against obvious violations of subjects' rights and suggested that IRBs were not effective at denying approval of scientifically unsound studies.42
Several steps may strengthen IRBs' ability to prevent seeding trials. First, IRBs require stricter government oversight. All IRBs should be registered, accredited, and evaluated, with penalties for the approval of trials that do not meet ethical standards. Second, commercial IRBs should not be accredited. There is an inherent conflict of interest when an organization responsible for protecting human subjects subsists on payments from trial sponsors, potentially leading to companies shopping protocols to find the most receptive IRB. Third, IRBs should use a publicly available repository to circulate previous reviews and rejections. Although the FDA requires that prior IRB reviews be submitted as part of subsequent applications,43 this is not always practiced. For the STEPS trial, the concerns raised by the Johns Hopkins IRB might have alerted others. Finally, posting of original protocols within a publicly available repository may also help to identify seeding trials, post hoc, so that investigators, sponsors, and the IRBs that approved them can be identified. Even published seeding trials are challenging to recognize. But if a study is never published, or published misleadingly, there is no way for patients, physicians, or regulators to know the true nature of the trial. While mandatory trial registration within ClinicalTrials.gov is an important first step to address selective publication of all types of clinical trials, at this time registration does not include posting of study protocols, which could identify marketing-focused studies.
Our analysis has several limitations. First, because our analysis is limited to only 1 pharmaceutical company and describes events that took place 10 to 15 years ago, our findings may not be generalizable to today's marketplace. However, our findings are consistent with the practices identified during another seeding trial.2 Second, we did not communicate with any company representatives or scientific investigators involved with STEPS. We based our analysis entirely on document review, although we did also have access to deposition testimony. Third, any qualitative assessment of documents (or historical work in general) is susceptible to misinterpretation and unconscious bias. This analysis amounts to the authors' best effort to faithfully and accurately reconstruct the planning, implementation, and execution of the STEPS trial. Finally, given the large size of the document database, we may have missed relevant documents in the course of our search, although we used comprehensive search strategies to minimize this possibility.
In conclusion, the STEPS trial was a seeding trial masquerading as a scientific study. Parke-Davis performed an in-depth marketing analysis to track the effect of attendance at the STEPS introductory briefing and participation as a study investigator on the rate and dosage of gabapentin prescribing. No study publications mentioned the internal data quality problems, tampering (representatives filling out study forms) or the study's marketing goals. Our analysis provides critical evidence suggesting that seeding trials are used as a promotional strategy by pharmaceutical companies. Reform of the current IRB system, as well as promoting better clinical trial practice in the human subjects research community, are necessary to prevent continued conduct of seeding trials by the pharmaceutical industry.
Correspondence: Joseph S. Ross, MD, MHS, Section of General Internal Medicine, Department of Medicine, Yale University School of Medicine, PO Box 208093, New Haven, CT 06520 (email@example.com).
Accepted for Publication: January 6, 2011.
Author Contributions:Study concept and design: Krumholz, Egilman, and Ross. Acquisition of data: Krumholz. Analysis and interpretation of data: Krumholz, Egilman, and Ross. Drafting of the manuscript: Krumholz, Egilman, and Ross. Critical revision of the manuscript for important intellectual content: Krumholz, Egilman, and Ross. Administrative, technical, and material support: Egilman. Study supervision: Ross.
Financial Disclosure: Mr Krumholz and Dr Egilman are currently paid consultants at the request of plaintiffs in litigation against Pfizer Inc related to gabapentin in the United States, and Dr Ross is currently an unpaid consultant in the same litigation. Drs Egilman and Ross were previously paid consultants at the request of plaintiffs in litigation against Merck & Co Inc related to rofecoxib in the United States.
Funding/Support: This project was not directly supported by any external grants or funds. Dr Ross is currently supported by the National Institute on Aging (K08AG032886) and the American Federation of Aging Research through the Paul B. Beeson Career Development Award Program.