Evaluation of Serious Adverse Drug Reactions: A Proactive Pharmacovigilance Program (RADAR) vs Safety Activities Conducted by the Food and Drug Administration and Pharmaceutical Manufacturers

Background  The Food and Drug Administration (FDA) and pharmaceutical manufacturers conduct most postmarketing pharmaceutical safety investigations. These efforts are frequently based on data mining of databases. In 1998, investigators initiated the Research on Adverse Drug events And Reports (RADAR) project to investigate reports of serious adverse drug reactions (ADRs) and prospectively obtain information on these cases. We compare safety efforts for evaluating serious ADRs conducted by the FDA and pharmaceutical manufacturers vs the RADAR project.

Methods  We evaluated the completeness of serious ADR descriptions in the FDA and RADAR databases and the comprehensiveness of notifications disseminated by pharmaceutical manufacturers and the RADAR investigators. A serious ADR was defined as an event that led to death or required intensive therapies to reverse.

Results  The RADAR investigators evaluated 16 serious ADRs. Compared with descriptions of these ADRs in FDA databases (2296 reports), reports in RADAR databases (472 reports) had a 2-fold higher rate of including information on history and physical examination (92% vs 45%; P<.001) and a 9-fold higher rate of including basic science findings (34% vs 4%; P = .08). Safety notifications were disseminated earlier by pharmaceutical suppliers (2 vs 4 years after approval, respectively), although notifications were less likely to include information on incidence (46% vs 93%; P = .02), outcomes (8% vs 100%; P<.001), treatment or prophylaxis (25% vs 93%; P<.001), or references (8% vs 80%; P<.001).

Conclusion  Proactive safety efforts conducted by the RADAR investigators are more comprehensive than those conducted by the FDA and pharmaceutical manufacturers, but dissemination of related safety notifications is less timely.

Adverse drug reactions (ADRs) result in health care costs of $3.6 billion annually and are among the top 10 leading causes of death.^1-3 After product recalls for rofecoxib and cerivastatin, concern exists over the ability of the Food and Drug Administration (FDA) and pharmaceutical manufacturers to identify serious adverse events and to notify physicians about these findings.^4-8 Serious ADRs can go unrecognized if initial safety signals are unclear.² New approaches to identifying unexpected pharmaceutical toxic effects are needed.^9,10

One concern is that current pharmacovigilance efforts are hindered by low reporting rates, and when ADRs are reported the information is of variable quality.¹¹ When pharmaceutical manufacturers and the FDA become convinced that an event represents a serious ADR, safety notifications are disseminated.^2,12 Physicians may not be aware of serious ADRs if dissemination of these notifications is delayed, the safety notification is not comprehensive, or they do not read the notifications.

Academic researchers and nonprofit organizations have developed systems to identify and evaluate serious ADRs.^13-16 As noted previously, surveillance efforts conducted by these groups differ from those of the FDA: the organizations focus on serious ADRs, and additional reports are obtained after queries to health care professionals who provide care for patients with the relevant toxic effects.¹⁷ The deputy director of the FDA proposed that adverse event reports obtained by proactive pharmacovigilance should be transmitted to the FDA and would augment existing pharmacovigilance systems.¹⁸ However, before augmentation can occur, comparisons of these pharmacovigilance efforts are needed. Herein, we evaluate 2 questions that can inform this comparison: how complete are adverse event reports included in databases obtained by these systems, and how timely and complete are safety notifications disseminated by these programs? We limit these comparisons to recent safety evaluations conducted by one of the largest nonprofit pharmacovigilance programs, the Research on Adverse Drug events And Reports (RADAR) project.¹⁴

The RADAR project is a pharmacovigilance program that focuses on identification, evaluation, and dissemination of information describing serious ADRs, such as those resulting in death or severe organ failure or precipitating major therapeutic interventions. The RADAR project is funded by grants from the National Heart, Lung, and Blood Institute, the National Cancer Institute, the American Cancer Society, and the Department of Veterans Affairs. Pharmaceutical manufacturers do not provide support, although they are asked to provide relevant clinical information on each serious ADR.

The RADAR project is led by a hematologist-oncologist and health services researcher (C.L.B.) and consists of 25 investigators with training in internal medicine and the medical subspecialties of clinical pharmacology, epidemiology, statistics, and pharmacy. An investigation is initiated when a clinical event that represents a possible occurrence of a serious ADR is seen by a RADAR coinvestigator or reported by unaffiliated physicians. Senior members of the RADAR team review the indicator event, as well as associated literature and package inserts, to determine if the event represents an ADR. If further investigation of the ADR is meritorious, queries for additional case reports are submitted to the FDA and institutional review board approval is requested at collaborating institutions. The FDA reports are subject to a preliminary review to inform hypothesis generation and refine data collection tools. The case forms include demographics, source of information (eg, clinical trial or physician queries), medication history, event description (eg, date of the event, time elapsed between exposure and ADR event, event duration, other relevant history, physical findings, and study results), organ-specific history, and treatments. World Health Organization criteria are used to score causal associations between the suspect drug and the event. The RADAR investigators refine hypotheses about pathophysiologic features. Data elements and coding accommodate the range of data available in the FDA databases and address underlying causality hypotheses. Coding of the case classification form is designed to facilitate algorithmic analysis of case findings (Figure).¹⁴

Initial sources for RADAR investigations are descriptions of single events of serious ADRs identified by a RADAR investigator, often originating with queries from health care workers, attorneys, or patients. The RADAR data sources for obtaining additional information include communications with clinical trialists or referral centers that treat large numbers of individuals with the given toxic effect. The FDA data sources for obtaining follow-up reports include spontaneous adverse event reports contained in MedWatch¹⁹ (for drugs) and the Manufacturer and User Facility Device Experience Database (MAUDE)²⁰ (for drug-coated devices). Dissemination of RADAR findings occurs via published abstracts or articles. Dissemination of FDA and pharmaceutical supplier findings is made via “Dear Doctor” letters and package insert revisions.

Safety elements were abstracted onto syndrome-specific forms.^21-32 These forms are based on the ADR systems of the World Health Organization³³ and Naranjo et al.^34,35 Data elements included history and physical examination, laboratory,radiologic, and basic science findings; age and sex; clinical diagnosis for which the drug was being prescribed; duration and dose of the drug administered; use of concomitant drugs; relevant clinical and laboratory findings that supported the diagnosis of the adverse event; duration of clinical abnormalities; exposure-adjusted incidence rates; treatments and prophylaxis; diagnosis date; date of reporting to the FDA or RADAR investigators; rechallenge and dechallenge; and outcome.

Data elements in the summary safety notification analysis included date of publication (for manuscripts and abstracts) or distribution (for package insert revisions or Dear Doctor letters); descriptions of clinical findings, exposure-adjusted incidence rates, and treatment and prophylaxis regimens; and references to relevant peer-reviewed articles. For each of the safety evaluations, completeness percentages were derived for individual case descriptions contained in the respective safety databases.^21-32 Specific categories included history and physical examination findings, laboratory and radiologic test results, and basic science correlative studies. For each summary safety notification published as a peer-reviewed publication, Dear Doctor letter, or package insert revision, mean percentages were derived for frequency of inclusion in these reports of relevant clinical findings, estimated reporting and exposure-adjusted incidence rates, treatment and prophylaxis recommendations, and references to relevant peer-reviewed articles. A time to initial publication was derived for each summary notification, operationally defined as the time from FDA approval to publication (for RADAR notifications) or ADR notification via a Dear Doctors letter (for FDA or pharmaceutical supplier notification) or revised package inserts (for pharmaceutical supplier notifications).

Overall unweighted mean percentages of completeness for inclusion of information from history and physical examinations, laboratory and radiologic tests, and basic science correlative studies were derived for clinical adverse event reports included in the FDA and RADAR safety databases. Statistical comparisons were made of completeness percentages for these individual data elements in FDA vs RADAR databases. With respect to dissemination, we reported mean completeness percentages for inclusion of relevant safety information in notifications from RADAR and the pharmaceutical manufacturers, as well as median time to publication of RADAR notification vs safety notifications from pharmaceutical manufacturers and/or the FDA.^36-47 Statistical analyses were conducted using optimal discriminant analysis, a nonparametric exact statistic.⁴⁸ The project received institutional review board approval from Northwestern University.

Between 1998 and 2006, the RADAR investigators published safety notifications for 12 unique serious ADRs associated with 14 drugs and 2 devices.^21-32 The safety investigations were initiated after identification and reporting of a small number of serious individual clinical events by physicians conducting early-phase clinical trials (thalidomide- and lenalidomide-associated venous thromboembolism, gemtuzumab ozogamicin–associated sinusoidal obstructive syndrome, and gemcitabine hydrochloride–associated pneumonitis) or specialists who reviewed clinical and laboratory findings for individuals who experienced a likely drug- or device-related toxic effect (drug-eluting cardiac stent–associated hypersensitivity and erythropoietin-associated pure red blood cell aplasia) or provided care for individuals who experienced a probable drug-related toxic effect (ticlopidine hydrochloride– and clopidogrel bisulfate–associated thrombocytopenic purpura, vardenafil hydrochloride– and tadalafil-associated optic neuritis, nevirapine-associated hepatoxicity among health care workers not infected with human immunodeficiency virus [HIV], enoxaparin sodium–associated hemorrhage after cardiac angioplasty procedure, and bisphosphonate-associated jaw osteonecrosis; Table 1).

These clinical events required primarily intensive interventions, and occasionally patients died of the toxic effect. For each ADR, descriptions of additional detailed case information were obtained from health care professionals who conducted early-phase clinical trials, had referral practices that evaluated or treated patients with the relevant toxic effect, or directed specialty centers such as blood banks or leukemia referral centers. The number of individual event reports associated with each of the ADRs ranged from 3 to 1348. Overall, RADAR databases included 472 individual reports and FDA databases included 2296 reports.

Individual case descriptions were more complete in RADAR vs FDA databases (Table 1). The RADAR database had significantly higher mean rates of inclusion of history and physical examination information (92% vs 45%; P<.001) and a trend toward higher mean inclusion rates of basic science laboratory studies (34% vs 4%; P = .08). For laboratory and radiologic findings, reporting comprehensiveness rates were comparable (54% vs 46%; P = .61). Items frequently found in RADAR but not in FDA reports included physical examination findings, radiologic reports (venogram, computed tomogram, or magnetic resonance imaging findings), laboratory studies (hemoglobin level, creatinine level, and platelet count), pathologic findings (bone marrow aspirate, liver biopsy, bone biopsy, and autopsy findings), and antibody assay results.

Timing and completeness of safety notifications disseminated by pharmaceutical manufacturers and RADAR investigators differed (Table 2). Safety notifications were disseminated earlier by pharmaceutical manufacturers, usually as package inserts or Dear Doctor letters (median of 2 vs 4 years after FDA approval). For 4 ADRs (thrombotic thrombocytopenic purpura associated with clopidogrel, venous thromboembolism associated with thalidomide and lenalidomide, nevirapine-associated hepatotoxicity, and bisphosphonate-associated osteonecrosis of the jaw), RADAR safety notifications were the primary sources of summary safety information for the RADAR project, as well as for the related Dear Doctor letters and package insert notifications disseminated by pharmaceutical manufacturers.^22,31 For 2 ADRs (fulminant hepatic failure with nevirapine that contained post–HIV-exposure prophylaxis regimens and enoxaparin-associated severe hemorrhages after invasive cardiologic procedures), safety notifications were made via RADAR publications but are not described in materials distributed by the pharmaceutical manufacturer because these toxic effects occurred in off-label settings.

Compared with notifications disseminated as pharmaceutical package insert revisions or Dear Doctor letters, RADAR notifications were more likely to include reporting rates or exposure-adjusted incidence rates (93% vs 46%; P = .02), outcomes (100% vs 8%; P<.001), treatment and prophylaxis recommendations (93% vs 25%; P<.001), and peer-reviewed references (80% vs 8%; P<.001). Of note, for drug-eluting cardiac stents, a Dear Doctor letter issued on October 29, 2003, described rare instances of hypersensitivity when drug-eluting stents were placed in coronary arteries.⁵² One month later, a follow-up notification from the FDA indicated that these cases were most likely related to hypersensitivity to concomitantly administered drugs, primarily clopidogrel.⁵³ In 2004, a RADAR notification indicated that some instances of hypersensitivity appeared to be caused by the stents, since autopsies revealed eosinophilic infiltrates at the stent site for 4 patients and information in FDA notifications rarely supported clopidogrel as the cause.^27,54

In this review of pharmacovigilance efforts associated with 16 serious ADRs, we found that clinical information obtained by the proactive RADAR program was of higher quality, but lower quantity, than that contained in the combination of spontaneous adverse event reports submitted to the FDA and adverse event reports submitted to the FDA by pharmaceutical manufacturers. Also, safety notifications from the RADAR investigators were more complete than those from pharmaceutical manufacturers or the FDA, although they were less timely. Our findings highlight differences between proactive pharmacovigilance efforts conducted by the RADAR group vs pharmacovigilance efforts conducted by the FDA and pharmaceutical manufacturers.⁵⁵ In interpreting our findings, several factors should be considered.

Pharmacovigilance efforts of the FDA or the pharmaceutical supplier did not result in the initiation of investigations of any of the serious ADRs described herein. Indeed, none of the 5 ADRs for which more than 200 individual case descriptions resided in the FDA safety databases were investigated before the initial reports of serious clinical events associated with these ADRs were published (Table 1). In contrast, each of the RADAR investigations was initiated when a serious clinical event that represented a potential serious ADR was identified by a RADAR investigator. As noted by others, despite having a large number of ADR reports in FDA databases, it is a difficult task to clearly identify a “signal” that suggests a new ADR has occurred vs “noise” associated with complex clinical settings.¹⁰

With respect to follow-up evaluations, the FDA receives more than 400 000 reports of serious and nonserious ADRs annually.⁵⁶ It is impractical for the FDA to request follow-up information on most of these cases. However, in rare instances in which follow-up clinical information for an individual case report has been requested by pharmaceutical manufacturers or the FDA, health care professionals rarely comply with these requests.¹⁰ In contrast, health care professionals almost universally responded to follow-up queries from RADAR investigators. Although not paid for these efforts, participating physicians have subsequently contributed as coauthors to the summary RADAR publications that are disseminated in peer-reviewed medical journals.¹¹ The end result is that large numbers of incomplete ADR reports are submitted by many health care professionals to the FDA and pharmaceutical manufacturers, whereas a small number of health care professionals submit fewer but more completely described safety reports to the RADAR program. It is the quality, not the quantity, of the case report information that primarily supports successful pharmacovigilance efforts. For 8 serious ADRs, RADAR summary reports were based on comprehensive review of a median of only 20 case reports for the individual toxic effect. Causality assessments, the most difficult and one of the most important aspects of pharmacovigilance studies, were included in several RADAR investigations, such as biopsy or autopsy findings or antibody test results for a small number of patients, but not in the pharmaceutical or FDA summary safety reports.

The RADAR notifications were disseminated a median of 2 vs 4 years after FDA approval. These notifications were disseminated primarily as peer-reviewed manuscripts, although some of the more urgent notifications were rapidly distributed as electronic preprints (clopidogrel-associated thrombotic thrombocytopenic purpura) or research letters (lenalidomide-associated thromboembolism) within 1 year of FDA approval of the relevant drug. The RADAR notifications, but not the FDA or pharmaceutical supplier safety notifications, included information on expected clinical outcomes, recommendations for treatment and prevention, and common strategies for diagnosis of the ADR. The recently released report from the Institute of Medicine's Committee on Assessment of the US Drug Safety System proposed changes that would improve the timing and comprehensiveness of safety notifications disseminated by the FDA.⁵⁷

For several RADAR investigations, resultant RADAR safety notifications were followed by actions that had a direct impact on the safety of pharmaceuticals in the postmarketing setting (Table 3). These actions were facilitated by market-related factors (clopidogrel rapidly replaced ticlopidine, in part for safety reasons related to lower rates of thrombotic thrombocytopenic purpura as outlined by the RADAR project), support from the Connecticut attorney general (for thalidomide-associated venous thromboembolism), and support from national health authorities (mandating intravenous epoetin alfa administration to patients with chronic kidney disease or contraindicating nevirapine-containing post–HIV-exposure prophylaxis regimens). The Institute of Medicine's Committee on Assessment of the US Drug Safety System recommended that the FDA be granted regulatory authority to affect the manner in which drugs are used or marketed when safety concerns are identified.⁵⁷

After publication of the recent Institute of Medicine report on drug safety, experts proposed funding a consortium of nonprofit pharmacovigilance centers, such as the Centers for Education and Research on Therapeutics.^10,56,58 Collaborations could build on the unique strengths of these centers. For example, the FDA may initially identify a weak safety signal by data-mining efforts and then issue a request for proposal to FDA-approved independent pharmacovigilance centers that have cooperative agreements. These centers might competitively bid on the proposed project, and the selected centers would then conduct proactive pharmacovigilance, identify additional reports of the serious adverse event, and obtain detailed case information on these individuals. The FDA and the selected pharmacovigilance centers would ultimately share safety information. Alternatively, the pharmacovigilance centers might identify signals of serious ADRs and present these findings to FDA officials at regularly scheduled meetings, similar to those that occur between the FDA and the Centers for Education and Research on Therapeutics. In addition, FDA data-mining efforts (application of statistical techniques to spontaneous ADR reports to identify potential associations between drug exposures and outcomes) would look for additional cases. Concise safety summaries may be prepared by pharmaceutical manufacturers with input from the FDA and the proactive safety centers, and dissemination may occur as revised package inserts and/or Dear Doctor letters. Academic investigators could prepare articles for submission to peer-reviewed journals that summarize reporting rates, outcomes, treatment, prophylaxis, and clinical and basic science findings. By tapping into the interests and expertise of safety centers while still maintaining control and focus, the FDA could highlight its strengths (sending rapid, nationwide feedback about new ADRs) while reducing bureaucratic burden.

Our study has limitations. First, additional sources of safety reports exist in databases maintained by institutional review boards, cooperative clinical trial groups, and insurance companies.⁵⁹ Second, physicians frequently do not respond to Dear Doctor letters or black-box advisories.^12,60-63 Third, the RADAR project focused primarily on ADRs related to drugs or drug-related devices (drug-eluting stents). In 2002, the Office of Devices and Radiologic Health of the FDA initiated the Medical Product Surveillance Network, a collaboration that resembles the proposed FDA-RADAR collaboration. However, the RADAR project, unlike the Medical Product Surveillance Network, benefits from institutional support of multidisciplinary researchers at a National Cancer Institute–designated comprehensive cancer center. As such, the RADAR effort would be difficult to replicate within the FDA. The RADAR project focuses on obtaining a small number of comprehensive safety reports from select centers and provides extensive review and analysis of the individual ADR reports to distinguish signal from noise. Finally, similar assessments of other nonprofit pharmacovigilance efforts, such as the Drug-Induced Liver Injury Network⁶⁴ and the National Registry of Drug-Induced Ocular Side Effects, should be reported.⁶⁵

In conclusion, our study identified differences in data quality for safety reports contained in databases obtained by proactive pharmacovigilance efforts conducted by the RADAR project vs the FDA, as well as in the completeness and timing of dissemination of safety notifications. Moving forward, collaborations could be developed that build on the strengths of independent nonprofit organizations that proactively conduct pharmacovigilance efforts and of safety efforts of the FDA and pharmaceutical manufacturers.

Correspondence: Charles L. Bennett, MD, PhD, MPP, 303 E Chicago Ave, Olson Pavilion, Suite 8-250, Chicago, IL 60611 (cbenne@northwestern.edu).

Accepted for Publication: September 20, 2006.

Author Contributions: Dr Bennett had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Study concept and design: Bennett, Nebeker, Yarnold, Angelotta, and Belknap. Acquisition of data: Bennett, Edwards, Angelotta, Weitzman, Tallman, and Raisch. Analysis and interpretation of data: Bennett, Nebeker, Yarnold, Tigue, Dorr, McKoy, Edwards, Hurdle, West, Lau, Angelotta, Weitzman, Belknap, Djulbegovic, Tallman, Kuzel, Benson, Evens, Trifilio, Courtney, and Raisch. Drafting of the manuscript: Bennett, Nebeker, Yarnold, Dorr, Edwards, Angelotta, Belknap, Trifilio, and Courtney. Critical revision of the manuscript for important intellectual content: Bennett, Nebeker, Yarnold, Tigue, McKoy, Edwards, Hurdle, West, Lau, Angelotta, Weitzman, Belknap, Djulbegovic, Tallman, Kuzel, Benson, Evens, Trifilio, Courtney, and Raisch. Statistical analysis: Yarnold and Trifilio. Obtained funding: Bennett and Belknap. Administrative, technical, and material support: Tigue, Dorr, Hurdle, and Angelotta. Study supervision: Bennett, McKoy, Edwards, Angelotta, Belknap, and Evens.

Financial Disclosure: None reported.

Funding/Support: Funding for this project was provided by grants 1 R01 CA102713-01 from the National Cancer Institute (principal investigator: Dr Bennett), 3 R01 CA102713-02S from the National Cancer Institute (principal investigator: Dr Bennett), HL69717-01A1 from the National Institutes of Health (principal investigator: Dr Bennett), 05-215-01-CPPB from the American Cancer Society (principal investigator: Dr Belknap), P30 CA60553 from the National Cancer Institute (Dr McKoy), RCD-02-176, TRP-02-147 from the Veterans Administration Health Services Research and Development (Dr Nebeker), K22 LM 8427-01 from the National Library of Medicine (Dr Dorr), and 5K23HL077404-02 from the National Heart, Lung, and Blood Institute (Dr Courtney).

Acknowledgment: We thank Matthew Fisher and Mark Mai for helpful comments and contributions to early drafts of the manuscript.