Lurie P, Almeida CM, Stine N, Stine AR, Wolfe SM. Financial Conflict of Interest Disclosure and Voting Patterns at Food and Drug Administration Drug Advisory Committee Meetings. JAMA. 2006;295(16):1921-1928. doi:10.1001/jama.295.16.1921
Author Affiliations: Public Citizen’s Health Research Group, Washington, DC (Drs Lurie, Almeida, and Wolfe and Mr N. Stine), and Department of Earth and Planetary Science, University of California, Berkeley (Mr A. Stine).
Context In January 2002, the US Food and Drug Administration (FDA) issued a draft guidance requiring more detailed financial conflict of interest disclosure at advisory committee meetings.
Objectives To characterize financial conflict disclosures at drug-related meetings, and to assess the relationship between conflicts and voting behavior at meetings that considered specific products.
Design and Setting Cross-sectional study using agendas and transcripts from all FDA Drug Advisory Committee meetings (2001-2004) listed on the FDA Web site.
Main Outcome Measures Conflict rates, type, and size. The relationship between having a conflict and voting in favor of the index drug was described for each voter using Mantel-Haenszel relative risks and Monte Carlo simulations; Spearman rho was used for a meeting-level analysis comparing rates of conflict with voting patterns. The impact of the removal of persons with conflicts of interest on the vote margins was also evaluated.
Results A total of 221 meetings held by 16 advisory committees were included in the study. In 73% of the meetings, at least 1 advisory committee member or voting consultant disclosed a conflict; only 1% of advisory committee members were recused. For advisory committee members (n = 1957) and voting consultants combined (n = 990), 28% (n = 825) disclosed a conflict. The most commonly specified conflicts were consulting arrangements, contracts/grants, and investments. Nineteen percent of consulting arrangements involved over $10 000, 23% of contracts/grants exceeded $100 000, and 30% of investments were over $25 000. The meeting-level analysis did not show a statistically significant relationship between conflict rates (“index conflict,” “competitor conflict,” or “any conflict”) and voting patterns, but a weak, statistically significant positive relationship was apparent for competitor conflict and any conflict in the Mantel-Haenszel analyses. The Monte Carlo analyses produced similar findings in the competitor conflict analysis only. In all 3 conflict categories, the exclusion of advisory committee members and voting consultants with conflicts would have produced margins less favorable to the index drug in the majority of meetings, but this would not have changed whether the majority favored or opposed the drug.
Conclusions Disclosures of conflicts of interest at drug advisory committee meetings are common, often of considerable monetary value, and rarely result in recusal of advisory committee members. A weak relationship between certain types of conflicts and voting behaviors was detected, but excluding advisory committee members and voting consultants with conflicts would not have altered the overall vote outcome at any meeting studied.
The Center for Drug Evaluation and Research (CDER) at the US Food and Drug Administration (FDA) approves 25 to 30 new chemical entities each year, often relying in its decision making on the advice of advisory committees composed of outside scientific experts. In 2001, 21% of these approvals were preceded by an advisory committee meeting.1
Recent high-profile cases have called attention to the possible vulnerability of committee decisions to financial conflicts of interest. For example, a prominent newspaper revealed that 2 advisory committee members for a meeting concerning the safety of troglitazone had significant ties to the sponsoring company.2 Another newspaper investigation found that at 92% of all FDA Drug Advisory Committee meetings between January 1, 1998, and June 30, 2000, at least 1 voting member had a conflict of interest. Thirty-three percent of the experts had a conflict at the 102 meetings dealing with a specific product.3 One meeting considering the cardiovascular toxicity of the diabetes drug muraglitazar failed to include a cardiologist due in part to a conflict of interest held by the committee's sole cardiologist.4
These conflicts have been tied to altered voting behavior. At a 2005 advisory committee meeting to evaluate the risks of 3 cyclooxygenase 2 (COX-2) inhibitor drugs, 93% of votes cast by individuals who had received consulting fees from 1 or more of the drugs' makers favored the drugs, compared with 56% of votes by individuals without conflicts.5,6 Exclusion of the members with conflicts would have resulted in recommendations to remove 2 of the 3 drugs from the market. However, FDA procedures leave considerable room for flexibility. For example, those with investments exceeding $100 000 in a company likely to be affected by the meeting outcome or those whose investments exceed 15% of their net worth are automatically excluded; lower levels of investment conflict can be permitted.7
Scrutiny of financial conflict of interests has intensified in recent years.8 In September 2001, the consumer advocacy group Public Citizen threatened a lawsuit against the FDA because the agency had not provided the more detailed disclosure of advisory committee members' financial interests required by the FDA Modernization Act of 1997.9 In response, the FDA conducted a survey of advisory committee members' attitudes toward disclosure. In each of 6 separate financial categories, over three quarters of advisory committee members stated that they would approve of more information being disclosed than was FDA policy at the time.10 Following the survey, the FDA drafted a guidance document that went into effect in January 2002 and outlined new conflict of interest disclosure policies for advisory committee meetings discussing specific products, but not meetings that consider general issues.11
Prior to any advisory committee meeting, both voting advisory committee members and voting consultants invited by the FDA are required to fill out form FDA 341012 detailing any financial interests they have, have had, or are currently negotiating. Financial conflict types on the form include current investments, employment, patents, and contracts/grants/cooperative research and development agreements (CRADAs), as well as consulting arrangements and speaking/writing arrangements within the last 12 months.
Any relevant conflicts of interest are reported at the beginning of the advisory committee meeting in a statement read aloud by the committee executive secretary. Occasionally, conflicts are deemed so great that advisory committee members are restricted in their activity at the meeting (usually prohibited from voting) or even recused from attendance at the meeting. Before the 2002 guidance document, the executive secretary typically declared that the committee member or voting consultant with a conflict had been granted a “waiver” to allow him or her to participate, but provided no further details. According to the draft guidance, advisory committee members and voting consultants invited by the FDA to product-specific meetings must now disclose whether the conflict involves the sponsor or a competitor company (although not the name of the competitor company) and, in dollar ranges, the conflicts listed on form FDA 3410.11 Conflicts involving an advisory committee member's employment (or the employment of his/her spouse) must also be disclosed.
Others participating in the meeting typically include a nonvoting industry advisory committee member, employees of the FDA, and experts invited by the FDA to make presentations before the committee. Companies whose products are being considered usually make presentations and bring a team of employees and consultants. Company employees, the company's consultants, and guest speakers invited by the FDA are not required to make conflict of interest statements. According to the Federal Advisory Committee Act (5 USC App II §10), each meeting includes an opportunity for public input; disclosure of financial conflicts is encouraged but not required for public participants.
This study has 4 objectives. First, it characterizes disclosed advisory committee member, voting consultant, and public participant conflicts in terms of prevalence and magnitude. Second, it describes whether current disclosure practices comply with the draft guidance. Third, it examines the impact the draft guidance has had on actual disclosures. Fourth, it examines the relationship between conflict of interest and voting behavior.
We collected data covering the period January 1, 2001, to December 31, 2004 (approximately 1 year before and 3 years after the guidance) from the meeting agendas and transcripts found on the CDER Web site.13 For each meeting, we entered the committee name, meeting date, and topic. Meetings discussing specific products were considered “product meetings,” while those discussing more general scientific issues were considered “nonproduct meetings.” We treated meetings that covered a single topic over more than 1 day as 1 meeting. Conversely, we considered a meeting that discussed multiple products sequentially over 1 day to be multiple meetings (1 for each product). For each meeting, we entered data on advisory committee members (including those recused), voting consultants to the FDA, and participants in the open public hearing who were listed in the transcript. We excluded FDA and other government employees, industry employees and consultants, guest speakers invited by the FDA, and nonvoting industry advisory committee members from our analysis. Because this study involved publicly available records, it is not considered human subjects research under 45 CFR 101 (b) (4). However, at the request of the journal, the chair of the Public Citizen's Health Research Group Institutional Review Board retrospectively reviewed this study and determined that it did not require formal review and approval.
The executive secretary's oral statement at the beginning of each meeting, obtained from the meeting transcript, provided conflict information for advisory committee members and voting consultants, including information on restrictions or recusals. The meeting transcripts also provided conflict information for participants in the open public sessions, to the extent these were disclosed.
For each attendee disclosing a conflict, we entered each conflict as a separate entry. One attendee could have multiple conflicts of various types at each meeting. Conflict type categories and monetary ranges corresponded to those specified in form FDA 3410. Specific details of the conflict, such as the company name and whether it involved the sponsor or competing company, were also collected, if available. Attendees who identified the conflict as being with the sponsor were considered to have named the company.
Questions posed to panelists for votes at advisory committee meetings varied in detail from meeting to meeting and did not adhere to a standardized format. At many meetings there were no formal votes. At others there were multiple votes that did not necessarily relate to a product. We restricted our analysis of voting behavior to meetings at which a dichotomous yes/no vote on at least 1 question relating to a specific product was cast. From these, we selected a single question per meeting by first assigning questions to 1 of 3 ranked categories: (1) questions considering whether to recommend approval of a drug, approval of an indication, or withdrawal of a drug (these included questions on whether both safety and efficacy had been established, or if the drug had a favorable risk-benefit profile); (2) questions considering whether to recommend accelerated approval of a drug; and (3) questions considering whether either safety or efficacy had been established for a drug.
For each meeting, we selected the question(s) from the category closest to the top of the ranking and if there were multiple questions within that top category, we randomly selected one of these questions for inclusion using the VassarStats integer randomization program.14 For each included question, we used the meeting transcript to determine how FDA advisory committee members and voting consultants voted, with responses coded as either favoring or not favoring the index drug. Our database was structured so that conflicts were not visible to the person entering the voting data.
We entered the data using Microsoft Access 2002 (Microsoft Corp, Redmond, Wash). For characterization of the type of conflict, data are presented on both a per-meeting and a per “person-meeting” basis, the latter defined as each appearance at a meeting by any attendee. Many attendees were thus involved in several person-meetings. We also calculated per person-meeting conflict rates for each advisory committee. We conducted 3 stratified analyses: attendee type (advisory committee member, voting consultant, public hearing participant); product-specific vs non–product-specific meetings; and meetings before or after the January 31, 2002, draft guidance. We also conducted a subanalysis of advisory committee members and voting consultants at product-specific meetings, as these were the categories covered by the draft guidance.
For the analyses of voting behavior, we took 6 distinct approaches in 2 broad categories: the meeting level (4 analyses) and the individual level (2 analyses). For the first 2 meeting-level analyses, which included all product meetings meeting our criteria, we analyzed votes by calculating as a predictor variable the percentage of advisory committee members and voting consultants having a conflict with the index product's manufacturer (“index conflict”), a competing pharmaceutical company (“competitor conflict”), and either the sponsor or a competitor (“any conflict”). This sought to address group-level effects, such as attendees with conflicts affecting even attendees without conflicts. Voting outcomes were calculated in 2 ways: continuously, as the percentage of advisory committee members and voting consultants casting votes favoring the index product, and dichotomously, as whether the majority cast votes favoring the index product, the latter intended to capture any tendency to conform to a strong majority. These analyses included abstentions. We then calculated Spearman rho for the continuous outcome and the Wilcoxon rank sum test for the dichotomous outcome.
In the third meeting-level analysis, which included unanimous meetings but not those without conflicts, we analyzed the impact of the exclusion of advisory committee members with conflicts and voting consultants on the absolute vote margin. In the fourth, we determined whether such exclusions would have altered the vote outcome from unfavorable to favorable or vice versa.
In the second broad category, we first analyzed votes on an individual level (excluding abstentions) for each meeting to examine the effect of a conflict held by a particular advisory committee member or voting consultant on his or her vote. We used the same 3 predictor variables (index conflict, competitor conflict, and any conflict) and the outcome variable was having cast a vote favoring/not favoring the index product. Votes that were unanimous or meetings in which there were no conflicts were excluded from the relevant analyses, as these cannot help assess the relationship between conflict and voting. We calculated relative risks (RRs) for each meeting with qualifying votes and combined them according to the Mantel-Haenszel method to provide a single weighted RR. For one meeting, in which one of several members with conflicts voted for the drug but no members without conflicts voted for the drug (producing an undefined RR), we calculated an RR based on the method of Jewell.15
In the sixth analysis, we used a Monte Carlo simulation16 to test the null hypothesis that panel members with conflicts have the same voting behavior as panel members without conflicts. We tested whether the votes of panel members with conflicts could be understood as random draws from a pool in which the probability of voting for the index drug was the same as that for members without conflicts (ie, that holding a conflict had no impact). Thus, within each meeting, we took the ratio of yes votes to total votes among voters without conflicts as our best estimate of the probability that a “new” voter with a conflict would vote in favor of that particular index drug. For each member with a conflict at a meeting, a random number between 0 and 1 was drawn; if this number was less than the proportion of members without conflicts who voted in favor of the drug in that particular meeting, the member with a conflict was considered to have voted in favor of the drug. This process was repeated for each member with a conflict at each meeting and the total number of yes votes was added up; this process was considered to be 1 trial. One hundred thousand such trials were run to produce a probability density function for the total number of yes votes by panel members with conflicts across all meetings, from which we determined the 2-tailed 95% confidence interval (CI). If the observed number of yes votes by panel members with conflicts fell outside this CI, the voting behavior of panel members with conflicts was considered statistically significantly different (P<.05) from that of members without conflicts.
A total of 221 meetings held by 16 advisory committees were analyzed (median, 8 meetings per committee [range, 4-44 meetings]). One hundred ten (50%) of the meetings were product-specific, and 165 (75%) took place after January 31, 2002. One hundred ninety meetings (86%) included at least 1 voting consultant, and 142 meetings (64%) had at least 1 public hearing participant. There were 1860 unique individuals accounting for 3718 person-meetings, 53% of which were for advisory committee members, 26% were for voting consultants, and 21% were for public hearing participants. At each meeting, there were medians of 9 advisory committee members (range, 1-24 members), 4 voting consultants (range, 0-19 consultants), and 3 public speakers (range, 0-66 speakers).
Twenty-two advisory committee member person-meetings (1%) led to recusals, mostly in product-specific meetings prior to January 2002. Three advisory committee members were recused from 2 meetings, and 1 was recused from 3 meetings. The transcript explained the reason for the recusal in only 4 of the 22 cases, all after January 2002: research and consulting on the products at issue; research on a competing product and consulting on the product at issue; principal investigator in a related clinical trial; and inventing a competing product. (The advisory committee member recused for inventing a competing product made a presentation in the open public hearing of the same meeting and disclosed his conflict.)
The activities of advisory committee members or voting consultants present during meetings were restricted in 1% of the total person-meetings for these 2 groups. All but one of these restrictions were in product meetings, and 14 were in meetings after January 2002. Voting consultants were restricted at a higher rate (2% of person-meetings) than advisory committee members (0.5%). Restrictions are included in the remainder of the analysis to the extent that relevant information was disclosed, but recusals are not.
A conflict of interest statement was read at all but 1 of the 221 meetings. At 73% of meetings, at least 1 conflict was declared for at least 1 advisory committee member or voting consultant; this percentage was 81% in product meetings and 66% in nonproduct meetings (Table 1). Sixty-six percent of all meetings included at least 1 advisory committee member with a conflict, and 53% of those with voting consultants had at least 1 voting consultant conflict. At 14% of meetings, 75% to 100% of advisory committee members had a financial conflict of interest, and at 22% of meetings over half the advisory committee members had such conflicts.
Twenty-eight percent of all person-meetings for advisory committee members and voting consultants had at least 1 publicly disclosed conflict, a rate that was lower in product meetings (23%) than in nonproduct meetings (34% [Table 1]). Voting consultants and advisory committee members had similar conflict rates (27% vs 29%). Conflict rates per person-meeting for advisory committee members and voting consultants varied significantly by advisory committee, from a high of 51% for the Anesthetic and Life Support Drugs Advisory Committee to a low of 10% for the Reproductive Health Drugs Advisory Committee (median of advisory committee percentages = 25%).
A total of 1556 conflicts were disclosed, but the transcript contained no details (eg, no conflict type or amount) for 573 conflicts (37%). Overall, conflicts without specific details in the transcript were more common in nonproduct meetings than in product meetings (60% of conflicts vs 14%). The remaining 983 conflicts comprised 256 advisory committee member conflicts, 208 voting consultant conflicts, and 519 public speaker conflicts. The most common conflict type for both advisory committee members and voting consultants was consulting (34% and 51%, respectively, followed by investments [30% and 19%]).
Of 771 public speakers, 454 (59%) included a disclosure statement, of which 284 (63% of those disclosing) disclosed a specific conflict, with the remaining 37% claiming no conflict. Even if every one of the public speakers who made no disclosure at all had no conflict, the overall public speaker conflict rate (37%) would still have been higher than those for advisory committee members (29%) and voting consultants (27% [Table 1]).
Contracts/grants/CRADAs (37% of conflicts, representing 191 unique contracts/grants/CRADAs) was the most common conflict-type category for public speakers, followed by consulting (21%). Of these contracts, 101 were organizational rather than personal conflicts. Of 44 public session testimonies offered by patient groups (ie, excluding public testimony delivered by professional associations, corporations, or academics) that cited organizational conflicts, 32 acknowledged receiving funds from a company whose products were potentially affected by the hearing (the others made more general disclosure statements). Employment and travel compensation comprised 14% and 9% of all public speaker conflicts, respectively.
Forty-one percent of all conflicts were with sponsoring companies. Index conflicts accounted for 22% of advisory committee member conflicts, 20% of voting consultant conflicts, and 89% of public speaker conflicts.
The percentage of conflicts in which the companies were named was 37%. Overall, company names were provided for 10% of advisory committee member conflicts, 17% of voting consultant conflicts, and 92% of public speaker conflicts. Companies were named in 43% of conflicts in product meetings and 31% of conflicts in nonproduct meetings.
The overall percentage of conflicts in which monetary values were specified was 41%, a percentage that was much higher in product meetings than in nonproduct meetings (55% vs 9%). As seen in Table 2, for advisory committee members and voting consultants, 19% of consulting conflicts were valued over $10 000, 23% of contracts/grants/CRADAs were over $100 000, 30% of investments were over $25 000, and 44% of lecturing/honoraria conflicts were over $10 000. Only 6 public speakers provided monetary values for their conflicts.
Table 3 demonstrates the impact of the January 2002 draft guidance by examining data for advisory committee members and voting consultants in product-specific meetings. The guidance has had little impact on rates of recusal, restriction, or disclosed conflict. Disclosure of conflict-type details, specifying sponsor or competitor and providing monetary value, all required by the draft guidance, were markedly improved and all approached 100% compliance. However, the draft guidance did not require the competitor company to be named and consequently disclosure of this detail dropped from 54% to 1%.
Seventy-six of 110 product-specific meetings met our inclusion criteria for the voting analysis (Table 4). In the first 2 analyses of voting behavior at the meeting level, there was no relationship between the conflict rate and voting outcome, regardless of which of the 3 predictors (index conflict, competitor conflict, or any conflict) or 2 outcomes (percentage favoring the index drug or whether there was a majority favoring the index drug) was used (all P values >.15).
In the third analysis, the exclusion of advisory committee members or voting consultants with conflicts would have produced absolute vote margins less favorable to the index drug in the majority of meetings in all 3 conflict categories. For example, exclusion of advisory committee members or voting consultants with any conflict would have produced a vote less favorable to the index drug at 31 meetings (median change, 3 votes; maximum change, 8 votes), more favorable to the drug at 8 meetings (median change, 2.5 votes; maximum change, 7 votes), and would have had no impact in 4 meetings. (Up to half of the votes in the various conflict categories were unanimously in favor of the drug.) However, in no meeting did the exclusion of advisory committee members or consultants with conflicts change whether the vote result was favorable or unfavorable toward the index drug (fourth analysis).
For the analyses using individual voting behavior, we excluded 34 meetings that were unanimous and others that had no conflict, leaving 11 eligible meetings with index conflicts, 18 with competitor conflicts, and 23 with any conflict. In the Mantel-Haenszel analysis, combined RRs (fifth analysis) were statistically significant in 2 of the 3 categories: 0.74 (95% CI, 0.39-1.39) for index conflicts, 1.20 (95% CI, 1.12-1.28) for competitor conflicts, and 1.10 (95% CI, 1.03-1.17) for any conflict (an RR >1 means that those with conflicts were more likely to vote for the index drug).
Using the Monte Carlo method in our sixth analysis, there were no statistically significant differences in the total number of votes favorable to the index drug between members with conflicts and members without conflicts in both the index conflict and any conflict cases. In the competitor conflict case, members with conflicts were statistically significantly more likely to vote in favor of the index drug than members without conflicts (expected: 29.5 positive votes [95% CI, 23.5-34.5]; observed: 36 positive votes).
We analyzed all advisory committee meetings held by CDER from 2001 through 2004 to characterize conflict of interest disclosure before and after the implementation of a new disclosure policy in January 2002 and to examine the impact on voting patterns. Based on meeting transcripts, at least 1 advisory committee member or voting consultant disclosed at least 1 financial conflict of interest at 73% of meetings and 28% of person-meetings. These conflict rates are consistent with rates found in a journalistic investigation of CDER advisory committee members covering a period preceding ours.3
The most common conflict types for advisory committee members and voting consultants were investments and consulting (together comprising 64% of advisory committee member conflicts and 70% of voting consultant conflicts). Nineteen percent of consulting conflicts were valued over $10 000, 23% of contracts/grants/CRADAs were over $100 000, 30% of investments were over $25 000, and 44% of lecturing/honoraria conflicts were over $10 000. In our view, advisory committee members with such large conflicts of interest should be recused from the meetings in which these conflicts exist, and alternative voting consultants should be found if they have conflicts of interest of similar magnitude. However, only 1% of advisory committee members were recused.
The FDA has argued that most advisory committee member conflicts are with the competitors of the sponsor company under discussion and that most conflicts reported by attendees are institutional rather than personal.8 We found that a significant proportion of conflicts (22% of advisory committee conflicts, 20% of voting consultant conflicts) were with the sponsor company. The most common conflict types for advisory committee members and voting consultants were investments and consulting, and these are usually conflicts held by individuals rather than their institutions. It is noteworthy that 7 of 16 advisory committees were able to keep their conflict rates at or below 20%, suggesting that, with assiduous effort, advisory committees largely (or even completely) free of conflict of interest can be assembled.
For advisory committee members and voting consultants in product meetings after January 2002, disclosure of the conflict type, specific conflict values, and whether the conflict involves the sponsor or a competitor company all show compliance rates of 94% or more, substantial increases over disclosure rates prior to the guidance. However, disclosure of the specific competitor name involved in the conflict has actually worsened (54% vs 1%), as this is not required by the guidance. The guidance also does not require disclosure of the reason for advisory committee member recusals, and although such disclosure has improved from being nonexistent before the guidance, it still occurs only one third of the time in meetings covered by the guidance.
The disclosed prevalence of conflicts for public hearing participants was higher than for advisory committee members and voting consultants. Eighty-nine percent of public conflicts were index conflicts, much higher than for advisory committee members or voting consultants. There were 47 instances in which a public speaker was flown by sponsors to a meeting, including one meeting in which 4 people were flown in. This demonstrates that sponsors are using the public session to influence the advisory committee meetings' outcome and may create an unbalanced representation of the public's viewpoint. Moreover, because it is not required, there is considerably less detail in the disclosures of public speakers than there is for advisory committee members and voting consultants.
In at least 32 instances, groups appearing to represent patients had received funding from a company potentially affected by the day's deliberations. This finding amplifies the growing concern that pharmaceutical industry sponsorship is becoming more prominent in nonprofit, patient advocacy groups that were once viewed as grassroots organizations independent of industry influence.17,18
Conflicts of interest could have an impact on voting patterns in at least 2 general ways: on the meeting level or on the individual level. In the first, it is possible that the overall amount of conflict in a meeting could have an influence on overall voting behavior. In this scenario, voters are influenced by the voting behaviors of others and need not have conflicts themselves. Alternatively, but still at the meeting level, voting behaviors might be different in close votes (as opposed to unanimous or nearly unanimous ones); we thus considered the vote outcome both continuously (percentage of voters favoring the index drug) and dichotomously (overall vote favored or did not favor the index drug). At the individual level, particular voters would be affected only by their personal conflicts and vote accordingly. We sought to address both of these potential mechanisms.
Both analyses were hampered by small sample sizes. Only half the meetings were product-specific, and only 76 of those posed questions amenable to analysis. Thirty-four of those were unanimous and many other meetings had no conflict, particularly in the index conflict category, which had only 11 analyzable meetings in the Mantel-Haenszel analysis, and so was greatly underpowered. Our results are thus most meaningful in the competitor conflict and any conflict categories.
Both our meeting-level and individual-level approaches produced some positive findings. For meeting-level analyses, although exclusion of advisory committee members and voting consultants would not have swayed any overall votes, they typically produced overall votes more favorable toward the index drug. (The COX-2 inhibitor meeting referred to previously5,6 occurred outside the study time frame and, in any event, was classified as a nonproduct meeting by the FDA and thus would not have been included in our study.)
For individual-level analyses, competitor conflict was associated with voting for the index drug in both analyses (Mantel-Haenszel and Monte Carlo), and any conflict was so associated in the Mantel-Haenszel analysis only. The demonstration of positive findings using 2 distinct approaches in several different analyses supports our conclusion that a weak association between certain conflicts and voting behavior exists.
There are several possible explanations for the paradox of competitor conflicts being associated with voting for the index drug in the Mantel-Haenszel and Monte Carlo analyses, while index conflicts were not. First, as mentioned, the index conflict analyses were particularly underpowered. Second, it is possible that the screening processes used by the FDA, either in selecting advisory committee members or in deciding whether to recuse or restrict members, might have produced a relatively lower level of conflict among those with index conflicts compared with those with competitor conflicts. Third, having a conflict, regardless of whether it involves the index drug or a competitor, might be a proxy for an attitude relatively favorable to industry. We were unable to control for the many other potential confounding factors that could have an impact on an advisory committee's vote (secular trends in drug approval rates, forcefulness of the advisory committee chair, press coverage, and the like).
The statistically significant Mantel-Haenszel RRs observed, while perhaps low in magnitude by the standards of randomized controlled trials, must be viewed in context. An RR of 1.1 (that calculated for any conflict) means that for each one person with any conflict, there was a 10% greater likelihood that the meeting would favor the index drug. Such a level of bias would never be tolerated in a jury (individual jurors are frequently dismissed simply for reading newspaper coverage of their trial); decisions reached by advisory committees have much greater societal impacts.
The data for this study were extracted from advisory committee meeting transcripts and are limited to self-reported conflicts. This method could underreport actual conflict rates. Thus, the observed modest decreases in conflict rates over time may represent fewer conflicts and/or less disclosure. A better method to evaluate the extent of financial conflicts of interest among advisory committee members would be to extract data from the form FDA 3410 itself (although our data reflect what those in attendance actually heard). However, the FDA turned down our Freedom of Information request for these documents.
Per-meeting conflict rates should be interpreted carefully, particularly when comparing the different attendee groups. Because there are generally more advisory committee members at meetings than voting consultants or public speakers, it is more likely that at least one of them will have a conflict.
Since the draft guidance was published, the FDA has surveyed advisory committee audience members and advisory committee members about the current conflict of interest policies. Fifty-nine percent of audience members were satisfied with these procedures (27% “in the middle” and 11% dissatisfied). Using a 1 to 7 scale (least-most), advisory committee members on average rated the procedures at 6.0 and 6.1 for fairness to advisory committee members and the public, respectively.19
With the implementation of the new guidance document, advisory committee members now disclose their financial conflicts of interest in a more systematic way. The FDA should ensure total compliance with the current draft guidance, particularly for the conflict details (conflict value, competitor status) that we found undisclosed in about 5% of conflicts covered by the draft guidance. Although the draft guidance has produced an improvement in some respects, there is room for more public disclosure beyond the guidance's requirements, such as naming the competitor company, explaining the reasons for a member's recusal, and extending the guidance to cover nonproduct meetings. Guest speakers invited by the FDA should have to make disclosures similar to those required of advisory committee members and voting consultants.
Ideally, all panels of scientific experts advising a federal decision-making body would be free of financial conflicts of interest with the affected companies. A bill to prevent advisory committee members with conflicts of interest from serving has been introduced in the House of Representatives.20 Certainly, advisory committee members who have conflicts of interest with higher dollar values should not be allowed to participate. For advisory committee members with smaller conflicts of interest, full transparency, including disclosure several days before the meeting, is necessary to allow the objective evaluation of committee decisions.
Corresponding Author: Peter Lurie, MD, MPH, Public Citizen's Health Research Group, 1600 20th St NW, Washington, DC 20009 (firstname.lastname@example.org).
Author Contributions: Dr Lurie had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Lurie, Almeida, N. Stine, Wolfe.
Acquisition of data: Lurie, Almeida, N. Stine.
Analysis and interpretation of data: Lurie, Almeida, N. Stine, A. Stine, Wolfe.
Drafting of the manuscript: Lurie, Almeida.
Critical revision of the manuscript for important intellectual content: Lurie, Almeida, N. Stine, A. Stine, Wolfe.
Statistical analysis: Lurie, Almeida, N. Stine, A. Stine.
Administrative, technical, or material support: Wolfe.
Study supervision: Lurie, Wolfe.
Financial Disclosures: None reported.
Acknowledgment: Willam F. McCarthy, PhD, of the Maryland Research Institute provided advice on statistical approaches to measuring the impact of conflicts on voting behavior (he received no compensation for this advice). Meredith Larson of Public Citizen's Health Research Group entered the data on voting behavior.