Association Between Financial Incentives and Participant Deception About Study Eligibility

IMPORTANCE Offers of payment for research participation are ubiquitous but may lead prospective participants to deceive about eligibility, jeopardizing study integrity and participant protection. To date, neither the rate of payment-induced deception nor the influence of payment amount has been systematically studied in a nationally representative randomized survey experiment. OBJECTIVES To estimate payment-associated deception about eligibility for an online survey and to assess whether there is an association between payment amount and deception frequency. DESIGN, SETTING, AND PARTICIPANTS Randomized, 7-group survey experiment. Data were collected in March 2018 and analyzed from March to August 2018. The setting was a nationally representative online survey among US adults drawn from the GfK KnowledgePanel. INTERVENTIONS Varying payment amounts for participation ($5, $10, or $20 cash equivalent) and direction of eligibility criterion (having received or not having received an influenza vaccination in the past 6 months). out a series of online experiments using Amazon’s Mechanical Turk platform and determined that participants often misrepresent relevant characteristics to meet the eligibility criteria.


Introduction
Offering payment for research participation is often both practically necessary and ethically fraught.
Relying on participant altruism or the prospect of direct medical benefit may be insufficient to achieve recruitment goals, necessitating offers of reimbursement, compensation, and incentives to encourage enrollment and retention. 1 Concern about undue influence stemming from such payment can be minimized by the requirement that institutional review boards (IRBs) approve research only when its balance of risks and benefits is objectively reasonable. 2 However, even if ethical concerns can be managed, an important pragmatic concern persists: anecdotal evidence and common sense suggest that payment may encourage participants to mislead investigators about their eligibility or other aspects of participation by concealing or fabricating information to secure payment.
Intentional misrepresentation of this nature, otherwise known as deception, may have serious consequences for participant safety and the validity of research results. 3 Systematic, prospective data regarding whether and to what extent participants will deceive when offered payment in a real study are lacking. [4][5][6][7][8][9] Therefore, we designed a randomized survey experiment in a nationally representative sample of US adults participating in an online panel, with the objectives of quantifying payment-associated deception about eligibility and evaluating whether variable payment amounts are associated with variable rates of deception.

Design
The University of Pennsylvania IRB approved the study as involving no greater than minimal risk to participants. To avoid influencing participant behavior, the IRB approved an alteration of informed consent to omit a statement disclosing the study's true objective of evaluating payment-associated deception. Instead, at the end of the survey, each participant was told that "results will be used to help us learn how offering payment affected people's responses to survey questions. This will help better understand how payment should be used in future research studies." Primary data collection for this project was conducted in collaboration with GfK Knowledge Networks, an online survey company. GfK is a member of the American Association for Public Opinion Research (AAPOR) Transparency Initiative and follows all of the association's reporting guidelines. GfK uses probability-based recruitment procedures to populate its nationally representative survey panel, known as the GfK KnowledgePanel (GfK Custom Research North America), and standing KnowledgePanel members are randomly selected to participate in ongoing studies. Selected members receive an email invitation with a unique password permitting a single use; GfK provides internet access and connected devices to members who lack these tools. The GfK KnowledgePanel allows the generation of fully representative samples to produce statistically valid inferences for study populations. 10 For this study, GfK invited a random sample to complete a 1-minute online questionnaire about attitudes toward vaccines. The email invitation sent to prospective participants stated the amount of payment offered but not the eligibility criteria, which were disclosed only on clicking the survey link.
Participants were randomly assigned into 7 groups, and the survey was held open from March 2 to 13, 2018, until there were 325 respondents in each group. Control group participants (group 1) were told that they were eligible if they had received any vaccine in their lifetime, a condition intended to achieve near-universal eligibility and avoid any incentive to provide either affirmative or negative answers to a subsequent question specifically about influenza vaccination status (the primary outcome measure). In the 6 experimental groups, participants were exposed to the following 2 interventions: (1) direction of the self-reported eligibility criterion (ie, whether having received an influenza vaccination in the past 6 months made them eligible or ineligible) and (2) amount of incentive. Unlike group 1, these participants were told that they were eligible if they had After Q1, all participants who self-reported eligibility for the survey were presented with 3 statements selected from the Parent Attitudes About Childhood Vaccines (PACV) survey tool. 11 They were asked to state their level of agreement on a 5-point scale (1 is strongly agree, 2 is agree, 3 is not sure, 4 is disagree, and 5 is strongly disagree) with the following statements: "Children get more shots than are good for them" (Q2), "I believe that many of the illnesses that shots prevent are severe" (Q3), and "It is better for a child to develop immunity by getting sick than to get a shot" (Q4). These statements were included to provide content to the survey so that it would not end immediately after Q1, as well as to test an exploratory hypothesis that deception about eligibility may be associated with biased responses to other survey questions. More specifically, if deception was detected in the A and B groups, we sought to test whether reported vaccine attitudes in these groups would differ from those expressed by similar participants in group 1. The objective was not to assess vaccine attitudes in the study population but rather to evaluate whether participants who were deceptive about their eligibility might also adjust their responses to subsequent survey questions, perhaps based on speculation as to how a truly eligible participant might respond.

JAMA Network Open | Ethics
GfK provided participants' demographic data along with survey results. KnowledgePanel members self-report demographics in a multiple-choice format.

Statistical Analysis
Data were analyzed from March to August 2018. We performed χ 2 tests within pairs of groups (group 2A v 2B, group 3A v 3B, group 4A v 4B) to compare the proportions of participants who reported influenza vaccination in the past 6 months at each of the 3 payment levels ($5, $10, or $20). Sample sizes (325 respondents per group) were calculated to achieve greater than 90% power to detect a true difference of at least 13 percentage points in reported influenza vaccination rates within each A and B pair (2-sided test, α = .05). We also tested for an association between payment amount and reported influenza vaccination in the A and B groups separately using binary logistic regression.
To estimate the percentage of truly ineligible participants in each experimental group who answered the eligibility question (Q1) deceptively, we assumed that, in the absence of incentives, the distribution of responses in each experimental group would mirror that in the control group. For respondents incentivized to report that they had received an influenza vaccine (A groups), we calculated the percentage of respondents answering deceptively as follows: (% reporting vaccination in the experimental group − % reporting vaccination in the control group) / % reporting no

JAMA Network Open | Ethics
Financial Incentives and Participant Deception About Study Eligibility vaccination in the control group. Conversely, for respondents incentivized to report that they had not received an influenza vaccine (B groups), we calculated the percentage of respondents answering deceptively as follows: (% reporting no vaccination in the experimental group − % reporting no vaccination in the control group) / % reporting vaccination in the control group. We assumed that all participants who self-reported ineligibility to participate in the survey were truthful because that response led to no payment. Therefore, they were excluded from these calculations.
To evaluate responses to PACV statements, we developed a single index score for each eligible participant by averaging the responses to the 3 questions after reverse scoring Q3. 12 Individual index scores were then averaged by group to create a single mean score for those in group 1 reporting recent influenza vaccination, those in group 1 reporting no recent influenza vaccination, those in the A groups reporting recent influenza vaccination (with incentive to do so), and those in the B groups reporting no recent influenza vaccination (with incentive to do so). The mean scores could range from 1 (reflecting maximal vaccine hesitancy) to 5 (reflecting minimal vaccine hesitancy). The mean scores were compared across groups using paired t tests.
For this study, design weights were computed to reflect the selection probabilities of assignees. Based on the estimated true proportion of recent influenza vaccination in the study population (52.2%), estimated proportions of participants who were actually not eligible to participate but who answered deceptively about their eligibility ranged from 10.5% to 22.8% in the 6 experimental groups (Table 2). There was no evidence of a dose-response relationship with increasing payment amounts either among participants incentivized to report or not report recent influenza vaccination.
In both the A and B groups, self-reported influenza vaccination rates remained almost flat across increasing incentives to deceive, as noted earlier (Figure).
The mean score on PACV questions among group 1 (control) participants reporting recent influenza vaccination was 3.87 (95% CI, 3.75-4.00) of 5; among group 1 participants reporting no

Discussion
In this randomized survey experiment conducted in a nationally representative sample of US adults, we observed evidence of substantial payment-associated deception regarding self-reported eligibility even at the lowest amount of payment offered. However, we did not observe evidence that higher payment was associated with more frequent deception. These findings provide novel insight regarding the potential association between offers of payment and deception by participants.  Our study was motivated by gaps in the empirical literature on payment-induced deception.
Prior studies have raised concern that payment may lead research participants to mislead investigators, but the studies are limited to identifying cases in the published literature, retrospective self-report, hypothetical projections of potential behavior, and nonrepresentative populations. For example, Lee et al 3 identified 103 instances of deception, including concealment, fabrication, drug  Respondents in groups 2B, 3B, and 4B had a financial incentive to respond that they had not received a recent influenza vaccine. P values for the 7-way comparison and for all 2-way comparisons between A groups and B groups were less than .001, indicating statistically significant levels of deception in the intervention groups. In contrast, the P value for the 3-way comparison across the A groups was .81 and across the B groups was .75, providing no statistically significant evidence of dose response based on increasing payment amount.  Our study builds on these findings by analyzing deception in a broader population sample offered payment to participate. The randomized nature of the experiment allowed us to assess the association of payment, payment amount, and direction of the incentive with aggregate deception about self-reported eligibility to participate. Although this study was carried out in the context of survey research, the data offer a rigorous and internally valid signal about the potential influence of payment on deception by research participants. Our main finding that there was significant paymentassociated deception is broadly relevant to the work of all scientists who carry out data collection with human participants, including clinical trialists using survey methods to evaluate participantreported outcomes or otherwise supplement clinical data, as well as those conducting trials of interventions relying on self-reported adherence, because these data may be influenced by participant desire to continue receiving study payment. 14,15 Our results offer a compelling reason to carry out similar research in the clinical trial context, where payment amounts and other benefitssuch as access to potentially desirable investigational medicines-are often more substantial than typically associated with survey research. They also suggest that trialists should use objective rather than self-reported eligibility (and other) criteria whenever possible and that deception regarding eligibility may be a particular concern for pragmatic trials enrolling participants on the basis of selfreported behaviors and conditions.

JAMA Network Open | Ethics
The absence of higher rates of deception at higher payment amounts in our study may reflect that some proportion of individuals will not deceive even when the financial reward of doing so is great or that some individuals have "tipping points" higher than the amounts we offered. Addressing these questions about dose response is important because it may be that attempts to avoid deception by keeping offers of payment low will be ineffective (based on our finding that deception may occur even at low payment amounts) and that high payment offers may have the benefit of speeding recruitment without posing greater risk to study integrity (based on our finding that deception may not increase at higher payment amounts). In either case, as noted in the previous paragraph, adopting objective eligibility criteria should be a goal whenever possible.

JAMA Network Open | Ethics
A potential result of deception about eligibility is that the validity of other information reported by participants may also be affected. We hypothesized that individuals who deceived about eligibility might seek to answer subsequent questions regarding vaccine attitudes in a biased manner according to their estimate of how someone who was truly eligible might answer. However, it is unclear whether participants engaged in such role playing. Based on the absence of statistical significance on comparative mean vaccine attitude scores, we found no evidence that deception about eligibility by participants who were incentivized to report having either received or not received a recent influenza vaccine influenced responses to other survey questions.

Limitations and Research Agenda
The primary limitation of our study is that it was conducted in the context of a survey relying on selfreported eligibility; therefore, it cannot directly answer questions about the possibility and influence of payment-induced deception in the clinical trial context. In addition, there was no comparison of responders (59.4%) with nonresponders, which may limit generalizability of these findings. In contrast to some clinical trials that may entail high levels of burden and risk, participants in this brief online survey were asked to undertake little burden and no risk for proportionately high payment.
They could also engage in deception without fear of being caught or of incurring physical risk by evading eligibility criteria intended to protect them. It is unclear whether clinical trial participants will be more inclined to deceive as a result of typically higher offers of payment in that setting or less inclined given greater potential consequences for themselves or investment in the outcome of the research.
Our study design cannot rule out the possibility that reported rates of recent influenza vaccination varied according to the eligibility criteria not as a result of payment-associated deception but rather as a result of participant desire to report eligibility in a way that would assist researchers (a social desirability or "helpfulness" bias). However, there is no evidence to suggest that individuals are eager to participate in research for which they are not eligible in the absence of incentive payments.
Another limitation is that we tested only 3 payment levels, which could obscure signals that may have become evident by evaluating more payment amounts at tighter increments. In addition, our assessment of whether payment was associated with different information provided by research participants after deception about eligibility was limited to an exploratory measure based on a small number of additional questions regarding vaccine attitudes not validated for that particular purpose.
One pressing research need is to replicate these findings among participants in clinical trials that have some self-reported criteria for initial and continuous eligibility. Further research is also needed to understand whether a dose-response relationship between payment amount and deception rates might be seen in other contexts or at different amounts, as well as to systematically assess the downstream influence of the inclusion of individuals in research who were deceptive about their eligibility. As further data are developed, research evaluating the effectiveness of approaches to minimize payment-induced deception will also be important.

Conclusions
Ethical concerns about undue influence have dominated discussions about payment for research participation, but investigators and IRBs also should be aware of the potential for payment to interfere with a study's scientific integrity and participant protections. Our study found that as many as 10.5% to 22.8% of ineligible participants were motivated to deceive about their eligibility to enroll in an online survey to secure payment, although higher rates of payment were not associated with higher rates of deception. Because offering payment can be an important and ethical tool for