[Skip to Content]
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Figure.  Summary of Delphi Consensus Process
Summary of Delphi Consensus Process

A summary of the actions taken during each step of the Delphi consensus process is shown. BMS indicates breast milk substitute.

Table 1.  Definition of Consensus for the Delphi Process
Definition of Consensus for the Delphi Process
Table 2.  Abbreviated Checklist of Criteria Specific for Clinical Trials of BMS
Abbreviated Checklist of Criteria Specific for Clinical Trials of BMS
1.
Rollins  NC, Bhandari  N, Hajeebhoy  N,  et al; Lancet Breastfeeding Series Group.  Why invest, and what it will take to improve breastfeeding practices?   Lancet. 2016;387(10017):491-504. doi:10.1016/S0140-6736(15)01044-2 PubMedGoogle ScholarCrossref
2.
Willumsen  J. Regulation of marketing of breastmilk substitutes: biological, behavioural and contextual rationale. World Health Organization, Department of Nutrition for Health and Development. Published July 2013. Accessed December 1, 2019. https://www.who.int/elena/bbc/regulation_breast-milk_substitutes/en/
3.
Victora  CG, Bahl  R, Barros  AJ,  et al; Lancet Breastfeeding Series Group.  Breastfeeding in the 21st century: epidemiology, mechanisms, and lifelong effect.   Lancet. 2016;387(10017):475-490. doi:10.1016/S0140-6736(15)01024-7 PubMedGoogle ScholarCrossref
4.
Abrams  SA.  Is it time to put a moratorium on new infant formulas that are not adequately investigated?   J Pediatr. 2015;166(3):756-760. doi:10.1016/j.jpeds.2014.11.003 PubMedGoogle ScholarCrossref
5.
Belamarich  PF, Bochner  RE, Racine  AD.  A critical review of the marketing claims of infant formula products in the United States.   Clin Pediatr (Phila). 2016;55(5):437-442. doi:10.1177/0009922815589913 PubMedGoogle ScholarCrossref
6.
Boyle  RJ, Ierodiakonou  D, Khan  T,  et al.  Hydrolysed formula and risk of allergic or autoimmune disease: systematic review and meta-analysis.   BMJ. 2016;352:i974. doi:10.1136/bmj.i974 PubMedGoogle ScholarCrossref
7.
Hughes  HK, Landa  MM, Sharfstein  JM.  Marketing claims for infant formula: the need for evidence.   JAMA Pediatr. 2017;171(2):105-106. doi:10.1001/jamapediatrics.2016.3837 PubMedGoogle ScholarCrossref
8.
Kent  G.  Regulating fatty acids in infant formula: critical assessment of U.S. policies and practices.   Int Breastfeed J. 2014;9(1):2. doi:10.1186/1746-4358-9-2 PubMedGoogle ScholarCrossref
9.
Tijhuis  M, Doets  E, Vonk Noordegraaf-Schouten  M.  Extensive literature search and review as preparatory work for the evaluation of the essential composition of infant and follow-on formulae and growing-up milk.   EFSA Supporting Publications. 2014;11(1):551E. doi:10.2903/sp.efsa.2014.EN-551 Google ScholarCrossref
10.
Chalmers  I, Glasziou  P.  Avoidable waste in the production and reporting of research evidence.   Lancet. 2009;374(9683):86-89. doi:10.1016/S0140-6736(09)60329-9 PubMedGoogle ScholarCrossref
11.
Lundh  A, Lexchin  J, Mintzes  B, Schroll  JB, Bero  L.  Industry sponsorship and research outcome.   Cochrane Database Syst Rev. 2017;2(2):MR000033. doi:10.1002/14651858.MR000033.pub3PubMedGoogle Scholar
12.
Nestle  M.  Corporate funding of food and nutrition research: science or marketing?   JAMA Intern Med. 2016;176(1):13-14. doi:10.1001/jamainternmed.2015.6667 PubMedGoogle ScholarCrossref
13.
Rasmussen  K, Bero  L, Redberg  R, Gøtzsche  PC, Lundh  A.  Collaboration between academics and industry in clinical trials: cross sectional study of publications and survey of lead academic authors.   BMJ. 2018;363:k3654. doi:10.1136/bmj.k3654 PubMedGoogle ScholarCrossref
14.
World Health Organization. Global strategy for infant and young child feeding. Published 2003. Accessed December 1, 2019. https://www.who.int/nutrition/topics/global_strategy/en/
15.
Jones  J, Hunter  D.  Consensus methods for medical and health services research.   BMJ. 1995;311(7001):376-380. doi:10.1136/bmj.311.7001.376 PubMedGoogle ScholarCrossref
16.
Sinha  IP, Smyth  RL, Williamson  PR.  Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies.   PLoS Med. 2011;8(1):e1000393. doi:10.1371/journal.pmed.1000393 PubMedGoogle Scholar
17.
Jünger  S, Payne  SA, Brine  J, Radbruch  L, Brearley  SG.  Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review.   Palliat Med. 2017;31(8):684-706. doi:10.1177/0269216317690685 PubMedGoogle ScholarCrossref
18.
Powell  C.  The Delphi technique: myths and realities.   J Adv Nurs. 2003;41(4):376-382. doi:10.1046/j.1365-2648.2003.02537.x PubMedGoogle ScholarCrossref
19.
Harman  NL, Bruce  IA, Callery  P,  et al.  MOMENT—Management of Otitis Media with Effusion in Cleft Palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey.   Trials. 2013;14(1):70. doi:10.1186/1745-6215-14-70 PubMedGoogle ScholarCrossref
20.
Guyatt  GH, Oxman  AD, Kunz  R,  et al.  GRADE guidelines, 2: framing the question and deciding on important outcomes.   J Clin Epidemiol. 2011;64(4):395-400. doi:10.1016/j.jclinepi.2010.09.012 PubMedGoogle ScholarCrossref
21.
Leonardi-Bee  J, Garcia-Larssen  V, Jarrold  K,  et al. Risk of bias and study conduct in clinical trials of breastmilk substitutes: a systematic review. Published 2018. Accessed December 1, 2019. http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018091928
22.
World Health Organization. International code of marketing of breast-milk substitutes. Accessed April 9, 2020. https://www.who.int/nutrition/publications/code_english.pdf
23.
Baggerly  K.  Disclose all data in publications.   Nature. 2010;467(7314):401. doi:10.1038/467401b PubMedGoogle ScholarCrossref
24.
Doshi  P, Goodman  SN, Ioannidis  JP.  Raw data from clinical trials: within reach?   Trends Pharmacol Sci. 2013;34(12):645-647. doi:10.1016/j.tips.2013.10.006 PubMedGoogle ScholarCrossref
25.
Ioannidis  JPA, Greenland  S, Hlatky  MA,  et al.  Increasing value and reducing waste in research design, conduct, and analysis.   Lancet. 2014;383(9912):166-175. doi:10.1016/S0140-6736(13)62227-8 PubMedGoogle ScholarCrossref
26.
Knip  M, Åkerblom  HK, Al Taji  E,  et al; Writing Group for the TRIGR Study Group.  Effect of hydrolyzed infant formula vs conventional formula on risk of type 1 diabetes: the TRIGR randomized clinical trial.   JAMA. 2018;319(1):38-48. doi:10.1001/jama.2017.19826 PubMedGoogle ScholarCrossref
27.
US Food and Drug Administration. Electronic Code of Federal Regulations, Title 21, Chapter I, Subchapter B, Part 106, §106.96 Requirements for quality factors for infant formulas and §106.121 Quality factor assurances for infant formulas. US Department of Health and Human Services. Accessed April 22, 2019. https://www.ecfr.gov
28.
Eskander  M, Nguyen  L. Protocol for a growth and tolerance clinical trial for healthy term newborn infants: a guidance document for infant formula manufacturers. Bureau of Nutritional Sciences, Health Canada; October 22, 2019.
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Views 2,565
    Citations 0
    Original Investigation
    May 11, 2020

    Guidance for the Conduct and Reporting of Clinical Trials of Breast Milk Substitutes

    Author Affiliations
    • 1National Heart and Lung Institute, Imperial College London, London, United Kingdom
    • 2Bureau of Nutritional Sciences, Food Directorate, Health Canada, Ottawa, Ontario, Canada
    • 3First Steps Nutrition Trust, London, United Kingdom
    • 4Scientific and Technical Advisory Group on the Inappropriate Promotion of Foods for Infants and Young Children, World Health Organization, Geneva, Switzerland
    • 5Department of Behavioral Health and Nutrition, University of Delaware, Newark
    • 6Center for Human Nutrition, The Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland
    • 7Department of Public Health Nutrition Standards, Food Standards Australia New Zealand, Canberra, Australia
    • 8Section of Neonatal Medicine, Imperial College London, London, United Kingdom
    • 9International Board of Certified Lactation Consultant Examiners, Fairfax, Virginia
    • 10Children’s Hospital, Helsinki University Hospital, University of Helsinki, Helsinki, Finland
    • 11Research Program for Clinical and Molecular Metabolism, Faculty of Medicine, University of Helsinki, Helsinki, Finland
    • 12Medical Statistics, University of Nottingham, Nottingham, United Kingdom
    • 13Research, British Medical Journal, London, United Kingdom
    • 14Department of Neurology, Harvard Medical School, Cambridge, Massachusetts
    • 15Allergy and Lung Health Unit, Melbourne School of Population and Global Health, University of Melbourne, Melbourne, Victoria, Australia
    • 16Centre for Reviews and Dissemination, University of York, York, United Kingdom
    • 17Division of Obstetrics, Gynaecology and Neonatology, University of Sydney, Sydney, New South Wales, Australia
    • 18Department of Food Safety, Federal Institute for Risk Assessment, Berlin, Germany
    • 19Mother and Infant Research Unit, University of Dundee School of Nursing and Health Sciences, Dundee, United Kingdom
    • 20Nutrition Programs, Food and Drug Administration, Silver Spring, Maryland
    • 21Department of Nutrition, University of California, Davis, Davis
    • 22Centre of Evidence-Based Dermatology, University of Nottingham, Nottingham, United Kingdom
    JAMA Pediatr. Published online May 11, 2020. doi:10.1001/jamapediatrics.2020.0578
    Key Points

    Question  What is the best way to ensure the validity of clinical trials of breast milk substitutes while protecting trial participants?

    Findings  Through a Delphi consensus project, guidance was developed to address issues specific to trials of breast milk substitutes assessing growth and tolerance, as well as trials of breast milk substitutes with other objectives. This consensus guidance summarizes best practice for the design, conduct, analysis, and reporting of trials of breast milk substitutes.

    Meaning  Use of this guidance, in conjunction with existing clinical trial regulations, should enhance the quality and validity of trials of breast milk substitutes, protect trial participants, and support the evidence base for infant nutrition recommendations.

    Abstract

    Importance  Breast milk substitutes (BMS) are important nutritional products evaluated in clinical trials. Concerns have been raised about the risk of bias in BMS trials, the reliability of claims that arise from such trials, and the potential for BMS trials to undermine breastfeeding in trial participants. Existing clinical trial guidance does not fully address issues specific to BMS trials.

    Objectives  To establish new methodological criteria to guide the design, conduct, analysis, and reporting of BMS trials and to support clinical trialists designing and undertaking BMS trials, editors and peer reviewers assessing trial reports for publication, and regulators evaluating the safety, nutritional adequacy, and efficacy of BMS products.

    Design, Setting, and Participants  A modified Delphi method was conducted, involving 3 rounds of anonymous questionnaires and a face-to-face consensus meeting between January 1 and October 24, 2018. Participants were 23 experts in BMS trials, BMS regulation, trial methods, breastfeeding support, infant feeding research, and medical publishing, and were affiliated with institutions across Europe, North America, and Australasia. Guidance development was supported by an industry consultation, analysis of methodological issues in a sample of published BMS trials, and consultations with BMS trial participants and a research ethics committee.

    Results  An initial 73 criteria, derived from the literature, were sent to the experts. The final consensus guidance contains 54 essential criteria and 4 recommended criteria. An 18-point checklist summarizes the criteria that are specific to BMS trials. Key themes emphasized in the guidance are research integrity and transparency of reporting, supporting breastfeeding in trial participants, accurate description of trial interventions, and use of valid and meaningful outcome measures.

    Conclusions and Relevance  Implementation of this guidance should enhance the quality and validity of BMS trials, protect BMS trial participants, and better inform the infant nutrition community about BMS products.

    Introduction

    Breast milk substitutes (BMS) are important nutritional products for infants who are not receiving breast milk. Most North American and European infants are exposed to BMS during their first year.1 Infants are sensitive to health effects of BMS owing to their early stage of development when consuming it and their potentially high level of BMS exposure when BMS are used as a sole source of nutrition. The potential association of BMS with population health is therefore greater than for many other nutritional products, and BMS need a scientifically robust evidence base so that caregivers and health care professionals can make informed feeding choices.2,3 Clinical trials that test BMS safety and evaluate changes in BMS composition or formulation are the foundation of this evidence base. Several groups have questioned the methodological quality of published BMS trials and, in turn, the robustness of their conclusions.4-9 Specific issues identified include risk of bias related to trial methods, lack of independence from BMS manufacturers, and less stringent regulatory oversight compared with drug trials.6,10-13 In BMS trials in which some infants are breastfed at enrollment, trials may also be failing to support the establishment and maintenance of breastfeeding in participants.6,14 These concerns, and the specific issues related to designing BMS trials that answer relevant scientific questions without undermining breastfeeding, suggest a need for new guidance for BMS trials.

    We undertook a Delphi consensus to develop new standards for BMS trials. The new standards aim to support trialists in designing, conducting, analyzing, and reporting trials, as well as support regulators, critical appraisers, and reviewers in evaluating BMS trial reports. The guidance relates to intervention trials of BMS in infants enrolled prior to their first birthday, designed to demonstrate adequate growth and tolerance or other objectives. It is designed to complement other guidance such as that published by the US Food and Drug Administration or the European Food Standards Agency, Good Clinical Practice, or Consolidated Standards of Reporting Trials (CONSORT). Further details are summarized in the eAppendix in the Supplement.

    Methods

    A 3-step Delphi consensus process was used to derive new methodological guidance for BMS trials. This Delphi consensus was undertaken between January 1 and September 30, 2018, with a consensus meeting on October 24, 2018. This method enables aggregation of the anonymous and independent opinions of an expert panel to reach consensus on agreed criteria.15,16 It is a systematic process of sequential rounds used to resolve clinical problems for which evidence is limited and the opinion of stakeholders is important but might be conflicting.17,18 We invited experts in BMS trials designed to demonstrate adequate growth and tolerance, BMS trials with other objectives such as supporting health and nutrition claims, BMS regulation, trial methods, breastfeeding support, infant feeding research, and medical publishing. Experts were identified through literature review and consultation with others working in these fields. Initial criteria were developed through review of existing clinical trial and BMS guidance, regulatory standards, and critical appraisals. We conducted 3 rounds of email questionnaires to generate, score, and refine criteria (Figure) and used published requirements for consensus19 (Table 1).20 The UK Health Research Authority was consulted and confirmed that this study did not require approval by a research ethics committee because it was not considered to be research on patients. Informed consent was obtained by email from all study participants. The protocols for this Delphi process and an associated systematic review are registered on PROSPERO (CRD42018091928).21 See the eAppendix in the Supplement for further details.

    Each round of the Delphi survey was piloted by the study team prior to initiation, and experts were given 3 to 4 weeks to complete each round, with regular prompts to maximize participation. The study team (K.J., B.H., and R.J.B.) was not part of the Delphi process and did not vote on the criteria.

    Delphi Round 1

    Experts were asked to rate the importance of criteria that formed the initial guidance, using the GRADE (Grading of Recommendations, Assessment, Development and Evaluations) scale: a score of 1 to 3 corresponds to “not important,” 4 to 6 to “important but not critical,” and 7 to 9 to “critical.”20 If experts thought they could not comment on a criterion, they selected “unable to score.” Experts were also invited to provide free text comments, suggest adjustments to the wording of criteria, or suggest new criteria, and to comment on the scope of the guidance. The study team (K.J. and R.J.B.) summarized scores, anonymized comments, and classified criteria as essential, recommended, consensus out, or no consensus, as described in Table 1.20 All criteria other than consensus out were carried forward to round 2, together with proposed new criteria, proposed edits to existing wording, and any proposed merging or splitting of criteria. All changes or new criteria were highlighted in round 2, together with the anonymous comments from round 1.

    Delphi Round 2

    Experts were asked to rate the importance of the revised criteria. For criteria repeated from round 1, experts were shown the consensus outcome and their own scoring. Experts were asked to reevaluate the criteria in light of the consensus outcome and propose further edits or comments, but could not add new criteria at this stage. Responses were classified and criteria modified in the same way as for round 1, but criteria that were still classified as no consensus were removed after round 2.

    Pilot Systematic Review

    A pilot systematic review of a sample of recent BMS trials was undertaken by the study team (K.J., B.H., and R.J.B.) to evaluate adherence to the preliminary criteria generated in round 2. The findings were summarized for experts before round 3.

    Industry Consultation

    Revised guidance after round 2 was sent to BMS industry representatives for comment on the feasibility and relevance of the proposed criteria. Breast milk substitute industry representatives were not invited to score criteria, but their feedback was collated and added to the guidance document to review in round 3.

    Delphi Round 3

    Experts were asked to review the revised criteria arising from round 2, together with the findings of the systematic review and anonymized industry feedback. Experts were given an opportunity to suggest removal, merging, splitting, or changes to criteria or their ratings. Through analysis of round 3 responses, essential and recommended consensus criteria were finalized. Criteria for which the response to industry comments was unresolved or conflicting comments were received during round 3 were highlighted for discussion during the consensus meeting.

    Consensus Meeting

    Experts were invited to attend the final consensus meeting in person or by web link. The meeting focused on criteria for which consensus had not yet been achieved. Each relevant criterion was discussed until agreement was reached to retain, edit, or remove it from the guidance. The meeting was facilitated by an independent nonvoting chair with experience in BMS regulation, Peter Aggett, MD, PhD (University of Lancaster, UK). Experts were given the opportunity to comment on each criterion, and for those who wished to raise issues anonymously, opportunities were given to submit questions or comments prior to or during the meeting, to be raised by the chair on their behalf. The study team (K.J., B.H., and R.J.B.) circulated minutes after the consensus meeting, and the meeting was recorded. Any final edits and formatting changes were agreed on through email exchange after the meeting.

    Trial Participant and Ethics Committee Consultation

    After the consensus meeting, the final criteria were sent to parents of infants who had participated in a BMS trial and to the London Riverside National Health Service Research Ethics Committee for formal comment.

    Results
    Setting and Participants

    This Delphi consensus was undertaken between January 1 and September 30, 2018, with a consensus meeting on October 24, 2018. Twenty-eight experts were contacted and 23 participated in at least 1 stage of the Delphi survey: 6 clinical trialists, 9 experts in BMS regulation, 5 clinical trial methodologists, 2 experts in breastfeeding support and infant feeding research, and 1 medical journal editor. Experts were affiliated with institutions in Europe, North America, and Australasia. Sixteen of the experts were able to contribute to the final consensus meeting. Six of 7 invited BMS industry representatives provided comments between June 1 and September 30, 2018, comprising representatives from Danone Nutricia, Nestlé Nutrition, Abbott Nutrition, Hipp, Friesland Campina, and Dairy Goat Co-operative.

    Delphi Survey Results

    Initial guidance for round 1 included 73 criteria derived from clinical trials, BMS and breastfeeding guidance, and appraisals of the BMS trial literature. General comments raised in the BMS industry consultation related to overlap with existing clinical trial guidance, the value of study designs other than randomized clinical trials, definitions of BMS and other nutritional products, and the title and scope of the guidance. Preliminary findings from the pilot phase of the systematic review, which evaluated a sample of 61 recent BMS trials, were a lack of independently funded studies and a high prevalence of nonregistered trial outcomes highlighted in publication abstracts.

    The outcomes at each stage of the Delphi process are summarized in the Figure. The final guidance comprises 54 essential criteria (eTable 1 in the Supplement) and 4 recommended criteria (eTable 2 in the Supplement). Of these, 18 criteria are specific to BMS trials, which are summarized as a checklist in Table 2.20,22 The 58 criteria are elaborated in the eAppendix in the Supplement, including a list of definitions for the key terms used. Key issues discussed at the consensus meeting centered around 4 themes.

    Theme 1: Research Integrity and Reporting Transparency

    Experts stressed the importance of transparency of trial conduct and reporting: that all BMS trials are registered; that trial outcomes are made publicly available, in line with current initiatives in medical research that aim to increase access to original data sets23-25; and that oversight of trial conduct, analysis, and reporting, including adverse event coding, is independent. Independence was conceptualized as usually meaning that trial oversight was the responsibility of the principal investigator, and should not be the responsibility of an employee of the BMS industry or any other entity with a potential financial interest in the outcome of the trial. It was thought that in-house industry-led statistical planning and analysis is not appropriate unless there is complete transparency owing to audit by regulators or full publication of participant-level outcome data, such that all statistical analyses can be independently verified. When blinded BMS products are used as trial interventions, industry collaboration may be necessary, but trialists and BMS manufacturers should avoid creating financial dependencies and avoid industry control of trial conduct, analysis, or reporting. The TRIGR (Trial to Reduce Insulin-Dependent Diabetes Mellitus in the Genetically at Risk) study was cited as a good example of “arm’s length” BMS trial practice, in which the BMS manufacturer’s role was limited to provision of trial interventions.26 Experts also emphasized that significant trial amendments—especially changes to participant inclusion criteria, experimental or control treatment, and methods, timing, or nature of outcome measures—should be recorded by way of an update to the BMS trial’s record on a World Health Organization–approved clinical trial registry.

    Theme 2: Study Design and Breastfeeding Support

    The provision of breastfeeding support in BMS trials was a controversial area, resolved by experts through identifying the importance of distinguishing 2 different approaches to breastfeeding support for 2 different types of studies. In BMS trials designed to meet a noninferiority or equivalence objective—typically those aiming to demonstrate adequate infant growth and tolerance of a new BMS product—experts thought that participating infants should be fully BMS fed and the decision not to use breast milk should be firmly established prior to enrollment in the trial. After randomization, additional breastfeeding support is not usually required for participants in these studies, but it is important to ensure that appropriate breastfeeding support has been provided prior to enrollment. In some countries, regulators have additional specific requirements for infant growth and tolerance trials—for example, in the United States, growth trials must enroll infants at age 14 days or younger with an intervention period that lasts for 15 weeks or more.27 These noninferiority or equivalence trials should usually be analyzed using both intention-to-treat and a prespecified per-protocol data set.

    In a separate group of BMS trials, usually pragmatic superiority trials aiming to generate data to support a nutrition or health claim, some infants are receiving breast milk at enrollment. Superiority trials should usually be analyzed using an intention-to-treat data set. In trials in which some infants are receiving breast milk at enrollment, experts agreed that it is important to demonstrate adequate support for breast milk feeding within the trial. In these studies, it was thought that an international board–certified lactation consultant employed by an academic or health care institution would be best placed to offer skilled breast milk feeding support.

    Theme 3: Description of Trial Interventions

    Experts confirmed the scope of this guidance as being BMS, as defined by the World Health Organization, including all ingredient additives to BMS that are delivered to an infant within a BMS. Experts agreed that composition and formulation of the experimental and control BMS need to be fully described and related to existing marketed products, and that the timing of the intervention period should be appropriate for the trial objectives. Trial participants’ intake of both experimental and control BMS and any other foods should be accurately recorded.

    Theme 4: Study Outcomes

    Experts agreed that primary and secondary study outcomes should be clearly established a priori and that statistical power calculations for the primary outcome should be based on a clinically meaningful effect size. The end points used to measure each outcome should be valid and clinically relevant, and the use of surrogate end points in place of clinical end points should be appropriately justified and interpreted.

    Trial Participant Viewpoints

    After the consensus meeting, 16 BMS trial participants were contacted and 5 responded, with 3 providing detailed commentary and telephone discussion regarding the criteria. All responding BMS trial participants were supportive of the final criteria, especially independence of trial conduct and analysis and transparent reporting of outcomes. The BMS trial participants commented in detail on 2 criteria concerning the subset of BMS trials in which some infants are receiving breast milk at enrollment. These criteria (15c and 15d) are not relevant to trials in which infants are exclusively fed BMS prior to enrollment and the parents’ decision to not provide breast milk is firmly established prior to enrollment. In support of criterion 15c, they thought that provision of trial BMS should not occur until randomization, and that this provision should not occur during pregnancy or (where relevant) during exclusive breast milk feeding, to avoid providing an incentive to use BMS in place of breast milk. However, participants thought that once a parent decides to supplement breast milk feeding with a BMS, the use of other BMS products should be permitted prior to provision of trial BMS, to avoid feeding problems while awaiting delivery of the experimental or control BMS. In relation to criterion 15d, BMS trial participants viewed the provision of free trial BMS as useful, and supportive for participants with financial constraints, but recognized that this provision may incentivize breastfeeding women to use BMS in place of breast milk. One participant suggested that if free BMS is provided in a trial that includes breastfed infants, a financial incentive to continue breastfeeding could also be provided. The experts agreed by email to add this suggestion to criterion 15d.

    Discussion

    Clinical trials of BMS require specific guidance to ensure that they are methodologically sound, such that their results may reliably inform caregivers and health care professionals. This Delphi survey has derived, through expert consensus, a standard consisting of 58 criteria to support the design, conduct, analysis, transparent reporting, and evaluation of BMS trials. Implementation of this standard, in conjunction with existing methodological and ethical guidance, could better protect BMS trial participants and ultimately improve the quality of BMS products and information associated with them for consumers.

    The validity of this Delphi process is supported by the extensive review of relevant sources that informed the initial criteria and the engagement of a comprehensive panel of experts who provided a diverse range of experience and insight. The consistent and anonymous application of each iteration, as defined a priori in the protocol, minimized bias and manipulation of experts’ opinions. Outcomes from analysis of a sample of BMS trials identified by a pilot systematic review usefully informed the Delphi process. The inclusion of a face-to-face consensus meeting resolved any remaining issues. It was not possible to maintain anonymity of experts at this stage, but the meeting was carefully moderated by an independent chair, through whom experts were invited to submit questions or issues anonymously. Although only 13 of 23 Delphi experts attended the meeting, 3 others provided written comments that were considered during the meeting; a full summary of the discussions and decisions, and then the final manuscript, were shared with all experts for comment and approval after the meeting. One expert withdrew from authorship of the article because of disagreement with specific criteria, although these met the predefined requirements for consensus summarized in Table 1.20 To limit bias introduced during development of the criteria, the study team reproduced all experts’ comments anonymously and verbatim in each round. Industry representatives were asked to comment, but not to score the criteria.

    Limitations

    This study had some limitations. We had good representation from Europe and North America, where most BMS trials are conducted, but less good representation from other regions where BMS trials are less commonly conducted. We did not involve industry in the whole Delphi process, because that would represent a conflict of interest for some experts in relation to their regulatory work. This new guidance therefore represents the views of trialists, methodologists, lactation consultants, infant feeding researchers, regulators, and a journal editor rather than the views of industry representatives. Parents of infants who had participated in a BMS trial commented on the criteria at the final stage but were not members of the Delphi panel and did not score criteria.

    Conclusions

    We have developed new, consensus-based guidance for the design, conduct, analysis, and reporting of BMS trials. To achieve our aim of improving the conduct and reporting of BMS trials, this guidance must come to represent the expected standard in this field. Industry representatives, regulators, and clinical trialists have been able to contribute their views on the feasibility and practicality of these criteria, and some regulators such as Health Canada have already incorporated the criteria into their guidance.28 If BMS trialists incorporate this guidance in their clinical trials, in conjunction with existing methodological and ethical guidance, the quality and validity of their trials will benefit, so participants will be protected and the infant nutrition communitywill be better informed about the safety and potential efficacy of BMS products.

    Back to top
    Article Information

    Accepted for Publication: December 18, 2020.

    Published Online: May 11, 2020. doi:10.1001/jamapediatrics.2020.0578

    Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Jarrold K et al. JAMA Pediatrics.

    Corresponding Author: Robert J. Boyle, MD, PhD, National Heart and Lung Institute, Imperial College London, Norfolk Place, Wright Fleming Building, London W2 1PG, United Kingdom (r.boyle@imperial.ac.uk).

    Author Contributions: Dr Boyle had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Jarrold and Helfer contributed equally to the manuscript.

    Concept and design: Jarrold, Helfer, Eskander, Caulfield, Garcia-Larsen, Loder, Osborn, Warner, Boyle.

    Acquisition, analysis, or interpretation of data: Jarrold, Helfer, Eskander, Crawley, Trabulsi, Duffy, Garcia-Larsen, Hayward, Hyde, Jeffries, Knip, Leonardi-Bee, Lodge, Lowe, McGuire, Osborn, Przyrembel, Renfrew, Trumbo, Schneeman, Boyle.

    Drafting of the manuscript: Jarrold, Helfer, Trabulsi, Hayward, Hyde, Osborn, Warner, Boyle.

    Critical revision of the manuscript for important intellectual content: All authors.

    Statistical analysis: Eskander, Boyle.

    Administrative, technical, or material support: Jarrold, Helfer, Eskander, Garcia-Larsen, Hayward, Knip, Loder.

    Supervision: Hyde, Warner, Boyle.

    Conflict of Interest Disclosures: Dr Helfer reported that his employing institution, Imperial College London, has a formal research and innovation partnership with Nestlé; Dr Helfer is not directly involved in this partnership. Dr Trabulsi reported receiving personal fees from Paidion Clinical Research and ByHeart Inc outside the submitted work. Ms Hayward reported being Head of Infant Nutrition for Health Canada, Government of Canada, and responsible for the analysis of clinical growth and tolerance data for the premarket notification of infant formulas in Canada; the Government of Canada supports breastfeeding when possible, and Ms Hayward reported taking part in work on human milk composition with the US Government. Dr Hyde reported having spoken for the UNICEF UK Baby Friendly Hospital Initiative and working in a research group in receipt of research funding from Nestlé, Danone, and Prolacta. Ms Jeffries reported working in a research group that has received research funding from Nestlé, Danone, and Prolacta; receiving funding from Nestlé to attend a conference; and beng previously employed as a research midwife for a breast milk substitute study funded by Nutricia. Dr Leonardi-Bee reported receiving personal fees from Dairy Goat Co-operative, Danone Nutricia Research, and UK Food Standards Agency outside the submitted work. Dr Lowe reported receiving grants from National Health and Medical Research Council, Australia; and nonfinancial support from PuraCap Pharmaceuticals outside the submitted work. Dr McGuire reported receiving grants from the National Institute for Health Research (UK) outside the submitted work. Dr Warner reported receiving grants and personal fees from Danone/Nutricia and Airsonett; personal fees from Anaphylaxis Campaign and Friesland Campina; and grants from the National Institute for Health Research (UK) outside the submitted work. Dr Boyle reported receiving nonfinancial support from the National Institute for Health Research during the conduct of the study; personal fees from Dairy Goat Co-operative, Prota Therapeutics, DBV Technologies, Cochrane, Squiteri and Fearon, Taus, Cebulash and Landau, and ALK Abelo outside the submitted work; and Dr Boyle’s employing institution, Imperial College London, has a formal research and innovation partnership with Nestlé, who manufacture and market infant formula products and sponsor infant formula research; Dr Boyle is not directly involved in this research and innovation partnership and does not undertake research work with Nestlé. No other disclosures were reported.

    Additional Contributions: We acknowledge Michael Landa, JD, Center for Food Safety and Applied Nutrition, US Food and Drug Administration, James Webbe, MD, Neonatology, Imperial College London, and Debby Webb, BSc, UK Department of Health and Social Care, who have advised and contributed to this work, but who were not included as experts in the Delphi survey. We acknowledge Donnell Alexander, MSc, Ministry for Primary Industries, Wellington, New Zealand, Taraneh Dean, PhD, University of Brighton, Tara Kaufmann, BSc, Childhood Obesity Team, UK Department of Health and Social Care, Julie Mennella, PhD, Monell Chemical Senses Center, Nigel Rollins, MD, PhD, Department of Maternal, Newborn, Child and Adolescent Health, World Health Organization, and Dominique Turck, MD, PhD, Hôpital Jeanne de Flandre, for their contributions as experts to the Delphi survey. We thank Hywel Williams, MD, PhD, Centre of Evidence-based Dermatology, Nottingham University, and National Institute for Health Research Health Technology Assessment Programme, for helpful comments on an early draft of the study protocol. We thank the following breast milk substitute industry representatives for their comments on a previous draft of this guidance: Jan van der Mooren, MD, PhD, Danone Nutricia, José Saavedra, MD, PhD, Nestlé Nutrition, Barbara Marriage, PhD, Abbott Nutrition, Stephanie Meyer, Hipp, Rolf Bos, PhD, Friesland Campina, and Liz Carpenter, PhD, and Colin Prosser, PhD, Dairy Goat Co-operative. We thank the clinical trial participants for contributing their views to this guidance. We also thank Peter Aggett, MD, PhD, University of Lancaster, for his role as the independent and external chair of the concluding face-to-face consensus meeting. None of the individuals were compensated for their contributions.

    Additional Information: The results of this study will be disseminated to trial participants, regulators, and industry representatives who contributed to the study, and to relevant patient organizations. Copyright/license for publication: the corresponding author has the right to grant on behalf of all authors and does grant on behalf of all authors a worldwide license to the publishers and its licensees in perpetuity, in all forms, formats and media (whether known now or created in the future), to (1) publish, reproduce, distribute, display and store the contribution; (2) translate the contribution into other languages, create adaptations, reprints, include within collections, and create summaries, extracts and/or abstracts of the contribution; (3) create any other derivative work(s) based on the contribution; (4) exploit all subsidiary rights in the contribution; (5) include electronic links from the contribution to third party material wherever it may be located; and (6) license any third party to do any or all of the above. Patient and public involvement: patients were involved in reviewing the consensus guidance after the Delphi process had concluded. Patients who had participated in breast milk substitute trials commented on the guidance, especially specific criteria related to the patient experience in breast milk substitute trials. Their comments resulted in changes to the detailed guidance around some of the consensus criteria.

    References
    1.
    Rollins  NC, Bhandari  N, Hajeebhoy  N,  et al; Lancet Breastfeeding Series Group.  Why invest, and what it will take to improve breastfeeding practices?   Lancet. 2016;387(10017):491-504. doi:10.1016/S0140-6736(15)01044-2 PubMedGoogle ScholarCrossref
    2.
    Willumsen  J. Regulation of marketing of breastmilk substitutes: biological, behavioural and contextual rationale. World Health Organization, Department of Nutrition for Health and Development. Published July 2013. Accessed December 1, 2019. https://www.who.int/elena/bbc/regulation_breast-milk_substitutes/en/
    3.
    Victora  CG, Bahl  R, Barros  AJ,  et al; Lancet Breastfeeding Series Group.  Breastfeeding in the 21st century: epidemiology, mechanisms, and lifelong effect.   Lancet. 2016;387(10017):475-490. doi:10.1016/S0140-6736(15)01024-7 PubMedGoogle ScholarCrossref
    4.
    Abrams  SA.  Is it time to put a moratorium on new infant formulas that are not adequately investigated?   J Pediatr. 2015;166(3):756-760. doi:10.1016/j.jpeds.2014.11.003 PubMedGoogle ScholarCrossref
    5.
    Belamarich  PF, Bochner  RE, Racine  AD.  A critical review of the marketing claims of infant formula products in the United States.   Clin Pediatr (Phila). 2016;55(5):437-442. doi:10.1177/0009922815589913 PubMedGoogle ScholarCrossref
    6.
    Boyle  RJ, Ierodiakonou  D, Khan  T,  et al.  Hydrolysed formula and risk of allergic or autoimmune disease: systematic review and meta-analysis.   BMJ. 2016;352:i974. doi:10.1136/bmj.i974 PubMedGoogle ScholarCrossref
    7.
    Hughes  HK, Landa  MM, Sharfstein  JM.  Marketing claims for infant formula: the need for evidence.   JAMA Pediatr. 2017;171(2):105-106. doi:10.1001/jamapediatrics.2016.3837 PubMedGoogle ScholarCrossref
    8.
    Kent  G.  Regulating fatty acids in infant formula: critical assessment of U.S. policies and practices.   Int Breastfeed J. 2014;9(1):2. doi:10.1186/1746-4358-9-2 PubMedGoogle ScholarCrossref
    9.
    Tijhuis  M, Doets  E, Vonk Noordegraaf-Schouten  M.  Extensive literature search and review as preparatory work for the evaluation of the essential composition of infant and follow-on formulae and growing-up milk.   EFSA Supporting Publications. 2014;11(1):551E. doi:10.2903/sp.efsa.2014.EN-551 Google ScholarCrossref
    10.
    Chalmers  I, Glasziou  P.  Avoidable waste in the production and reporting of research evidence.   Lancet. 2009;374(9683):86-89. doi:10.1016/S0140-6736(09)60329-9 PubMedGoogle ScholarCrossref
    11.
    Lundh  A, Lexchin  J, Mintzes  B, Schroll  JB, Bero  L.  Industry sponsorship and research outcome.   Cochrane Database Syst Rev. 2017;2(2):MR000033. doi:10.1002/14651858.MR000033.pub3PubMedGoogle Scholar
    12.
    Nestle  M.  Corporate funding of food and nutrition research: science or marketing?   JAMA Intern Med. 2016;176(1):13-14. doi:10.1001/jamainternmed.2015.6667 PubMedGoogle ScholarCrossref
    13.
    Rasmussen  K, Bero  L, Redberg  R, Gøtzsche  PC, Lundh  A.  Collaboration between academics and industry in clinical trials: cross sectional study of publications and survey of lead academic authors.   BMJ. 2018;363:k3654. doi:10.1136/bmj.k3654 PubMedGoogle ScholarCrossref
    14.
    World Health Organization. Global strategy for infant and young child feeding. Published 2003. Accessed December 1, 2019. https://www.who.int/nutrition/topics/global_strategy/en/
    15.
    Jones  J, Hunter  D.  Consensus methods for medical and health services research.   BMJ. 1995;311(7001):376-380. doi:10.1136/bmj.311.7001.376 PubMedGoogle ScholarCrossref
    16.
    Sinha  IP, Smyth  RL, Williamson  PR.  Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies.   PLoS Med. 2011;8(1):e1000393. doi:10.1371/journal.pmed.1000393 PubMedGoogle Scholar
    17.
    Jünger  S, Payne  SA, Brine  J, Radbruch  L, Brearley  SG.  Guidance on Conducting and REporting DElphi Studies (CREDES) in palliative care: recommendations based on a methodological systematic review.   Palliat Med. 2017;31(8):684-706. doi:10.1177/0269216317690685 PubMedGoogle ScholarCrossref
    18.
    Powell  C.  The Delphi technique: myths and realities.   J Adv Nurs. 2003;41(4):376-382. doi:10.1046/j.1365-2648.2003.02537.x PubMedGoogle ScholarCrossref
    19.
    Harman  NL, Bruce  IA, Callery  P,  et al.  MOMENT—Management of Otitis Media with Effusion in Cleft Palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey.   Trials. 2013;14(1):70. doi:10.1186/1745-6215-14-70 PubMedGoogle ScholarCrossref
    20.
    Guyatt  GH, Oxman  AD, Kunz  R,  et al.  GRADE guidelines, 2: framing the question and deciding on important outcomes.   J Clin Epidemiol. 2011;64(4):395-400. doi:10.1016/j.jclinepi.2010.09.012 PubMedGoogle ScholarCrossref
    21.
    Leonardi-Bee  J, Garcia-Larssen  V, Jarrold  K,  et al. Risk of bias and study conduct in clinical trials of breastmilk substitutes: a systematic review. Published 2018. Accessed December 1, 2019. http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018091928
    22.
    World Health Organization. International code of marketing of breast-milk substitutes. Accessed April 9, 2020. https://www.who.int/nutrition/publications/code_english.pdf
    23.
    Baggerly  K.  Disclose all data in publications.   Nature. 2010;467(7314):401. doi:10.1038/467401b PubMedGoogle ScholarCrossref
    24.
    Doshi  P, Goodman  SN, Ioannidis  JP.  Raw data from clinical trials: within reach?   Trends Pharmacol Sci. 2013;34(12):645-647. doi:10.1016/j.tips.2013.10.006 PubMedGoogle ScholarCrossref
    25.
    Ioannidis  JPA, Greenland  S, Hlatky  MA,  et al.  Increasing value and reducing waste in research design, conduct, and analysis.   Lancet. 2014;383(9912):166-175. doi:10.1016/S0140-6736(13)62227-8 PubMedGoogle ScholarCrossref
    26.
    Knip  M, Åkerblom  HK, Al Taji  E,  et al; Writing Group for the TRIGR Study Group.  Effect of hydrolyzed infant formula vs conventional formula on risk of type 1 diabetes: the TRIGR randomized clinical trial.   JAMA. 2018;319(1):38-48. doi:10.1001/jama.2017.19826 PubMedGoogle ScholarCrossref
    27.
    US Food and Drug Administration. Electronic Code of Federal Regulations, Title 21, Chapter I, Subchapter B, Part 106, §106.96 Requirements for quality factors for infant formulas and §106.121 Quality factor assurances for infant formulas. US Department of Health and Human Services. Accessed April 22, 2019. https://www.ecfr.gov
    28.
    Eskander  M, Nguyen  L. Protocol for a growth and tolerance clinical trial for healthy term newborn infants: a guidance document for infant formula manufacturers. Bureau of Nutritional Sciences, Health Canada; October 22, 2019.
    ×