Guidance for the Conduct and Reporting of Clinical Trials of Breast Milk Substitutes

Key Points

Question  What is the best way to ensure the validity of clinical trials of breast milk substitutes while protecting trial participants?

Findings  Through a Delphi consensus project, guidance was developed to address issues specific to trials of breast milk substitutes assessing growth and tolerance, as well as trials of breast milk substitutes with other objectives. This consensus guidance summarizes best practice for the design, conduct, analysis, and reporting of trials of breast milk substitutes.

Meaning  Use of this guidance, in conjunction with existing clinical trial regulations, should enhance the quality and validity of trials of breast milk substitutes, protect trial participants, and support the evidence base for infant nutrition recommendations.

Importance  Breast milk substitutes (BMS) are important nutritional products evaluated in clinical trials. Concerns have been raised about the risk of bias in BMS trials, the reliability of claims that arise from such trials, and the potential for BMS trials to undermine breastfeeding in trial participants. Existing clinical trial guidance does not fully address issues specific to BMS trials.

Objectives  To establish new methodological criteria to guide the design, conduct, analysis, and reporting of BMS trials and to support clinical trialists designing and undertaking BMS trials, editors and peer reviewers assessing trial reports for publication, and regulators evaluating the safety, nutritional adequacy, and efficacy of BMS products.

Design, Setting, and Participants  A modified Delphi method was conducted, involving 3 rounds of anonymous questionnaires and a face-to-face consensus meeting between January 1 and October 24, 2018. Participants were 23 experts in BMS trials, BMS regulation, trial methods, breastfeeding support, infant feeding research, and medical publishing, and were affiliated with institutions across Europe, North America, and Australasia. Guidance development was supported by an industry consultation, analysis of methodological issues in a sample of published BMS trials, and consultations with BMS trial participants and a research ethics committee.

Results  An initial 73 criteria, derived from the literature, were sent to the experts. The final consensus guidance contains 54 essential criteria and 4 recommended criteria. An 18-point checklist summarizes the criteria that are specific to BMS trials. Key themes emphasized in the guidance are research integrity and transparency of reporting, supporting breastfeeding in trial participants, accurate description of trial interventions, and use of valid and meaningful outcome measures.

Conclusions and Relevance  Implementation of this guidance should enhance the quality and validity of BMS trials, protect BMS trial participants, and better inform the infant nutrition community about BMS products.

Breast milk substitutes (BMS) are important nutritional products for infants who are not receiving breast milk. Most North American and European infants are exposed to BMS during their first year.¹ Infants are sensitive to health effects of BMS owing to their early stage of development when consuming it and their potentially high level of BMS exposure when BMS are used as a sole source of nutrition. The potential association of BMS with population health is therefore greater than for many other nutritional products, and BMS need a scientifically robust evidence base so that caregivers and health care professionals can make informed feeding choices.^2,3 Clinical trials that test BMS safety and evaluate changes in BMS composition or formulation are the foundation of this evidence base. Several groups have questioned the methodological quality of published BMS trials and, in turn, the robustness of their conclusions.^4-9 Specific issues identified include risk of bias related to trial methods, lack of independence from BMS manufacturers, and less stringent regulatory oversight compared with drug trials.^6,10-13 In BMS trials in which some infants are breastfed at enrollment, trials may also be failing to support the establishment and maintenance of breastfeeding in participants.^6,14 These concerns, and the specific issues related to designing BMS trials that answer relevant scientific questions without undermining breastfeeding, suggest a need for new guidance for BMS trials.

We undertook a Delphi consensus to develop new standards for BMS trials. The new standards aim to support trialists in designing, conducting, analyzing, and reporting trials, as well as support regulators, critical appraisers, and reviewers in evaluating BMS trial reports. The guidance relates to intervention trials of BMS in infants enrolled prior to their first birthday, designed to demonstrate adequate growth and tolerance or other objectives. It is designed to complement other guidance such as that published by the US Food and Drug Administration or the European Food Standards Agency, Good Clinical Practice, or Consolidated Standards of Reporting Trials (CONSORT). Further details are summarized in the eAppendix in the Supplement.

A 3-step Delphi consensus process was used to derive new methodological guidance for BMS trials. This Delphi consensus was undertaken between January 1 and September 30, 2018, with a consensus meeting on October 24, 2018. This method enables aggregation of the anonymous and independent opinions of an expert panel to reach consensus on agreed criteria.^15,16 It is a systematic process of sequential rounds used to resolve clinical problems for which evidence is limited and the opinion of stakeholders is important but might be conflicting.^17,18 We invited experts in BMS trials designed to demonstrate adequate growth and tolerance, BMS trials with other objectives such as supporting health and nutrition claims, BMS regulation, trial methods, breastfeeding support, infant feeding research, and medical publishing. Experts were identified through literature review and consultation with others working in these fields. Initial criteria were developed through review of existing clinical trial and BMS guidance, regulatory standards, and critical appraisals. We conducted 3 rounds of email questionnaires to generate, score, and refine criteria (Figure) and used published requirements for consensus¹⁹ (Table 1).²⁰ The UK Health Research Authority was consulted and confirmed that this study did not require approval by a research ethics committee because it was not considered to be research on patients. Informed consent was obtained by email from all study participants. The protocols for this Delphi process and an associated systematic review are registered on PROSPERO (CRD42018091928).²¹ See the eAppendix in the Supplement for further details.

Each round of the Delphi survey was piloted by the study team prior to initiation, and experts were given 3 to 4 weeks to complete each round, with regular prompts to maximize participation. The study team (K.J., B.H., and R.J.B.) was not part of the Delphi process and did not vote on the criteria.

Experts were asked to rate the importance of criteria that formed the initial guidance, using the GRADE (Grading of Recommendations, Assessment, Development and Evaluations) scale: a score of 1 to 3 corresponds to “not important,” 4 to 6 to “important but not critical,” and 7 to 9 to “critical.”²⁰ If experts thought they could not comment on a criterion, they selected “unable to score.” Experts were also invited to provide free text comments, suggest adjustments to the wording of criteria, or suggest new criteria, and to comment on the scope of the guidance. The study team (K.J. and R.J.B.) summarized scores, anonymized comments, and classified criteria as essential, recommended, consensus out, or no consensus, as described in Table 1.²⁰ All criteria other than consensus out were carried forward to round 2, together with proposed new criteria, proposed edits to existing wording, and any proposed merging or splitting of criteria. All changes or new criteria were highlighted in round 2, together with the anonymous comments from round 1.

Experts were asked to rate the importance of the revised criteria. For criteria repeated from round 1, experts were shown the consensus outcome and their own scoring. Experts were asked to reevaluate the criteria in light of the consensus outcome and propose further edits or comments, but could not add new criteria at this stage. Responses were classified and criteria modified in the same way as for round 1, but criteria that were still classified as no consensus were removed after round 2.

A pilot systematic review of a sample of recent BMS trials was undertaken by the study team (K.J., B.H., and R.J.B.) to evaluate adherence to the preliminary criteria generated in round 2. The findings were summarized for experts before round 3.

Revised guidance after round 2 was sent to BMS industry representatives for comment on the feasibility and relevance of the proposed criteria. Breast milk substitute industry representatives were not invited to score criteria, but their feedback was collated and added to the guidance document to review in round 3.

Experts were asked to review the revised criteria arising from round 2, together with the findings of the systematic review and anonymized industry feedback. Experts were given an opportunity to suggest removal, merging, splitting, or changes to criteria or their ratings. Through analysis of round 3 responses, essential and recommended consensus criteria were finalized. Criteria for which the response to industry comments was unresolved or conflicting comments were received during round 3 were highlighted for discussion during the consensus meeting.

Experts were invited to attend the final consensus meeting in person or by web link. The meeting focused on criteria for which consensus had not yet been achieved. Each relevant criterion was discussed until agreement was reached to retain, edit, or remove it from the guidance. The meeting was facilitated by an independent nonvoting chair with experience in BMS regulation, Peter Aggett, MD, PhD (University of Lancaster, UK). Experts were given the opportunity to comment on each criterion, and for those who wished to raise issues anonymously, opportunities were given to submit questions or comments prior to or during the meeting, to be raised by the chair on their behalf. The study team (K.J., B.H., and R.J.B.) circulated minutes after the consensus meeting, and the meeting was recorded. Any final edits and formatting changes were agreed on through email exchange after the meeting.

After the consensus meeting, the final criteria were sent to parents of infants who had participated in a BMS trial and to the London Riverside National Health Service Research Ethics Committee for formal comment.

This Delphi consensus was undertaken between January 1 and September 30, 2018, with a consensus meeting on October 24, 2018. Twenty-eight experts were contacted and 23 participated in at least 1 stage of the Delphi survey: 6 clinical trialists, 9 experts in BMS regulation, 5 clinical trial methodologists, 2 experts in breastfeeding support and infant feeding research, and 1 medical journal editor. Experts were affiliated with institutions in Europe, North America, and Australasia. Sixteen of the experts were able to contribute to the final consensus meeting. Six of 7 invited BMS industry representatives provided comments between June 1 and September 30, 2018, comprising representatives from Danone Nutricia, Nestlé Nutrition, Abbott Nutrition, Hipp, Friesland Campina, and Dairy Goat Co-operative.

Initial guidance for round 1 included 73 criteria derived from clinical trials, BMS and breastfeeding guidance, and appraisals of the BMS trial literature. General comments raised in the BMS industry consultation related to overlap with existing clinical trial guidance, the value of study designs other than randomized clinical trials, definitions of BMS and other nutritional products, and the title and scope of the guidance. Preliminary findings from the pilot phase of the systematic review, which evaluated a sample of 61 recent BMS trials, were a lack of independently funded studies and a high prevalence of nonregistered trial outcomes highlighted in publication abstracts.

The outcomes at each stage of the Delphi process are summarized in the Figure. The final guidance comprises 54 essential criteria (eTable 1 in the Supplement) and 4 recommended criteria (eTable 2 in the Supplement). Of these, 18 criteria are specific to BMS trials, which are summarized as a checklist in Table 2.^20,22 The 58 criteria are elaborated in the eAppendix in the Supplement, including a list of definitions for the key terms used. Key issues discussed at the consensus meeting centered around 4 themes.

Experts stressed the importance of transparency of trial conduct and reporting: that all BMS trials are registered; that trial outcomes are made publicly available, in line with current initiatives in medical research that aim to increase access to original data sets^23-25; and that oversight of trial conduct, analysis, and reporting, including adverse event coding, is independent. Independence was conceptualized as usually meaning that trial oversight was the responsibility of the principal investigator, and should not be the responsibility of an employee of the BMS industry or any other entity with a potential financial interest in the outcome of the trial. It was thought that in-house industry-led statistical planning and analysis is not appropriate unless there is complete transparency owing to audit by regulators or full publication of participant-level outcome data, such that all statistical analyses can be independently verified. When blinded BMS products are used as trial interventions, industry collaboration may be necessary, but trialists and BMS manufacturers should avoid creating financial dependencies and avoid industry control of trial conduct, analysis, or reporting. The TRIGR (Trial to Reduce Insulin-Dependent Diabetes Mellitus in the Genetically at Risk) study was cited as a good example of “arm’s length” BMS trial practice, in which the BMS manufacturer’s role was limited to provision of trial interventions.²⁶ Experts also emphasized that significant trial amendments—especially changes to participant inclusion criteria, experimental or control treatment, and methods, timing, or nature of outcome measures—should be recorded by way of an update to the BMS trial’s record on a World Health Organization–approved clinical trial registry.

The provision of breastfeeding support in BMS trials was a controversial area, resolved by experts through identifying the importance of distinguishing 2 different approaches to breastfeeding support for 2 different types of studies. In BMS trials designed to meet a noninferiority or equivalence objective—typically those aiming to demonstrate adequate infant growth and tolerance of a new BMS product—experts thought that participating infants should be fully BMS fed and the decision not to use breast milk should be firmly established prior to enrollment in the trial. After randomization, additional breastfeeding support is not usually required for participants in these studies, but it is important to ensure that appropriate breastfeeding support has been provided prior to enrollment. In some countries, regulators have additional specific requirements for infant growth and tolerance trials—for example, in the United States, growth trials must enroll infants at age 14 days or younger with an intervention period that lasts for 15 weeks or more.²⁷ These noninferiority or equivalence trials should usually be analyzed using both intention-to-treat and a prespecified per-protocol data set.

In a separate group of BMS trials, usually pragmatic superiority trials aiming to generate data to support a nutrition or health claim, some infants are receiving breast milk at enrollment. Superiority trials should usually be analyzed using an intention-to-treat data set. In trials in which some infants are receiving breast milk at enrollment, experts agreed that it is important to demonstrate adequate support for breast milk feeding within the trial. In these studies, it was thought that an international board–certified lactation consultant employed by an academic or health care institution would be best placed to offer skilled breast milk feeding support.

Experts confirmed the scope of this guidance as being BMS, as defined by the World Health Organization, including all ingredient additives to BMS that are delivered to an infant within a BMS. Experts agreed that composition and formulation of the experimental and control BMS need to be fully described and related to existing marketed products, and that the timing of the intervention period should be appropriate for the trial objectives. Trial participants’ intake of both experimental and control BMS and any other foods should be accurately recorded.

Experts agreed that primary and secondary study outcomes should be clearly established a priori and that statistical power calculations for the primary outcome should be based on a clinically meaningful effect size. The end points used to measure each outcome should be valid and clinically relevant, and the use of surrogate end points in place of clinical end points should be appropriately justified and interpreted.

After the consensus meeting, 16 BMS trial participants were contacted and 5 responded, with 3 providing detailed commentary and telephone discussion regarding the criteria. All responding BMS trial participants were supportive of the final criteria, especially independence of trial conduct and analysis and transparent reporting of outcomes. The BMS trial participants commented in detail on 2 criteria concerning the subset of BMS trials in which some infants are receiving breast milk at enrollment. These criteria (15c and 15d) are not relevant to trials in which infants are exclusively fed BMS prior to enrollment and the parents’ decision to not provide breast milk is firmly established prior to enrollment. In support of criterion 15c, they thought that provision of trial BMS should not occur until randomization, and that this provision should not occur during pregnancy or (where relevant) during exclusive breast milk feeding, to avoid providing an incentive to use BMS in place of breast milk. However, participants thought that once a parent decides to supplement breast milk feeding with a BMS, the use of other BMS products should be permitted prior to provision of trial BMS, to avoid feeding problems while awaiting delivery of the experimental or control BMS. In relation to criterion 15d, BMS trial participants viewed the provision of free trial BMS as useful, and supportive for participants with financial constraints, but recognized that this provision may incentivize breastfeeding women to use BMS in place of breast milk. One participant suggested that if free BMS is provided in a trial that includes breastfed infants, a financial incentive to continue breastfeeding could also be provided. The experts agreed by email to add this suggestion to criterion 15d.

Clinical trials of BMS require specific guidance to ensure that they are methodologically sound, such that their results may reliably inform caregivers and health care professionals. This Delphi survey has derived, through expert consensus, a standard consisting of 58 criteria to support the design, conduct, analysis, transparent reporting, and evaluation of BMS trials. Implementation of this standard, in conjunction with existing methodological and ethical guidance, could better protect BMS trial participants and ultimately improve the quality of BMS products and information associated with them for consumers.

The validity of this Delphi process is supported by the extensive review of relevant sources that informed the initial criteria and the engagement of a comprehensive panel of experts who provided a diverse range of experience and insight. The consistent and anonymous application of each iteration, as defined a priori in the protocol, minimized bias and manipulation of experts’ opinions. Outcomes from analysis of a sample of BMS trials identified by a pilot systematic review usefully informed the Delphi process. The inclusion of a face-to-face consensus meeting resolved any remaining issues. It was not possible to maintain anonymity of experts at this stage, but the meeting was carefully moderated by an independent chair, through whom experts were invited to submit questions or issues anonymously. Although only 13 of 23 Delphi experts attended the meeting, 3 others provided written comments that were considered during the meeting; a full summary of the discussions and decisions, and then the final manuscript, were shared with all experts for comment and approval after the meeting. One expert withdrew from authorship of the article because of disagreement with specific criteria, although these met the predefined requirements for consensus summarized in Table 1.²⁰ To limit bias introduced during development of the criteria, the study team reproduced all experts’ comments anonymously and verbatim in each round. Industry representatives were asked to comment, but not to score the criteria.

This study had some limitations. We had good representation from Europe and North America, where most BMS trials are conducted, but less good representation from other regions where BMS trials are less commonly conducted. We did not involve industry in the whole Delphi process, because that would represent a conflict of interest for some experts in relation to their regulatory work. This new guidance therefore represents the views of trialists, methodologists, lactation consultants, infant feeding researchers, regulators, and a journal editor rather than the views of industry representatives. Parents of infants who had participated in a BMS trial commented on the criteria at the final stage but were not members of the Delphi panel and did not score criteria.

We have developed new, consensus-based guidance for the design, conduct, analysis, and reporting of BMS trials. To achieve our aim of improving the conduct and reporting of BMS trials, this guidance must come to represent the expected standard in this field. Industry representatives, regulators, and clinical trialists have been able to contribute their views on the feasibility and practicality of these criteria, and some regulators such as Health Canada have already incorporated the criteria into their guidance.²⁸ If BMS trialists incorporate this guidance in their clinical trials, in conjunction with existing methodological and ethical guidance, the quality and validity of their trials will benefit, so participants will be protected and the infant nutrition communitywill be better informed about the safety and potential efficacy of BMS products.

Accepted for Publication: December 18, 2020.

Published Online: May 11, 2020. doi:10.1001/jamapediatrics.2020.0578

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Jarrold K et al. JAMA Pediatrics.

Corresponding Author: Robert J. Boyle, MD, PhD, National Heart and Lung Institute, Imperial College London, Norfolk Place, Wright Fleming Building, London W2 1PG, United Kingdom (r.boyle@imperial.ac.uk).

Author Contributions: Dr Boyle had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Jarrold and Helfer contributed equally to the manuscript.

Concept and design: Jarrold, Helfer, Eskander, Caulfield, Garcia-Larsen, Loder, Osborn, Warner, Boyle.

Acquisition, analysis, or interpretation of data: Jarrold, Helfer, Eskander, Crawley, Trabulsi, Duffy, Garcia-Larsen, Hayward, Hyde, Jeffries, Knip, Leonardi-Bee, Lodge, Lowe, McGuire, Osborn, Przyrembel, Renfrew, Trumbo, Schneeman, Boyle.

Drafting of the manuscript: Jarrold, Helfer, Trabulsi, Hayward, Hyde, Osborn, Warner, Boyle.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Eskander, Boyle.

Administrative, technical, or material support: Jarrold, Helfer, Eskander, Garcia-Larsen, Hayward, Knip, Loder.

Supervision: Hyde, Warner, Boyle.

Conflict of Interest Disclosures: Dr Helfer reported that his employing institution, Imperial College London, has a formal research and innovation partnership with Nestlé; Dr Helfer is not directly involved in this partnership. Dr Trabulsi reported receiving personal fees from Paidion Clinical Research and ByHeart Inc outside the submitted work. Ms Hayward reported being Head of Infant Nutrition for Health Canada, Government of Canada, and responsible for the analysis of clinical growth and tolerance data for the premarket notification of infant formulas in Canada; the Government of Canada supports breastfeeding when possible, and Ms Hayward reported taking part in work on human milk composition with the US Government. Dr Hyde reported having spoken for the UNICEF UK Baby Friendly Hospital Initiative and working in a research group in receipt of research funding from Nestlé, Danone, and Prolacta. Ms Jeffries reported working in a research group that has received research funding from Nestlé, Danone, and Prolacta; receiving funding from Nestlé to attend a conference; and beng previously employed as a research midwife for a breast milk substitute study funded by Nutricia. Dr Leonardi-Bee reported receiving personal fees from Dairy Goat Co-operative, Danone Nutricia Research, and UK Food Standards Agency outside the submitted work. Dr Lowe reported receiving grants from National Health and Medical Research Council, Australia; and nonfinancial support from PuraCap Pharmaceuticals outside the submitted work. Dr McGuire reported receiving grants from the National Institute for Health Research (UK) outside the submitted work. Dr Warner reported receiving grants and personal fees from Danone/Nutricia and Airsonett; personal fees from Anaphylaxis Campaign and Friesland Campina; and grants from the National Institute for Health Research (UK) outside the submitted work. Dr Boyle reported receiving nonfinancial support from the National Institute for Health Research during the conduct of the study; personal fees from Dairy Goat Co-operative, Prota Therapeutics, DBV Technologies, Cochrane, Squiteri and Fearon, Taus, Cebulash and Landau, and ALK Abelo outside the submitted work; and Dr Boyle’s employing institution, Imperial College London, has a formal research and innovation partnership with Nestlé, who manufacture and market infant formula products and sponsor infant formula research; Dr Boyle is not directly involved in this research and innovation partnership and does not undertake research work with Nestlé. No other disclosures were reported.

Additional Contributions: We acknowledge Michael Landa, JD, Center for Food Safety and Applied Nutrition, US Food and Drug Administration, James Webbe, MD, Neonatology, Imperial College London, and Debby Webb, BSc, UK Department of Health and Social Care, who have advised and contributed to this work, but who were not included as experts in the Delphi survey. We acknowledge Donnell Alexander, MSc, Ministry for Primary Industries, Wellington, New Zealand, Taraneh Dean, PhD, University of Brighton, Tara Kaufmann, BSc, Childhood Obesity Team, UK Department of Health and Social Care, Julie Mennella, PhD, Monell Chemical Senses Center, Nigel Rollins, MD, PhD, Department of Maternal, Newborn, Child and Adolescent Health, World Health Organization, and Dominique Turck, MD, PhD, Hôpital Jeanne de Flandre, for their contributions as experts to the Delphi survey. We thank Hywel Williams, MD, PhD, Centre of Evidence-based Dermatology, Nottingham University, and National Institute for Health Research Health Technology Assessment Programme, for helpful comments on an early draft of the study protocol. We thank the following breast milk substitute industry representatives for their comments on a previous draft of this guidance: Jan van der Mooren, MD, PhD, Danone Nutricia, José Saavedra, MD, PhD, Nestlé Nutrition, Barbara Marriage, PhD, Abbott Nutrition, Stephanie Meyer, Hipp, Rolf Bos, PhD, Friesland Campina, and Liz Carpenter, PhD, and Colin Prosser, PhD, Dairy Goat Co-operative. We thank the clinical trial participants for contributing their views to this guidance. We also thank Peter Aggett, MD, PhD, University of Lancaster, for his role as the independent and external chair of the concluding face-to-face consensus meeting. None of the individuals were compensated for their contributions.

Additional Information: The results of this study will be disseminated to trial participants, regulators, and industry representatives who contributed to the study, and to relevant patient organizations. Copyright/license for publication: the corresponding author has the right to grant on behalf of all authors and does grant on behalf of all authors a worldwide license to the publishers and its licensees in perpetuity, in all forms, formats and media (whether known now or created in the future), to (1) publish, reproduce, distribute, display and store the contribution; (2) translate the contribution into other languages, create adaptations, reprints, include within collections, and create summaries, extracts and/or abstracts of the contribution; (3) create any other derivative work(s) based on the contribution; (4) exploit all subsidiary rights in the contribution; (5) include electronic links from the contribution to third party material wherever it may be located; and (6) license any third party to do any or all of the above. Patient and public involvement: patients were involved in reviewing the consensus guidance after the Delphi process had concluded. Patients who had participated in breast milk substitute trials commented on the guidance, especially specific criteria related to the patient experience in breast milk substitute trials. Their comments resulted in changes to the detailed guidance around some of the consensus criteria.