Williams RL, Johnson SB, Greene SM, Larson EB, Green LA, Morris A, Confer D, Reaman G, Madigan R, Kahn J, . Signposts Along the NIH Roadmap for Reengineering Clinical ResearchLessons From the Clinical Research Networks Initiative. Arch Intern Med. 2008;168(17):1919-1925. doi:10.1001/archinte.168.17.1919
Copyright 2008 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2008
The National Institutes of Health (NIH) Roadmap for Medical Research aims to increase the efficiency and speed of clinical research. We report results and lessons learned from a key component of the Roadmap, the Clinical Research Networks initiative.
Twelve diverse, experienced, large, clinical research networks were funded for 3 years to develop strategies for integrating, expanding, and increasing the interoperability of clinical research networks in support of the Roadmap goals. Network leaders met periodically in person and by teleconference to describe common challenges encountered and solutions used for expansion and increased interoperability.
These networks developed innovative solutions to technical challenges, including strategies for interoperability of information systems and management of complex information system technologies (eg, “brokering” to address data system incompatibility, data transfer, and security requirements), and solutions to human factor challenges at the individual, group, intraorganizational, and interorganizational levels (eg, applying collaborative organizing and decision-making processes based on key principles).
These solutions can provide guidance to existing and future clinical research networks, particularly those forming as part of the NIH Clinical Translation Science Award program. Remaining technical and human factor challenges, however, as well as the largely unmet need for consistent funding for network infrastructure and maintenance, stand in the way of fulfilling the vision of a robust future role for clinical research networks.
The environment for clinical research is rapidly evolving, driven by an accelerating rate of scientific discovery, a need for greater efficiency in the research process, and an imperative to markedly reduce the time from discovery to patient benefit.1,2 It is clear that previously successful approaches to clinical research need to change to accommodate the volume of new research and to bring about added efficiencies for research. In 2002, the National Institutes of Health (NIH) created its Roadmap for Medical Research to help chart a way through this rapidly changing environment with a set of strategic initiatives.3
One of these initiatives, Clinical Research Networks, aims to “promote and expand clinical research networks that can rapidly conduct high-quality clinical studies that address multiple research questions.”4 Greater efficiencies and speed of clinical research can be gained through integrating and expanding clinical research networks and increasing their interoperability.
In 2004, the NIH funded a diverse set of 12 clinical research networks (Table 1) to complete specific scientific goals and begin to explore methods for integrating, expanding, and increasing interoperability. Complementary initiatives were an inventory of existing clinical research networks and a study of the feasibility of training large numbers of clinicians to participate in clinical research.5
The expansion and linkage of clinical research networks creates difficult challenges in governance, informatics interoperability, data standardization, recruitment and retention of clinicians, training, funding of infrastructure, intellectual property considerations, fidelity of study methods, and regulatory barriers. Whereas many of these challenges have been studied in isolation in other contexts, their collective presentation creates substantial barriers to expansion and increased efficiency of clinical research networks and thereby deserves focused attention.6- 8
Resolution of these challenges in the clinical research network context, considering the variability of networks, requires the coalescence of knowledge from other fields together with innovative strategies. To date, while some articles have described the process of creating networks for clinical care, for education, or for disease surveillance, there has been little published literature describing approaches to these problems in clinical research networks.9- 12 Articles that exist typically describe processes of a single study or type of network.13
We report herein lessons learned from the Clinical Research Networks initiative with these 12 diverse networks reflecting their efforts with expansion and linkage, including both successes and barriers encountered. We addressed these challenges over a broad range of clinical research settings, varying from decision-support application at the clinician-patient encounter level to integration of large networks and databases across continents (Table 1). The initial lessons learned will be useful for other networks attempting to accelerate the rate of discovery in clinical and translational research.
The 12 clinical research networks were selected in a competitive, peer-reviewed process, and represent diversity in size, structure, scope, governance, and areas of focus (Table 1). Each was charged with addressing the overall goal of integrating and expanding clinical research networks through a set of programmatic objectives:
Increase the scope of network activities to include new scientific questions, disciplines, and/or tools and approaches;
Increase participation, including appropriate training, within the network to include new sites, new patient populations, and/or new investigators;
Facilitate the communication and cooperation of 1 network with 1 or more additional networks.
Informatics system developments were seen as key to the interoperability underlying greater network integration. Networks were encouraged to use standard messaging, terminology, and software platforms to enhance interoperability potential.
Each network had objectives and evaluation criteria specific to its context, though all shared the overall programmatic objectives. Examples of specific network achievements as a result of the program include the following: (1) the creation and operation of a national, multinetwork consortium of primary care, practice-based research networks in underserved communities; (2) the creation of functional linkages between adult and pediatric acute respiratory distress networks; and (3) the creation of an electronic architecture for linking networks for rapid identification of potential subjects for clinical trials. All networks focused on the development of tools and infrastructure to support collaborative clinical research.
Clinical Research Networks initiative investigators met semiannually over the course of the 3-year demonstration project to discuss and troubleshoot common problems. Representatives from the NIH and content experts on relevant topics were present at each meeting as well. Workshops focused on topics such as multisite human subjects review, data quality and credibility, common data standards and information system platforms, and building collaborations.
To synthesize these experiences, the network leaders held discussions in semiannual conferences, supplemented by a series of teleconferences, aimed at identifying and describing a number of common themes across the 12 networks. Through these discussions and associated iteratively developed documents, we described common (1) challenges to increasing the scope of research activity and collaboration with other networks and (2) strategies used to address those challenges and (3) selected remaining challenges to increasing clinical research collaboration. We do not discuss herein other elements of clinical research network processes (eg, centralized laboratory services) that may be essential to network operations but that were not seen as challenges to expansion and interoperability of clinical research networks.
The synthetic discussions and document development highlighted issues and solutions that clustered into technical and human factors (Table 2).
We encountered a number of challenges that limit the ability of information systems to support clinical research network processes. These challenges fall into 2 broad categories: (1) interoperability of information systems and (2) management of complex information system technologies.
Interoperability challenges include the following:
Information system incompatibilities: Independent sites and networks often use differing information infrastructures, creating challenges in linking the systems.
Need for common data elements applicable to both clinical care and research requirements: A principal challenge is ensuring that terms have the same meaning across settings and that the terms are used in the same way by all the participants in the research enterprise.
Adaptation of information environments designed for clinical and billing purposes to research data needs.
Lack of a clear, consistent terminology for clinical data that is both free and widely used: Definitions for clinical data may vary in ways that are unimportant clinically but that are problematic for interoperability of data systems.
Nonstandard architecture of clinical data repositories: Varying structures for storing clinical data create challenges in linking clinical data within and across networks.
Challenges associated with management of complex information technologies (ITs) include the following:
Transport, security, storage, and retrieval of clinical data across independent systems: the technical and logistical aspects of combining clinical data sets can present substantial obstacles.
Electronic health records are used by a minority of clinical sites: Those in use are often legacy systems with databases and formats among which data are not easily shared.
Proprietary considerations of clinical sites that may accompany sharing and reporting clinical data.
Funding for accomplishing the complex tasks that these challenges present.
Among the strategies that the clinical research networks used to confront system interoperability challenges were the following:
Leveraging evolving national work on data standards (groups such as Health Level Seven [HL7],14 cancer Biomedical Informatics Grid [caBIG],15 Clinical Data Interchange Standards Consortium [CDISC],16 and others) to address the need for clear, consistent terminology for clinical data that is both free and widely used.
Brokering: Using a neutral, trusted information system to provide secure links among many existing local data sources, in a design similar to the hub-and-spoke pattern of airline routes. Such a system of linkages can address some of the challenges of disparate underlying data systems, data transfer, storage, local security requirements, and retrieval of clinical data across independent and otherwise incompatible systems.17,18
Creating a central information system that collects information in parallel with existing systems19: Data collection in a parallel system may use a generic Web interface to allow effective operation in sites with varied technological capabilities.
Strategies used to address the management of complex ITs included the following:
Bringing clinicians and informaticians together in planning the merger of clinical data with research informatics requirements.
Using highly secure software to support collaboration while negotiating with local IT managers to address their security concerns.
Providing hardware and IT consultation to clinical sites with insufficient IT resources to support research activities. For example, one clinical research network rebuilt the internal office information system and provided consultation for long-term IT management plans at 2 underfunded clinics where the existing information system had been a patchwork of donated hardware.
Using open source software in building network systems to reduce costs and time to conduct research by eliminating the need to buy or build the required software capability.
Several major barriers continue to impede the technological integration of clinical research networks. A remaining barrier to interoperability is the lack of an established and widely accepted standard for data elements that would support integration of cross-disciplinary data. It is encouraging that SNOMED (Systematized Nomenclature of Medicine)20 is gaining increasing acceptance.
Technology management barriers include the following:
Information security concerns continue to present difficulties for clinical research networks: Fearing breaches, and frequently dependent on software that is not intrinsically secure, clinical sites often manage security by keeping all systems behind extremely restrictive firewalls.
The proprietary nature of some data can limit integration, particularly when there is no economic incentive for sharing.
Differences between research quality data and clinical data can also pose substantial barriers to integration: Although some research designs (eg, analysis of administrative databases) may not require a high level of precision and concordance of data attributes, other designs (eg, clinical trials) may require more exacting standards for valid hypothesis testing. Unless quality assessment and control processes are clearly identified in data collection, data gathered for clinical purposes might not be useful for certain research purposes without a high level of data curation that would be difficult to sustain.
A number of human interactional and performance factors are critical for successful clinical research network operations. These factors may exist at the individual, group, intraorganizational, or interorganizational levels; commonly these issues exist at several levels simultaneously.
In the experience of the group of networks, the principal challenge for sustainable expansion and interoperability of clinical research networks is building and maintaining multiple levels of collaboration with partners in academia, industry, government, and community clinical practices, as well as with community members. This challenge commonly results from the different motivations, cultural norms, or organizational priorities of the collaborating partners. Scientific collaborators often seek one another out based on common research interests or complementary capabilities. Expanding a collaboration to include new kinds of partnerships and building “networks of networks” further amplifies challenges of collaboration.
Other human factors can present barriers to expanding the scope and interoperability of clinical research networks as well:
Lack of standardization of regulatory processes, particularly those related to institutional review boards, and the need to resolve differing requirements of multiple parallel regulatory bodies.
Lack of incentive for regulatory bodies to address regulatory inconsistencies or variable review processes.
The effects of the unpredictable flow of research funding on workflow and continuity of staffing, with resulting reductions in efficiency.
Individual variation during the clinical encounter—the basis of much of clinical research—in compliance with research protocols and in use of available information technology.
Regardless of the type of collaboration networks are entering into (eg, with community members, other networks, or some other entity), trust between the partners is an essential foundation. This requires frequent, open, and honest communication, negotiation in good faith, willingness to compromise, mutual understanding of the cultures of partners, and effective leadership and management. Frequent interactions between partners and early collaborative successes are important elements that maintain the collaboration. Partner stakeholders will need to both anticipate and receive benefits from the collaboration.21 Leadership must show respect for and appreciation of the perspectives and interests of all partners. While a single type of governance structure (eg, centralized, distributed, or some other type) will not fit all network collaborations, there must be a shared vision for the collaboration, despite differing motivations that might bring the partners together. The vision must be periodically revisited. Policies and procedures of the collaboration should be codified; they should guide governance and work flow; and compliance with them should be regularly assessed. Each of these attributes of successful network collaboration requires purposeful, dedicated effort.
Several strategies have been pursued to overcome the formidable human interactional factors challenges:
Collaborative organizing and decision-making processes that emphasize each of the principles identified.
Using a theoretical framework to guide reengineering efforts; multiple organizational levels may require leadership and targets for intervention.22
Using decision support systems to address individual variations in research protocol compliance: For example, networks grafted research protocol support into clinical decision support systems that were used in routine patient care for disease management and quality improvement. A key element of this process was the ability to use immediate feedback from users to produce changes within hours.
Using collaborative technologies, such as Web conferencing and Web-based collaboration and document-management platforms, that enable communications, sharing best practices, training, regulatory materials, and other items.
Centralizing and automating regulatory functions to address the multiple parallel regulatory requirements.
Convening meetings of regulatory personnel from network member sites to collectively identify problems and potential solutions.
Explicitly identifying roles and/or work products for each network participant, aligned with their interest and expertise.
Several important barriers remain:
The activities that promote communication and trust in collaboration require initial investment and ongoing costs in time and direct expenses. Current research funding practices do not often allow recovery of these costs, particularly when they occur in advance of project implementation.
Clinical research networks continue to be adversely affected by lack of consistent funding for the broader scope of network operations and infrastructure. Stable infrastructure funding would enhance the longer-term efficiency of the network enterprise.
Collaborative technologies have not been widely adopted.
The need to obtain approval for the use of human subjects from multiple review boards representing multiple institutions across networks and to resolve conflicts in their disparate requirements continues to present a major barrier to network research.
Institutional and commercial proprietary interests remain a barrier to sharing of research data within and between networks: For example, a network and a publicly traded company agreed to collaborate on a project. The intellectual property issues contributed to prolonged negotiations and almost scuttled the project. The delay encountered led to the project's becoming less scientifically compelling.
The creation of large, interoperable clinical research networks able to rapidly implement and conduct protocols on a variety of clinical topics, both independently and through collaborations, is an exciting goal. Such an environment for clinical research might help resolve the difficult problems of slow throughput and limited external validity of traditional clinical research processes while providing practical, evidence-based outcomes in both efficacy and effectiveness studies. These improvements would facilitate translational research activities and enhance resource utilization for clinical research.
The group of 12 clinical research networks that were funded with initial support from the NIH Roadmap was charged with developing and testing methods of expansion and interoperability to begin reengineering clinical research. The supported groups constituted a clinically and geographically diverse set of experienced networks and investigators. Over the 3-year demonstration, these networks succeeded in implementing strategies for overcoming roadblocks to collaborative network research while clarifying continuing challenges in this area, thereby enhancing the future for cross-network research. The group made important strides toward achieving the vision that the Roadmap's Clinical Research Networks initiative represents. New collaborations were established and became functional; new informatics tools were developed to support interoperability; and existing tools were adapted to support increased efficiencies.
Coincident with these considerable successes, the group's individual and collective experiences highlighted a number of challenges that stand in the way of fully realizing the vision of creating new network platforms that facilitate seamless translational research. Among these challenges are considerable technical limitations, such as information system incompatibilities across networks, inconsistently defined or applied data elements, concerns about security for the data in transit or in databases, and the limitations of data collected during routine clinical care for the highly specialized needs of research data. Ongoing work in each of these areas by a variety of groups offers the promise of overcoming these barriers. Of the many groups that have demonstrated sustained commitment to building platforms for reengineering clinical research, CaBIG15 and the National Library of Medicine should receive particular mention, the latter for its Unified Medical Language System.23 These 2 groups are working on critical elements to provide national standards to facilitate the interoperability of clinical research.
While much of the focus of creating interoperability between networks has been on the technical aspects of linking and managing data from various venues, success in achieving the vision underlying the Clinical Research Networks initiative is arguably more dependent on overcoming a variety of challenges related to human factors. In some instances, prospective collaborators may be competing for research dollars, market share, or scientists. Communication problems can result when collaborating investigators are geographically distant from one another; in-person interactions may be necessary to help form the trust needed to collaborate. Furthermore, in many academic institutions, the incentives to collaborate are not well aligned with promotion and tenure criteria.
Effective partnerships are those that assure mutual benefits for all participants. Thus, articulating the benefit from collaboration is a critical element of successful development and promulgation of network-based research. Creating and maintaining a shared sense of purpose and value and an agreed-on operational plan is essential for sustainability. Unfortunately, the loose organizational structures that characterize many clinical research networks create a challenge for maintaining collaborations.
Our experience has demonstrated that creating and maintaining collaborations involving clinical research networks requires relationship characteristics well known in the business and social sciences fields.24- 26 However, their application is less common in the research context.27 Organizational theory, which seeks to explain the structures, functions, and properties of organizations through understanding behavior and dynamics in both individual and group contexts, is a body of knowledge from the social sciences that may be helpful in improving our understanding of clinical research networks. The concepts of distributed networking and chaordic organizations, with their emphases on decentralization and purpose over structure, may be particularly relevant to efforts to link some clinical research networks.28 Regardless of the conceptual foundation, however, blending the needs, interests, and vision of stakeholders while achieving the unified mission of a network requires a conscientious approach and substantial continuing effort on the part of those organizing the collaboration.
An overarching challenge to the clinical research network enterprise is the largely unmet need for consistent funding for network infrastructure and maintenance. The dominant model of clinical research funding focuses on activities directly related to the study of specific scientific questions and does not provide for ongoing costs that are necessary for network operations and continuity. Clinical research networks are laboratories that have inherent costs, much as “wet” laboratories do. Investment in clinical research networks must move beyond episodic funding to provide stable, long-term funding for the maintenance infrastructure that is needed if networks are to contribute to the reengineering of clinical research.
Current concepts for the Clinical and Translational Science Award (CTSA) centers suggest that these centers provide support for clinical research networks. While this may be a very positive step in support of networks, it leaves unanswered questions about how existing clinical research networks might align with CTSA centers. Assuming compatibility issues can be resolved, it is not clear that CTSA centers, even when fully constituted, will have the resources to support the rich diversity of existing clinical research networks.5 Furthermore, certain clinical research networks exist primarily because of the need to leverage national resources to investigate clinical questions of interest. Many important pediatric diseases, for instance, have low incidence or prevalence rates; regional networks developed by CTSA centers are unlikely to have sufficient scope to study key clinical questions related to those diseases. It is not clear how the CTSA model will blend with the ongoing need for national or regional clinical research networks.
Our report is based on the experiences of only 12 networks and may therefore be limited in its generalizability. However, we report themes common to many of these 12 networks in their efforts to develop models for expansion and interoperability. Considering the diversity, depth, and breadth of experience of these networks, it is likely that many of these themes are indeed more broadly applicable.5
The Clinical Research Networks initiative yielded important concrete answers to proposed network research questions. The 12 networks moved beyond their individual projects to provide valuable insights into mechanisms for enhancing the general role of clinical research networks. They developed and tested new strategies for expanding and for increasing interoperability. Difficult technical and human factor challenges remain not only for fulfilling the vision of a robust role for clinical research networks but also for maintaining their current momentum in a time of reengineering the clinical research enterprise.
Correspondence: Robert L. Williams, MD, MPH, Department of Family and Community Medicine, MSC 09-5040, 1 University of New Mexico, Albuquerque, NM 87131 (firstname.lastname@example.org).
Accepted for Publication: March 30, 2008.
Clinical Research Networks Initiative: The following is a complete list of the Principal Investigators of the Clinical Research Networks Initiative: Dennis Confer, MD, National Marrow Donor Program; Stephen Durako, Westat, Rockville, Maryland; Lee Green, MD, University of Michigan; Carol Dukes Hamilton, MD, Duke University Medical Center; Robert Harrington, MD, Duke Clinical Research Institute; Stephen Johnson, PhD, Columbia University; James Kahn, MD, University of California, San Francisco; J. Richard Landis, PhD, University of Pennsylvania; Eric B. Larson, MD, MPH, Group Health Center for Health Studies; Alan Morris, MD, Intermountain Medical Center and University of Utah; Kevin Peterson, MD, University of Minnesota; Gregory Reaman, MD, Children's Oncology Group, National Childhood Cancer Foundation; Robert L. Williams, MD, MPH, University of New Mexico.
Author Contributions:Study concept and design: Williams, Johnson, Greene, Larson, Green, Morris, Confer, Reaman, and Kahn. Acquisition of data: Williams, Green, and Madigan. Analysis and interpretation of data: Williams, Larson, Green, Confer, Reaman, and Kahn. Drafting of the manuscript: Williams, Johnson, Greene, Larson, Green, Reaman, Madigan, and Kahn. Critical revision of the manuscript for important intellectual content: Williams, Johnson, Greene, Larson, Morris, Confer, Reaman, and Kahn. Obtained funding: Williams, Greene, Larson, Green, Morris, Confer, and Reaman. Administrative, technical, and material support: Larson, Morris, Confer, Reaman, Madigan, and Kahn. Study supervision: Williams, Greene, and Kahn.
Financial Disclosure: None reported.
Funding/Support: This project was funded in whole or in part with federal funds from the NIH under contract No. HHSN268200425208-220.
Role of the Sponsor: Apart from establishing broad contractual goals, the NIH did not participate in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation of the manuscript.
Additional Contributions: Nancy Dianis, RN, MS, contributed to the development of this article. Many clinicians and patients voluntarily chose to participate in the clinical research conducted by these networks in the continuing quest for knowledge that will directly benefit patients.