[Skip to Content]
[Skip to Content Landing]
Figure 1. 
Double selection procedure for research designs testing complementary and alternative medicine (CAM) techniques for treating patients with a given conventional medical diagnosis that meets criteria for one of multiple, different CAM system diagnoses. Reproduced with permission from Vincent C, Furnham A. Complementary Medicine: A Research Perspective. New York, NY: John Wiley & Sons Ltd; 1997.

Double selection procedure for research designs testing complementary and alternative medicine (CAM) techniques for treating patients with a given conventional medical diagnosis that meets criteria for one of multiple, different CAM system diagnoses. Reproduced with permission from Vincent C, Furnham A. Complementary Medicine: A Research Perspective. New York, NY: John Wiley & Sons Ltd; 1997.

Figure 2. 
A, Domains for research in conventional biomedicine. The person is the clinical focus, but research examines the subsystems of the person as separate, static units. Reductionism limits the number of variables under study and progressively eliminates consideration of the context (higher-order systems and dynamics) in which the studied subsystem functions. This approach permits optimal understanding of each component part of the person. B, Domains for research in integrative medicine. The person is the clinical focus, but the research examines the person as an intact, complex, dynamic system, composed of lower-order systems and existing within higher-order systems. Integrative research includes multiple variables in interaction and emphasizes the evolving context (higher-order systems and dynamics) in which the person as a system functions. This approach permits optimal understanding of the person as a living system within larger systems.

A, Domains for research in conventional biomedicine. The person is the clinical focus, but research examines the subsystems of the person as separate, static units. Reductionism limits the number of variables under study and progressively eliminates consideration of the context (higher-order systems and dynamics) in which the studied subsystem functions. This approach permits optimal understanding of each component part of the person. B, Domains for research in integrative medicine. The person is the clinical focus, but the research examines the person as an intact, complex, dynamic system, composed of lower-order systems and existing within higher-order systems. Integrative research includes multiple variables in interaction and emphasizes the evolving context (higher-order systems and dynamics) in which the person as a system functions. This approach permits optimal understanding of the person as a living system within larger systems.

1.
Mish  FCed in chief Merriam Webster's Collegiate Dictionary. 10th ed. Springfield, Mass Merriam-Webster Inc1993;
2.
Zollman  CVickers  A What is complementary medicine?  BMJ. 1999;319693- 696Google ScholarCrossref
3.
Vincent  CFurnham  A Why do patients turn to complementary medicine? an empirical study.  Br J Clin Psychol. 1996;3537- 48Google ScholarCrossref
4.
Burdi  MDBaker  LC Physicians' perceptions of autonomy and satisfaction in California.  Health Aff (Millwood). 1999;18134- 145Google ScholarCrossref
5.
Astin  JA Why patients use alternative medicine.  JAMA. 1998;2791548- 1553Google ScholarCrossref
6.
Gaudet  TW Integrative medicine.  Integr Med. 1998;167- 73Google ScholarCrossref
7.
Maizes  VCaspi  O The principles and challenges of integrative medicine.  West J Med. 1999;171148- 149Google Scholar
8.
World Health Organization, About WHO: Definition of Health. Available at: http://www.who.int/aboutwho/en/definition.html. Accessed October 9, 2001
9.
Burns  RNichols  LOMartindale-Adams  JGraney  MJ Interdisciplinary geriatric primary care evaluation and management: two-year outcomes.  J Am Geriatr Soc. 2000;488- 13Google Scholar
10.
Bretscher  MRummans  TSloan  J  et al.  Quality of life in hospice patients: a pilot study.  Psychosomatics. 1999;40309- 313Google ScholarCrossref
11.
Merboth  MKBarnason  S Managing pain.  Nurs Clin North Am. 2000;35375- 383Google Scholar
12.
Rydman  RJTrybus  DButki  NKampe  LMMarley  JA Outcome of case management and comprehensive support services following policy changes in mental health care delivery.  J Med Syst. 1999;23309- 323Google ScholarCrossref
13.
Thiels  CSchmidt  UTreasure  JGarthe  RTroop  N Guided self-change for bulimia nervosa incorporating use of a self-care manual.  Am J Psychiatry. 1998;155947- 953Google Scholar
14.
Marvel  MKDoherty  WJWeiner  E Medical interviewing by exemplary family physicians.  J Fam Pract. 1998;47343- 348Google Scholar
15.
Ferguson  RS Preventive care in daily practice.  J Okla State Med Assoc. 2000;93154- 160Google Scholar
16.
Anderson  RMFunnell  MMFitzgerald  JTMarrero  DG The Diabetes Empowerment Scale.  Diabetes Care. 2000;23739- 743Google ScholarCrossref
17.
Lorig  KSeleznick  MLubeck  DUng  EChastain  RLHolman  HR The beneficial outcomes of the arthritis self-management course are not adequately explained by behavior change.  Arthritis Rheum. 1989;3291- 95Google ScholarCrossref
18.
Jonas  WBLevin  JS Essentials of Complementary and Alternative Medicine.  Philadelphia, Pa Lippincott Williams & Wilkins1999;
19.
Fulder  S The Handbook of Alternative and Complementary Medicine. 3rd ed. Oxford, England Oxford University Press1996;
20.
Ernst  EedHahn  EGed Homoeopathy: A Critical Appraisal.  Oxford, England Butterworth Heinemann1998;
21.
Ernst  EedWhite  Aed Acupuncture: A Scientific Appraisal.  Oxford, England Butterworth Heinemann1998;
22.
Ezzo  JBerman  BHadhazy  VAJadad  ARLao  LSingh  BB Is acupuncture effective for the treatment of chronic pain? a systematic review.  Pain. 2000;86217- 225Google ScholarCrossref
23.
Lundeberg  T Acupuncture in headache.  Cephalalgia. 1999;19 ((suppl 25)) 65- 68Google Scholar
24.
Ter Riet  GKleijnen  JKnipshild  P Acupuncture and chronic pain: a criteria-based meta-analysis.  J Clin Epidemiol. 1990;431191- 1199Google ScholarCrossref
25.
Ernst  EWhite  AR Acupuncture for back pain.  Arch Intern Med. 1998;1582235- 2241Google ScholarCrossref
26.
Hammerschlag  RMorris  MM Clinical trials comparing acupuncture with biomedical standard care.  Complement Ther Med. 1997;5133- 140Google ScholarCrossref
27.
Vincent  CFurnham  A Research methods and research problems in complementary medicine.  Complement Medicine A Research Perspective West Sussex, England Wiley & Sons Ltd1997;148- 174Google Scholar
28.
Sampson  W On being a critic.  Sci Rev Altern Med. 1998;24- 5Google Scholar
29.
Coates  JRJobst  KA Integrated health care.  J Altern Complement Med. 1998;4209- 247Google ScholarCrossref
30.
Brownson  RCSimoes  EJ Measuring the impact of prevention research on public health practice.  Am J Prev Med. 1999;16 ((suppl 3)) 72- 79Google ScholarCrossref
31.
Levin  JSGlass  TAKuski  LHSchuck  JRSteele  LJonas  WB Quantitative methods in research on complementary and alternative medicine.  Med Care. 1997;351079- 1094Google ScholarCrossref
32.
Angell  MKassirer  J Alternative medicine.  N Engl J Med. 1998;339839- 841Google ScholarCrossref
33.
Kane  RL Understanding Health Care Outcomes Research.  Gaigthersburg, Md Aspen Publishers Inc1997;
34.
Sechrest  LStickle  TStewart  M Effective and Persuasive Case Studies.  Cambridge, Mass Human Services Research Institute1996;
35.
Darley  JMGross  PH A hypothesis-confirming bias in labeling effects.  J Pers Soc Psychol. 1983;4420- 33Google ScholarCrossref
36.
Schwartz  GERussek  LG The challenge of one medicine: theories of health and eight "world hypotheses."  Adv Mind Body Med. 1997;137- 23Google Scholar
37.
Bell  IRBaldwin  CMSchwartz  GERRussek  LGS Integrating belief systems and therapies in medicine.  Integr Med. 1998;195- 105Google ScholarCrossref
38.
Bell  IRWalsh  MRussek  LGSSchwartz  GER Proposed applications of conventional research concepts and tools to homeopathic clinical research.  J Am Inst Homeopath. 1999;92111- 128Google Scholar
39.
Coveney  PHighfield  R Frontiers of Complexity: The Search for Order in a Chaotic World.  New York, NY Ballantine Publishing Group1995;
40.
Gallagher  RAppenzeller  T Beyond reductionism.  Science. 1999;28479Google ScholarCrossref
41.
Goldberger  AL Non-linear dynamics for clinicians: chaos theory, fractals, and complexity at the bedside.  Lancet. 1996;3471312- 1314Google ScholarCrossref
42.
MacLean  DMacIntosh  R Health and disease in organizations.  J Altern Complement Med. 1998;4185- 188Google ScholarCrossref
43.
Donabedian  A Evaluating the quality of medical care.  Milbank Memorial Fund Q. 1966;44 (Suppl) 166- 206Google ScholarCrossref
44.
Figueredo  AJSechrest  L Approaches used in conducting health outcomes and effectiveness research.  Eval Program Planning. 2001;2441- 59Google ScholarCrossref
45.
Cook  TDCampbell  DT Quasi-Experimentation: Design and Analysis for Field Settings.  Boston, Mass Houghton Mifflin Co1979;
46.
National Center for Complementary and Alternative Medicine, National Institutes of Health, Major Domains of Complementary and Alternative Medicine. Available at: http://nccam.nih.gov/nccam/fcp/classify/. Accessed October 10, 2001
47.
Schnyer  RAllen  J Acupuncture in the Treatment of Depression: A Manual for Research and Practice.  London, England Churchill Livingstone2001;
48.
Caspi  OBell  IRRychener  DGaudet  TWWeil  AT The Tower of Babel: communication and medicine.  Arch Intern Med. 2000;1603193- 3195Google ScholarCrossref
49.
Vickers  ACassileth  BErnst  E  et al.  How should we research unconventional therapies?  Int J Technol Assess Health Care. 1997;13111- 121Google ScholarCrossref
50.
Relman  ASWeil  A Is integrative medicine the medicine of the future? a debate.  Presented at: University of Arizona April 9, 1999 Tucson, Ariz[A short version of the debate can be found in Dalen JE. Is integrative medicine the medicine of the future? a debate between Arnold S. Relman, MD, and Andrew Weil, MD. Arch Intern Med. 1999;159:2122-2126.]
51.
Little  PEveritt  HWilliamson  I  et al.  Preferences of patients for patient-centred approach to consultation in primary care: observational study.  BMJ. 2001;322468- 472Google ScholarCrossref
52.
Tabachnick  BGFidell  L Using Multivariate Statistics.  New York, NY Harper & Row1983;
53.
Green  L On studying effectiveness.  J Fam Pract. 1999;48423- 424Google Scholar
54.
Mitchell  ACormack  M The Therapeutic Relationship in Complementary Health Care.  London, England Churchill Livingstone1998;
55.
McAlister  FAO'Connor  AMWells  GGrover  SALaupacis  A When should hypertension be treated? the different perspectives of Canadian family physicians and patients.  CMAJ. 2000;163403- 408Google Scholar
56.
Reynolds  CFFrank  EPerel  JM  et al.  Nortriptyline and interpersonal psychotherapy as maintenance therapies for recurrent major depression: a randomized controlled trial in patients older than 59 years.  JAMA. 1999;28139- 45Google ScholarCrossref
57.
Mynors-Wallis  LMGath  DHDay  ABaker  F Randomised, controlled trial of problem-solving treatment, antidepressant medication, and combined treatment for major depression in primary care.  BMJ. 2000;32026- 30Google ScholarCrossref
58.
Sano  MErnesto  CThomas  R  et al.  A controlled trial of selegiline, alpha-tocopherol, or both as treatment for Alzheimer's disease.  N Engl J Med. 1997;3361216- 1222Google ScholarCrossref
59.
Humphreys  PAGevirtz  RN Treatment of recurrent abdominal pain: components analysis of four treatment protocols.  J Pediatr Gastroenterol Nutr. 2000;3147- 51Google ScholarCrossref
60.
Turk  DC Customizing treatment for chronic patients: who, what, and why.  Clin J Pain. 1990;6255- 270Google ScholarCrossref
61.
Pocock  SJElbourne  DR Randomized trials of observational tribulations?  N Engl J Med. 2000;3421907- 1909Google ScholarCrossref
62.
Eisenberg  DMDavis  RBEttner  SL  et al.  Trends in alternative medicine use in the United States, 1990-1997: results of a follow-up national survey.  JAMA. 1998;2801569- 1575Google ScholarCrossref
63.
Chalmers  TCCelano  PSacks  HSSmith  H  Jr Bias in treatment assignment in controlled clinical trials.  N Engl J Med. 1983;3091358- 1361Google ScholarCrossref
64.
Concato  JShah  NHorwitz  RI Randomized, controlled trials, observational studies, and the hierarchy of research designs.  N Engl J Med. 2000;3421887- 1892Google ScholarCrossref
65.
Benson  KHartz  AJ A comparison of observational studies and randomized, controlled trials.  N Engl J Med. 2000;3421878- 1886Google ScholarCrossref
66.
Britton  AMcPherson  KSanderson  CBlack  NBain  C Choosing between randomized and non-randomized studies: a systematic review.  Health Technol Assess. 1998;2i- iv1- 124Google Scholar
67.
Feinstein  AR Epidemiologic analyses of causation: the unlearned scientific lessons of randomized trials.  J Clin Epidemiol. 1989;42481- 489Google ScholarCrossref
68.
Ware  JESherbourne  CD The MOS 36-item short-form health survey (SF-36).  Med Care. 1992;30473- 483Google ScholarCrossref
69.
Idler  E Self-assessed health and mortality: a review of studies.  Int Rev Health Psychol. 1992;133- 54Google Scholar
70.
Long  MJMarshall  BS The relationship between self-assessed health status, mortality, service use, and cost in a managed care setting.  Health Care Manage Rev. 1999;2420- 27Google Scholar
71.
Taylor  SJBogdan  R Introduction to Qualitative Research Methods: The Search for Meanings. 2nd ed. New York, NY John Wiley & Sons Inc1984;
72.
Pope  CedMays  Ned Qualitative Research in Health Care.  London, England BMJ Books2000;
73.
Hungelmann  JKenkel-Rossi  EKlassen  LStollenwerk  R Focus on spiritual well-being: harmonious interconnectedness of mind-body-spirit: use of the JAREL Spiritual Well-Being Scale.  Geriatr Nurs. 1996;17262- 266Google ScholarCrossref
74.
Norman  GRStreiner  DL Principal components and factor analysis, path analysis and structural equation modeling.  Biostatistics The Bare Essentials Hamilton, Ontario BC Decker Inc2000;163- 200Google Scholar
75.
Engel  GL The need for a new medical model: a challenge for biomedicine.  Science. 1977;196129- 136Google ScholarCrossref
76.
Butz  MR Chaos and Complexity: Implications for Psychological Theory and Practice.  Washington, DC Taylor & Francis1997;
77.
Mandell  AJSelz  KA Nonlinear dynamical patterns as personality theory for neurobiology and psychiatry.  Psychiatry. 1995;58371- 390Google Scholar
78.
Coffey  DS Self-organization, complexity, and chaos: the new biology for medicine.  Nat Med. 1998;4882- 885Google ScholarCrossref
79.
Shepperd  J Chaos theory: implications for homeopathy.  J Am Inst Homeopath. 1994;8722Google Scholar
80.
Harris  WSGowda  MKolb  JW  et al.  A randomized, controlled trial of the effects of remote, intercessory prayer on outcomes in patients admitted to the coronary care unit.  Arch Intern Med. 1999;1592273- 2278Google ScholarCrossref
81.
Kiecolt-Glaser  JKGlaser  RCacioppo  JTMalarkey  WB Marital stress: immunologic, neuroendocrine, and autonomic correlates.  Ann N Y Acad Sci. 1998;840656- 663Google ScholarCrossref
Special Article
January 28, 2002

Integrative Medicine and Systemic Outcomes Research: Issues in the Emergence of a New Model for Primary Health Care

Author Affiliations

From the Program in Integrative Medicine, Departments of Medicine (Drs Bell, Caspi, Schwartz, Grant, Gaudet, Rychener, Maizes, and Weil), Psychiatry (Drs Bell and Schwartz), Psychology (Drs Bell, Caspi, and Schwartz), Neurology (Dr Schwartz), Family & Community Medicine (Drs Bell and Weil), Obstetrics and Gynecology (Dr Gaudet), and Pharmacy Practice (Dr Grant), The University of Arizona Colleges of Medicine and Pharmacy, and the Southern Arizona Veterans Affairs Health Care System (Dr Bell), Tucson. Dr Gaudet is now with Duke University Center for Integrative Medicine, Durham, NC.

Arch Intern Med. 2002;162(2):133-140. doi:10.1001/archinte.162.2.133
Abstract

Clinicians and researchers are increasingly using the term integrative medicine to refer to the merging of complementary and alternative medicine (CAM) with conventional biomedicine. However, combination medicine (CAM added to conventional) is not integrative. Integrative medicine represents a higher-order system of systems of care that emphasizes wellness and healing of the entire person (bio-psycho-socio-spiritual dimensions) as primary goals, drawing on both conventional and CAM approaches in the context of a supportive and effective physician-patient relationship. Using the context of integrative medicine, this article outlines the relevance of complex systems theory as an approach to health outcomes research. In this view, health is an emergent property of the person as a complex living system. Within this conceptualization, the whole may exhibit properties that its separate parts do not possess. Thus, unlike biomedical research that typically examines parts of health care and parts of the individual, one at a time, but not the complete system, integrative outcomes research advocates the study of the whole. The whole system includes the patient-provider relationship, multiple conventional and CAM treatments, and the philosophical context of care as the intervention. The systemic outcomes encompass the simultaneous, interactive changes within the whole person.

Any intelligent person can study medical literature and understand when or when not to use various treatments. What is so difficult, even for a skilled physician, is to apply this knowledge in individual cases. For those who know nothing about the fundamentals of healing and treat it casually and talk a lot, nothing seems difficult. They don't think there is any illness that requires careful deliberation. The common run of people thinks medicine can be learned quite easily, whereas it is really extremely difficult to master even for a conscientious physician.—Maimonides (1135-1204), The Regimen of Health Care; 4:7-8

Integration vs assimilation

The term integrative medicine is often used to refer to blending the best of conventional (allopathic) and complementary and alternative medicine (CAM). At face value, this goal seems straightforward and promises an improved package of medical care for the consumer. However, a closer examination of the process of this integration raises a number of complex practical and conceptual issues with which medicine as a field must grapple. This article deals with this process of integration and discusses implications for health care outcomes research. We suggest that by adopting a worldview derived from complex systems theory in which the whole equals more than the sum of its parts, a new perspective for medicine and health care research emerges.

The dictionary1 defines integrate as "to unite with something else," "to incorporate into a larger unit." Although the dictionary definition of the noun integration includes "the incorporation of equals into society,"1 it is evident that the assumption implicit in the merger process for mainstream medicine is of an initial inequality in power and worthiness between conventional and CAM approaches. That is, the politically dominant "larger unit" (ie, conventional medicine in the Western world; see Zollman and Vickers2) carries the values, culture, and conceptual framework into which it expects the smaller unit (ie, CAM) to assimilate. It operates from the assumption that each CAM intervention, once tested and proven effective, can be incorporated into conventional care as now practiced.2

Integrative medicine is not cam

However, much of the conventional contemporary practice of physicians, especially for the treatment of patients with chronic diseases, has continued to focus on a specific somatic disease process at the end organ rather than on healing the individual person. Disaffection with how physicians provide conventional care and rely on pharmaceutical medicine continues to grow among consumers and physicians alike.3,4 Integrative medicine is a comprehensive, primary care system that emphasizes wellness and healing of the whole person5 (bio-psycho-socio-spiritual dimensions) as major goals, above and beyond suppression of a specific somatic disease.6,7 In the ideal situation, the patient and the integrative practitioner are partners in the effort to develop and implement a comprehensive treatment plan for issues that extend far beyond the immediate chief complaint and/or conventional diagnostic category.6 Truly integrative medicine draws from conventional and alternative techniques to facilitate healing and to empower the patient because healing is believed to originate within the patient rather than from the physician.6 Thus, the philosophy of integrative medicine is compatible with the World Health Organization's definition of health that equates health with well-being: "Health is a state of complete physical, mental, and social well-being and not merely the absence of disease or infirmity."8

Within conventional care, some multidisciplinary approaches to patient care have already laid a foundation for the more fully integrative medicine that could emerge: for example, (1) in geriatrics, developing multidisciplinary special care teams9 and end-of-life programs to optimize quality of life in hospice care10; (2) in chronic pain treatment programs, applying multidisciplinary care to improve individual self-efficacy and quality of life11; (3) in psychiatry, blending social supports, psychotherapy, and medications12 as well as emphasizing the patient's responsibility for his or her own recovery13; (4) in family medicine, valuing good physician-patient relationships14 and preventive interventions15; and (5) in behavioral medicine/health psychology, using behavioral interventions to foster self-care and self-efficacy in patients with diabetes16 or arthritis.17 Although these examples of care-beyond-the-lesion offer important models for a more ideal form of medicine, even these models are not necessarily widely available and/or accessible within the current health care system.

As it evolves, truly integrative medicine also depends for its philosophical foundation and patient-centered approach on systems of CAM that emphasize healing the person as a whole (eg, traditional Chinese medicine, Ayurvedic medicine, and classic homeopathy).18 These CAM systems diverge the most in philosophy, diagnosis, and treatment technique from conventional medicine,18 and thus remain marginalized. Using philosophy-driven frameworks that seek balance, harmony, and proper flow throughout the patient, they diagnose and treat patterns of dysfunction within the entire person as an indivisible system.19 From the perspective of Western science, such CAM systems offer (1) virtually no commonly used, reductionist, scientific methods by which to study them; (2) no obvious ways to incorporate them into conventional practice; and (3) no Western conceptual framework into which they fit, even if aspects of care are found useful.20,21

As a result, clinicians and researchers often break off parts of these CAM systems from their original contexts to fit a few of these smaller pieces into the dominant model of conventional care and medical research. For example, numerous studies have investigated the efficacy of acupuncture for various Western disorders,22,23 but virtually no studies examine the effectiveness of the sum total of Chinese medicine as practiced. In acupuncture-only research, the effect sizes are often modest.24,25 Yet, traditionally, Chinese medicine uses a coordinated and individualized program that includes various combinations of diet, botanicals, acupuncture, acupressure, qi gong, and environmental interventions to address the unique, systemic disturbance patterns in a given patient.18 It is a testable hypothesis that the effect sizes of the full treatment program could be much larger and more clinically significant if the entire Chinese medicine treatment program were studied as used.26

Moreover, Western medical research usually assumes that its approach to diagnosis is the preferred way to label a patient. It requires homogeneous groups of patients with conventional diagnoses for study. But each system of CAM has its own theory-driven method for categorizing patients. Within a group of patients with asthma, for example, each CAM system is likely to identify several different subtypes, each requiring a unique set of interventions for optimal treatment response (Figure 1 A). In classic homeopathy, for instance, each of 4 different patients with chronic asthma might respond to only 1 remedy type chosen on the basis of his or her individualized symptom pattern (eg, Arsenicum album or Kali carbonicum or Medorrhinum or Natrum sulphuricum). In other words, homogeneity of sample selection from a conventional medical perspective is likely to break down into heterogeneous diagnoses from each CAM system's point of view.27 Even so, despite different diagnostic labels, these CAM systems share an emphasis on looking for patterns of dysfunction that manifest throughout the individual rather than isolated problems in separate bodily subsystems. For homogeneity of study samples, ideal designs would involve a double selection procedure: first, for a specific conventional diagnosis, and second, for a specific CAM system diagnosis from among the multiple possibilities (Figure 1 B).

Research designs that ignore the diagnostic approach of a given CAM system can achieve only weak tests of the intervention program's ability to benefit patients. The analogy would be for conventional researchers to look at the ability of a single antibiotic to treat "infections" as a homogeneous diagnosis. In a sufficiently large sample of patients with infections from various viral, bacterial, and fungal sources, the drug would demonstrate some overall efficacy, but the study design would obscure the drug's outstanding capacity to kill specific bacteria as opposed to all infectious agents.

A tenet of integrative medicine is that the sources of good medical practice can be conventional and/or CAM.6 Valuing scientific evidence as a method to augment societal understanding of human life and health, integrative medicine recognizes that good medicine must always be based in good science that is inquiry driven and open to new paradigms.6 Yet evidence suggests that the medical establishment does not necessarily reciprocate an openness to new paradigms.28 Rather, economic pragmatism, not idealism or a change toward an integrative philosophy that patients may prefer,5 mainly drives the explosive growth of mainstream services and research in CAM. At the same time, there is currently little evidence that any model for delivering the integrative medical care outlined above, not just isolated CAM techniques, is itself economically viable (eg, within health care payment systems in the United States or England).29 In this consumer-driven process, one challenge for health outcomes researchers is to reevaluate the types of outcomes on which to focus. Even in conventional research, studies on the cost-effectiveness of preventive interventions, for example, are difficult and costly.30

Moreover, even when researchers choose to study those treatment methods that are most similar to the pharmacological orientation of conventional medical care (eg, botanicals or nutrients), they usually do it with a focus on a specific action of a specific constituent on a bodily subsystem rather than with an emphasis on the entire simultaneous spectrum of actions the agent can exert on multiple, interdependent subsystems of the individual as a whole (Figure 2 A vs Figure 2 B). That is, all agents, both conventional and CAM in origin, have in common the likelihood of exerting simultaneous, multiple, interdependent actions. Research on reasons for patients' use of CAM indicates that the philosophical orientation toward the entire person attracts them to CAM treatments, despite the need for out-of-pocket expenditures.5 Reductionism has an essential place in research on the effects of conventional and CAM techniques on specific target subsystems, but it inherently misses the full systemic orientation of integrative medicine. Other scientific approaches are indicated to gain some objective distance and examine the larger picture.31

These considerations raise the issue of proper design and methods for integrative medical research. They do not suggest a need for 2 different standards for outcomes research.32 Rather, they indicate the importance of careful methodologies that include not only the same level of scientific rigor used for conventional medical research topics, but also thoughtful designs honoring the philosophical foundations and clinical practices of each CAM system.31

Worldviews and health outcomes

Health care outcomes research is an emerging field.33 It contributes substantially to the knowledge base of medicine and health care and provides the data necessary for health policy makers. The problem is that health care outcomes may be attributed to many factors, only some of which are under the clinician's control. Whereas it is outside the scope of this article to discuss the issue of causality in medicine, it is crucially important for the reader to understand that outcomes are essentially probability statements.33 As in court, only outcomes results that are persuasive enough will constitute acceptable evidence and will support one health policy over another.34

What might be persuasive for one might not be for another, and one's worldview plays a role in one's ability to relate to new information.35 Worldviews and the values placed on different health outcomes are closely related. Thus, the values that underlie medical care shape the scientific questions that researchers ask, the health outcomes they measure, and their interpretation of the results.36,37

In health care, for example, one convergent theme that many different systems of CAM share (eg, Native American, classic homeopathy, traditional Chinese medicine, and Ayurvedic medicine) is that a given disease may manifest at the spiritual level as well as on the physical plane.18,19 This type of worldview can lead to the belief that the most effective interventions must treat the spiritual disturbance, in many cases as a source of the physical manifestations. Integrative medicine proposes that the origins of disease are multifactorial more than hierarchical, and include genetic, physical, emotional, psychological, and spiritual issues. Thus, an integrative medicine approach seeks to discern multiple perceived origins of a disease process and addresses them all, but without necessarily emphasizing spirituality as a root cause. As a result, the scope of considerations in an integrative medicine practice can span the patient's spiritual life, relationships, and mind-body practices in addition to herbs, physical manipulation techniques, medications, or surgical procedures. Integrative medicine assumes that the individual has the potential for healing at the spiritual level, even when physical healing does not take place.6,7

Conventional medicine, while respecting patients' religious and spiritual values, has confined itself largely to the belief that the physical manifestations are the disease and the primary domain for medical intervention. Consequently, differences between the views of conventional medicine, various CAM systems, and integrative medicine on the nature of disease can lead to divergent treatment plans and even to different goals for healing.19,37 Future research must specify and compare the outcomes of different treatment programs derived from the different worldviews.

Systems theory–based cam and integrative outcomes research

Systems theory provides a rational conceptual framework within which to evaluate CAM systems, integrative medicine, and patient rather than disease-oriented clinical research.36-38 Systems theory and systems science involve the study of the whole as a whole.39 That is, a complex system such as a human being is "one whose properties are not fully explained by an understanding of its component parts [organs, cells, molecules]."40 A person is also a dynamic system in that he or she evolves over time.41 If we look at medicine itself as a complex, dynamic system (one whose properties change with time),42 the blending of conventional and CAM approaches could produce not merely a new toolbox of techniques with which to treat disease, but also an emergent worldview for the field overall.

An emergent is a property of a complex system in which the whole is more than the sum of its parts.39 That is, the larger system exhibits properties that are not the result of simply summing the properties of its component parts. Thus, no part by itself has certain properties that the larger system possesses. This emergent worldview engenders a unique form of health care and a new approach to medical outcomes research, as illustrated in Figure 2 A-B. But what are the implications for integrative medical research? We believe that only through the synthesis of reductionist and systemic science will outcomes research result in a qualitative transformation of health care.

Identifying and weighting health outcomes

It is logical to question the value of the healing-oriented integrative medical approach because limited data exist to suggest that this approach has any advantages over other medical worldviews. A critical step in developing any outcomes research is the creation of a conceptual model that indicates what is believed to contribute to the outcomes and that includes all the pertinent variables that are relevant to the evaluation of the system under study.33 The classic view of quality of health care can generally be divided into 3 components: structure (providers' competency, equipment, etc), process (what was done? how well?), and outcomes (the results of the intervention).43 Thus, optimal integrative outcomes research designs should be inclusive and comprehensive rather than anecdotal and narrow. Yet each of these 3 aspects presents its own unique challenges in CAM system and integrative outcomes research.

Structure

Clinical research generally sets randomized controlled trials as the gold standard. While this method may establish strong causality through the enhancement of internal validity, generalizability is sacrificed.44 This trade-off seems especially challenging in CAM and integrative medicine research where practices are so diverse and practitioner competency is far from being well defined.45 It is difficult, for example, to compare the effectiveness of acupuncture treatment for a given conventional diagnosis in different settings and/or using different techniques. The CAM systems and techniques in practice are indeed highly heterogeneous.46 As a start, some CAM researchers have generated methodological innovations that may improve the likelihood of obtaining reliable, if not generalizable, results (eg, development and dissemination of research practice manuals based on multiple expert practitioners' approaches within a given system for a given conventional diagnosis).47

Process

This facet of outcomes research deals with the appropriateness of the treatment in relation to its nature and quality. Problems here arise from 2 sources of potential biases and limitations: (1) the human factor (ie, Who would do the evaluation? Complementary and alternative medicine practitioners who are not stakeholders but who are well skilled in the scientific method are hard to find) and (2) the criteria and measures used (ie, allopathic or alternative?). These 2 practical problems of integrative research are especially challenging partially because conventional and CAM providers often speak "different languages" and value different outcomes.31,48,49

Outcomes

For obvious reasons this is a domain that is the most emotionally charged; the stakeholders are many. The outcomes controversy relates to the core of this article and to the frame of reference of the definition of health. It deals with what matters most. The World Health Organization definition of health2 is closer to the worldview of integrative medicine as discussed here than it is to that of conventional medicine. However, by considering various viewpoints, we may shed more light on this issue.

Should the primary goal of a physician be solely to eliminate disease, or should it also be to optimize well-being? According to Relman, in a debate with Weil,50 "Medicine cannot be expected to make unhappy people happy, or frightened people calm." Is it a proper role for a physician to assist a patient toward growing in inner peace and spiritual well-being, in addition to subduing the disease process in the body? What outcomes matter to the individual patient, and what differential weights do other stakeholders such as physicians, third-party payers, or hospital administrators place on the outcomes that the patient desires? Do the outcomes that matter most to these other stakeholders hold a comparable weight for the individual patient? Who chooses the outcome goals in the end, and how do researchers measure success?

The worldviews and roles of persons within the health care system influence the selection of outcome goals and their relative weighting. Implicit in the worldview of integrative medicine, consistent with the patient-centered approach to health care,6,51 is the belief that the patient is the most important stakeholder and that the rest of the system must give higher priority to the patient's needs and values than it does now within conventional care. The integrative practitioner would recognize and act on a patient's reluctance or readiness to use a particular CAM intervention when appropriate.

Integrative vs reductionist health outcomes research

Although consensus papers on performing CAM research conclude that it is entirely possible to apply available scientific methods to study systems of care intact,26,31 very few studies are using systems theory principles as a framework. Several reasons may underlie this phenomenon: (1) the unfamiliarity of most clinical researchers with systems constructs and multivariate scientific methods; (2) the implicit assumption that CAM should provide tools for better conventional care, not facilitate the transformation of conventional care into a new, integrative medicine; and (3) just as a given CAM system such as traditional Chinese medicine is a complex whole composed of multiple modalities, so integrative medicine is an even higher-order system of multiple systems. The appropriate scientific methods for assessing the multicausal illnesses, multiple interventions, and multidimensional outcomes (bio-psycho-socio-spiritual) claimed in integrative medicine require complex multivariate design and statistical techniques. Though familiar to many sociologists and behavioral scientists, these tools are typically unfamiliar to medical researchers.52

A reductionist approach to science is valuable. However, to rely exclusively on this approach to evaluate CAM systems and integrative medicine (1) reveals the bias toward the conventional system and investigators' unfamiliarity with the possibility of emergent properties in complex systems and (2) in general fails to reflect the way the real world operates.53 In clinical practice, physicians seldom recommend single interventions in isolation. Furthermore, one cannot study an emergent property without keeping the system that generates it intact. Our conceptual framework for systems theory–based outcomes clinical research thus includes 3 domains: outcome design, outcome measures, and outcome analysis.

Outcome Design

Comparison among randomized groups serves as the cornerstone of clinical studies. This is such a well-accepted design that only seldom do we debate the appropriateness of these comparison groups. Systems theory rationalizes the inclusion of at least 1 arm in the design of outcomes studies that tests the effect of the total sum of interventions under consideration, consistent with the assertion that the whole system may express characteristics not previously documented in or predicted by studies of its component parts.36,37,39 Other arms of such studies would examine the effects of each intervention by itself as well as in comparison with standard conventional care. Thus, when an integrative practitioner recommends multiple interventions, conventional and/or alternative, we will never know what the combination produces in terms of clinical benefit and risk unless we study them all together. Furthermore, to the extent that a provider is involved in the treatment, the provider-patient relationship is another key factor in evaluating the outcomes. For self-care treatments and for other treatments chosen from the cooperative partnership of patient and physician, changes in the patient's sense of self-efficacy may be important.54 Given many choices, the patient's preferences for the course and sequencing of action guide the ultimate decision-making process.55

At a practical level, multiarm studies that mix modalities are logistically difficult and costly. Nevertheless, psychiatry researchers have set precedents for this type of work in evaluating conventional medications vs psychotherapy vs combination therapy.56 Often, but not always, in these investigations, combination therapy is most effective.56,57 As an exception, in a well-known study of patients with Alzheimer disease, the antioxidant vitamin E alone was more effective in slowing disease progression than was the combination of vitamin E with the conventional drug selegiline.58

Furthermore, more treatments may not be better if the categories of the multiple interventions overlap excessively (ie, diet plus several modalities from the psychosocial realm rather than, for example, a package of biochemical/dietary, physical manipulation, psychological, and energy medicine options).37 For instance, pediatric researchers performed a randomized, 4-arm study of recurrent abdominal pain treatment in children, including (1) a fiber diet only, (2) fiber diet plus biofeedback, (3) fiber diet plus biofeedback and cognitive-behavioral interventions, and (4) fiber diet plus biofeedback, cognitive-behavioral treatments, and parental support.59 The 3 interventions added to the dietary manipulation were psychosocial. Notably, the data suggested that the full combination of the 4 interventions was not necessarily superior to a combination of fiber plus 1 psychosocial intervention. Such findings suggest a need to evaluate the categories and potential synergy, or lack thereof, among different interventions from which a treatment program emerges, not just the sheer number of interventions.

Moreover, evidence from conventional research in chronic pain raises an important question as to how to develop an individualized treatment plan for a given patient. Turk,60 for example, has found that pain patients with the same conventional diagnosis, same demographic characteristics, same physical and laboratory findings, and comparable functional capacity on testing differ markedly in rehabilitation program outcomes such as level of perceived pain and perceived interference with quality of life, on the basis of their concomitant psychosocial/behavioral coping style. He proposes that it is necessary to match each patient's psychosocial characteristics to a particular type of behavioral intervention, in addition to a basic treatment program, to foster higher levels of rehabilitative outcomes success in a given medical diagnostic group. In other words, Turk is suggesting a need for subtyping within conventional diagnoses.

This approach may pertain even before applying the diagnostic subtyping of CAM treatment systems to the same individuals. Previous studies suggest that pain conditions are among the leading problems for which patients seek CAM treatments.5,61 However, complementary and alternative medicine is a large umbrella term covering a diverse range of self-help and practitioner-administered techniques. Integrative medicine outcomes researchers may need to meld the subtyping approaches both from the conventional world (ie, psychosocial/behavioral coping styles) and from CAM systems to optimize improvements for the largest numbers of patients.

Thus, it is essential to carry out carefully planned, multifactorial intervention studies to develop an appropriate evidence base toward identifying what constitutes maximally effective and safe practice. In practice, the results from a whole treatment system may be different from a simple summing of results expected from studies of its separate constituents.

Observational study designs are a possible strategy for evaluating complex interventional systems. All observational studies have one crucial deficiency that may be considered as an important threat to their validity: the design is not a controlled experimental one. Each patient's treatment is deliberately chosen rather than randomly assigned, so there is an unavoidable risk of selection bias and systematic differences in outcomes that are not due to the intervention itself.62 Furthermore, concerns have been raised that observational studies artificially inflate effect size.63 Despite these concerns, 2 recent systematic reviews64,65 compared data from observational and randomized controlled trial studies across different medical conditions. It was discovered that observational studies gave results similar to those of randomized controlled trials.64,65 Another study66 concluded that there are no systematic biases in observational studies. Thus, not only are observational studies justifiable under certain conditions, they may even have several advantages over randomized controlled trials, including lower cost, greater timeliness, and a broader range of patients.67 Well-validated methods like quasi-experimentation should also be used more frequently because they allow an understanding of how our health care interventional world operates in actual practice.45

Outcome Measures

Systems theory advocates the measurement of not only disease-specific conventional type outcomes such as blood levels of biomarkers (eg, glycosylated hemoglobin or prostate-specific antigen), cardiac stress tests, or pulmonary function tests, but also psychological, social, and spiritual outcomes in the same study. A challenging clinical question to be asked is: What should we measure for integrative quality of life? Most generic quality of life outcome tools focus on "health-related" aspects (ie, forced-choice items on specific types of physical, emotional, and social functioning that may or may not reflect the functions on which the individual patient places value).33,68 These tools generally separate the individual's values and sense of purpose and meaning in life from "health," and they overlook the intrusions of the treatment into everyday living and its enjoyment within a given individual's values. What would happen if the meaning of the problem and impact of the treatment on daily life mattered not only to the individual, but also as a research outcome, with weight given in similar proportion to prolongation of life and improvement of functional status? Does optimization of wellness, not simply absence of disease, fall within the realm of health-related outcomes? Should all potential outcomes receive equal weight?33 Are "objective" measures more valuable in this research than are subjective measures? For example, geriatric patients' self-assessments of their global health status have been shown to be superior to physician assessments and laboratory tests as a predictor of mortality.69,70

To measure these variables, we need to develop multidimensional outcome measures that can accommodate this type of integrative, patient-oriented clinical research. It may also mean that qualitative research techniques will be important for developing a greater understanding of what matters to the patient as an individual.71,72 Even assuming that the patient's values are prioritized over those of other stakeholders, a great deal of initial research must be done to clarify the nature of those individualized values.

For research on personal meaning to the individual, qualitative research approaches, with their emphasis on interpretative methods, become more useful than quantitative studies.72 Qualitative research, for example, revealed a definition of spiritual well-being in elderly persons whose conventional health ranged from good to terminally ill (ie, "a sense of harmonious interconnectedness between self, others, nature, and Ultimate Other, which exists throughout and beyond time and space"73). In other words, the nature of medical research itself can and must expand beyond the prevailing reductionist approaches and quantitative study designs to measure the systemic effectiveness of integrative medical practice.

Outcome Analysis

Although it is beyond the scope of this article to describe the specifics of advanced statistical techniques in quantitative research, we advocate the use of state-of-the-art, user-friendly advanced methods of data analysis that enable investigators to test a CAM system as a whole, and within its own context (Figure 2 B). Path analysis (an extension of multiple regression), structural equation modeling (analysis that includes latent variables), and confirmatory factor analysis (a systematic analysis of the pattern of relationships among variables that attempts to explain that pattern in terms of a smaller number of underlying hypothetical factors) represent only a few of the examples applicable to this type of health care outcomes research.74 These techniques allow us to look at the complex relationships among many dependent and independent variables at the same time, consistent with the higher level of organization in a complex systems theory model.36,37 Furthermore, by specifying paths by which we think specific variables affect others, we can strengthen our confidence about causal inferences.74

Conclusions and implications

In conclusion, we propose that combination medicine (conventional plus CAM) is not integrative medicine. Integrative medicine is a system of care that considers health (or disease) as an emergent property of the person in an environmental context, conceptualized as an intact, indivisible dynamic system.19,20 Integrative medicine is a complex, dynamic, higher-order system of systems, conventional and CAM. As such, the life domains that medical care and medical outcomes research must address extend far beyond clinical laboratory test results or lesions in specific organs.

In the 25 years since Engel75 published his seminal article on the biopsychosocial model for medicine, a few theoreticians have tried to point out the relevance of dynamic systems theory, chaos, and complexity theory for conventional medicine, psychology, and CAM.76-79 However, medicine as a field has not yet incorporated these ideas on a wide scale. Although clinical researchers have dissected downstream effects of change at a higher-order level on lower-order levels of a system (eg, the effects of prayer on distant healing80 or of psychosocial stressors on neuroimmune markers81; Figure 2 A), they generally have not designed studies to examine the patient as a complex, interactive, dynamic system within the larger system as a whole (Figure 2 B). It is the challenge of health outcomes research to prove or disprove the relevance of this integrative, systemic worldview to the field of medicine and to test the feasibility of its emergence as a practical and desirable way to provide clinical care.

Accepted for publication May 9, 2001.

This research was supported in part by grant P50HL61212 from the National Institutes of Health Pediatric Center for Complementary and Alternative Medicine at the University of Arizona, Tucson (Dr Weil), and by Mid-Career Investigator Award K24AT00057 from the National Institutes of Health, Bethesda, Md (Dr Bell).

Corresponding author: Iris R. Bell, MD, MD(H), PhD, Program in Integrative Medicine, The University of Arizona Health Sciences Center, 1501 N Campbell Ave, Tucson, AZ 85724 (e-mail: ibell@u.arizona.edu).

References
1.
Mish  FCed in chief Merriam Webster's Collegiate Dictionary. 10th ed. Springfield, Mass Merriam-Webster Inc1993;
2.
Zollman  CVickers  A What is complementary medicine?  BMJ. 1999;319693- 696Google ScholarCrossref
3.
Vincent  CFurnham  A Why do patients turn to complementary medicine? an empirical study.  Br J Clin Psychol. 1996;3537- 48Google ScholarCrossref
4.
Burdi  MDBaker  LC Physicians' perceptions of autonomy and satisfaction in California.  Health Aff (Millwood). 1999;18134- 145Google ScholarCrossref
5.
Astin  JA Why patients use alternative medicine.  JAMA. 1998;2791548- 1553Google ScholarCrossref
6.
Gaudet  TW Integrative medicine.  Integr Med. 1998;167- 73Google ScholarCrossref
7.
Maizes  VCaspi  O The principles and challenges of integrative medicine.  West J Med. 1999;171148- 149Google Scholar
8.
World Health Organization, About WHO: Definition of Health. Available at: http://www.who.int/aboutwho/en/definition.html. Accessed October 9, 2001
9.
Burns  RNichols  LOMartindale-Adams  JGraney  MJ Interdisciplinary geriatric primary care evaluation and management: two-year outcomes.  J Am Geriatr Soc. 2000;488- 13Google Scholar
10.
Bretscher  MRummans  TSloan  J  et al.  Quality of life in hospice patients: a pilot study.  Psychosomatics. 1999;40309- 313Google ScholarCrossref
11.
Merboth  MKBarnason  S Managing pain.  Nurs Clin North Am. 2000;35375- 383Google Scholar
12.
Rydman  RJTrybus  DButki  NKampe  LMMarley  JA Outcome of case management and comprehensive support services following policy changes in mental health care delivery.  J Med Syst. 1999;23309- 323Google ScholarCrossref
13.
Thiels  CSchmidt  UTreasure  JGarthe  RTroop  N Guided self-change for bulimia nervosa incorporating use of a self-care manual.  Am J Psychiatry. 1998;155947- 953Google Scholar
14.
Marvel  MKDoherty  WJWeiner  E Medical interviewing by exemplary family physicians.  J Fam Pract. 1998;47343- 348Google Scholar
15.
Ferguson  RS Preventive care in daily practice.  J Okla State Med Assoc. 2000;93154- 160Google Scholar
16.
Anderson  RMFunnell  MMFitzgerald  JTMarrero  DG The Diabetes Empowerment Scale.  Diabetes Care. 2000;23739- 743Google ScholarCrossref
17.
Lorig  KSeleznick  MLubeck  DUng  EChastain  RLHolman  HR The beneficial outcomes of the arthritis self-management course are not adequately explained by behavior change.  Arthritis Rheum. 1989;3291- 95Google ScholarCrossref
18.
Jonas  WBLevin  JS Essentials of Complementary and Alternative Medicine.  Philadelphia, Pa Lippincott Williams & Wilkins1999;
19.
Fulder  S The Handbook of Alternative and Complementary Medicine. 3rd ed. Oxford, England Oxford University Press1996;
20.
Ernst  EedHahn  EGed Homoeopathy: A Critical Appraisal.  Oxford, England Butterworth Heinemann1998;
21.
Ernst  EedWhite  Aed Acupuncture: A Scientific Appraisal.  Oxford, England Butterworth Heinemann1998;
22.
Ezzo  JBerman  BHadhazy  VAJadad  ARLao  LSingh  BB Is acupuncture effective for the treatment of chronic pain? a systematic review.  Pain. 2000;86217- 225Google ScholarCrossref
23.
Lundeberg  T Acupuncture in headache.  Cephalalgia. 1999;19 ((suppl 25)) 65- 68Google Scholar
24.
Ter Riet  GKleijnen  JKnipshild  P Acupuncture and chronic pain: a criteria-based meta-analysis.  J Clin Epidemiol. 1990;431191- 1199Google ScholarCrossref
25.
Ernst  EWhite  AR Acupuncture for back pain.  Arch Intern Med. 1998;1582235- 2241Google ScholarCrossref
26.
Hammerschlag  RMorris  MM Clinical trials comparing acupuncture with biomedical standard care.  Complement Ther Med. 1997;5133- 140Google ScholarCrossref
27.
Vincent  CFurnham  A Research methods and research problems in complementary medicine.  Complement Medicine A Research Perspective West Sussex, England Wiley & Sons Ltd1997;148- 174Google Scholar
28.
Sampson  W On being a critic.  Sci Rev Altern Med. 1998;24- 5Google Scholar
29.
Coates  JRJobst  KA Integrated health care.  J Altern Complement Med. 1998;4209- 247Google ScholarCrossref
30.
Brownson  RCSimoes  EJ Measuring the impact of prevention research on public health practice.  Am J Prev Med. 1999;16 ((suppl 3)) 72- 79Google ScholarCrossref
31.
Levin  JSGlass  TAKuski  LHSchuck  JRSteele  LJonas  WB Quantitative methods in research on complementary and alternative medicine.  Med Care. 1997;351079- 1094Google ScholarCrossref
32.
Angell  MKassirer  J Alternative medicine.  N Engl J Med. 1998;339839- 841Google ScholarCrossref
33.
Kane  RL Understanding Health Care Outcomes Research.  Gaigthersburg, Md Aspen Publishers Inc1997;
34.
Sechrest  LStickle  TStewart  M Effective and Persuasive Case Studies.  Cambridge, Mass Human Services Research Institute1996;
35.
Darley  JMGross  PH A hypothesis-confirming bias in labeling effects.  J Pers Soc Psychol. 1983;4420- 33Google ScholarCrossref
36.
Schwartz  GERussek  LG The challenge of one medicine: theories of health and eight "world hypotheses."  Adv Mind Body Med. 1997;137- 23Google Scholar
37.
Bell  IRBaldwin  CMSchwartz  GERRussek  LGS Integrating belief systems and therapies in medicine.  Integr Med. 1998;195- 105Google ScholarCrossref
38.
Bell  IRWalsh  MRussek  LGSSchwartz  GER Proposed applications of conventional research concepts and tools to homeopathic clinical research.  J Am Inst Homeopath. 1999;92111- 128Google Scholar
39.
Coveney  PHighfield  R Frontiers of Complexity: The Search for Order in a Chaotic World.  New York, NY Ballantine Publishing Group1995;
40.
Gallagher  RAppenzeller  T Beyond reductionism.  Science. 1999;28479Google ScholarCrossref
41.
Goldberger  AL Non-linear dynamics for clinicians: chaos theory, fractals, and complexity at the bedside.  Lancet. 1996;3471312- 1314Google ScholarCrossref
42.
MacLean  DMacIntosh  R Health and disease in organizations.  J Altern Complement Med. 1998;4185- 188Google ScholarCrossref
43.
Donabedian  A Evaluating the quality of medical care.  Milbank Memorial Fund Q. 1966;44 (Suppl) 166- 206Google ScholarCrossref
44.
Figueredo  AJSechrest  L Approaches used in conducting health outcomes and effectiveness research.  Eval Program Planning. 2001;2441- 59Google ScholarCrossref
45.
Cook  TDCampbell  DT Quasi-Experimentation: Design and Analysis for Field Settings.  Boston, Mass Houghton Mifflin Co1979;
46.
National Center for Complementary and Alternative Medicine, National Institutes of Health, Major Domains of Complementary and Alternative Medicine. Available at: http://nccam.nih.gov/nccam/fcp/classify/. Accessed October 10, 2001
47.
Schnyer  RAllen  J Acupuncture in the Treatment of Depression: A Manual for Research and Practice.  London, England Churchill Livingstone2001;
48.
Caspi  OBell  IRRychener  DGaudet  TWWeil  AT The Tower of Babel: communication and medicine.  Arch Intern Med. 2000;1603193- 3195Google ScholarCrossref
49.
Vickers  ACassileth  BErnst  E  et al.  How should we research unconventional therapies?  Int J Technol Assess Health Care. 1997;13111- 121Google ScholarCrossref
50.
Relman  ASWeil  A Is integrative medicine the medicine of the future? a debate.  Presented at: University of Arizona April 9, 1999 Tucson, Ariz[A short version of the debate can be found in Dalen JE. Is integrative medicine the medicine of the future? a debate between Arnold S. Relman, MD, and Andrew Weil, MD. Arch Intern Med. 1999;159:2122-2126.]
51.
Little  PEveritt  HWilliamson  I  et al.  Preferences of patients for patient-centred approach to consultation in primary care: observational study.  BMJ. 2001;322468- 472Google ScholarCrossref
52.
Tabachnick  BGFidell  L Using Multivariate Statistics.  New York, NY Harper & Row1983;
53.
Green  L On studying effectiveness.  J Fam Pract. 1999;48423- 424Google Scholar
54.
Mitchell  ACormack  M The Therapeutic Relationship in Complementary Health Care.  London, England Churchill Livingstone1998;
55.
McAlister  FAO'Connor  AMWells  GGrover  SALaupacis  A When should hypertension be treated? the different perspectives of Canadian family physicians and patients.  CMAJ. 2000;163403- 408Google Scholar
56.
Reynolds  CFFrank  EPerel  JM  et al.  Nortriptyline and interpersonal psychotherapy as maintenance therapies for recurrent major depression: a randomized controlled trial in patients older than 59 years.  JAMA. 1999;28139- 45Google ScholarCrossref
57.
Mynors-Wallis  LMGath  DHDay  ABaker  F Randomised, controlled trial of problem-solving treatment, antidepressant medication, and combined treatment for major depression in primary care.  BMJ. 2000;32026- 30Google ScholarCrossref
58.
Sano  MErnesto  CThomas  R  et al.  A controlled trial of selegiline, alpha-tocopherol, or both as treatment for Alzheimer's disease.  N Engl J Med. 1997;3361216- 1222Google ScholarCrossref
59.
Humphreys  PAGevirtz  RN Treatment of recurrent abdominal pain: components analysis of four treatment protocols.  J Pediatr Gastroenterol Nutr. 2000;3147- 51Google ScholarCrossref
60.
Turk  DC Customizing treatment for chronic patients: who, what, and why.  Clin J Pain. 1990;6255- 270Google ScholarCrossref
61.
Pocock  SJElbourne  DR Randomized trials of observational tribulations?  N Engl J Med. 2000;3421907- 1909Google ScholarCrossref
62.
Eisenberg  DMDavis  RBEttner  SL  et al.  Trends in alternative medicine use in the United States, 1990-1997: results of a follow-up national survey.  JAMA. 1998;2801569- 1575Google ScholarCrossref
63.
Chalmers  TCCelano  PSacks  HSSmith  H  Jr Bias in treatment assignment in controlled clinical trials.  N Engl J Med. 1983;3091358- 1361Google ScholarCrossref
64.
Concato  JShah  NHorwitz  RI Randomized, controlled trials, observational studies, and the hierarchy of research designs.  N Engl J Med. 2000;3421887- 1892Google ScholarCrossref
65.
Benson  KHartz  AJ A comparison of observational studies and randomized, controlled trials.  N Engl J Med. 2000;3421878- 1886Google ScholarCrossref
66.
Britton  AMcPherson  KSanderson  CBlack  NBain  C Choosing between randomized and non-randomized studies: a systematic review.  Health Technol Assess. 1998;2i- iv1- 124Google Scholar
67.
Feinstein  AR Epidemiologic analyses of causation: the unlearned scientific lessons of randomized trials.  J Clin Epidemiol. 1989;42481- 489Google ScholarCrossref
68.
Ware  JESherbourne  CD The MOS 36-item short-form health survey (SF-36).  Med Care. 1992;30473- 483Google ScholarCrossref
69.
Idler  E Self-assessed health and mortality: a review of studies.  Int Rev Health Psychol. 1992;133- 54Google Scholar
70.
Long  MJMarshall  BS The relationship between self-assessed health status, mortality, service use, and cost in a managed care setting.  Health Care Manage Rev. 1999;2420- 27Google Scholar
71.
Taylor  SJBogdan  R Introduction to Qualitative Research Methods: The Search for Meanings. 2nd ed. New York, NY John Wiley & Sons Inc1984;
72.
Pope  CedMays  Ned Qualitative Research in Health Care.  London, England BMJ Books2000;
73.
Hungelmann  JKenkel-Rossi  EKlassen  LStollenwerk  R Focus on spiritual well-being: harmonious interconnectedness of mind-body-spirit: use of the JAREL Spiritual Well-Being Scale.  Geriatr Nurs. 1996;17262- 266Google ScholarCrossref
74.
Norman  GRStreiner  DL Principal components and factor analysis, path analysis and structural equation modeling.  Biostatistics The Bare Essentials Hamilton, Ontario BC Decker Inc2000;163- 200Google Scholar
75.
Engel  GL The need for a new medical model: a challenge for biomedicine.  Science. 1977;196129- 136Google ScholarCrossref
76.
Butz  MR Chaos and Complexity: Implications for Psychological Theory and Practice.  Washington, DC Taylor & Francis1997;
77.
Mandell  AJSelz  KA Nonlinear dynamical patterns as personality theory for neurobiology and psychiatry.  Psychiatry. 1995;58371- 390Google Scholar
78.
Coffey  DS Self-organization, complexity, and chaos: the new biology for medicine.  Nat Med. 1998;4882- 885Google ScholarCrossref
79.
Shepperd  J Chaos theory: implications for homeopathy.  J Am Inst Homeopath. 1994;8722Google Scholar
80.
Harris  WSGowda  MKolb  JW  et al.  A randomized, controlled trial of the effects of remote, intercessory prayer on outcomes in patients admitted to the coronary care unit.  Arch Intern Med. 1999;1592273- 2278Google ScholarCrossref
81.
Kiecolt-Glaser  JKGlaser  RCacioppo  JTMalarkey  WB Marital stress: immunologic, neuroendocrine, and autonomic correlates.  Ann N Y Acad Sci. 1998;840656- 663Google ScholarCrossref
×