PCC indicates palliative care consultation; PI, principal investigator.
eMethods. Supplementary Methods
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
May P, Normand C, Cassel JB, et al. Economics of Palliative Care for Hospitalized Adults With Serious Illness: A Meta-analysis. JAMA Intern Med. 2018;178(6):820–829. doi:10.1001/jamainternmed.2018.0750
What is the estimated association of palliative care consultation within 3 days of admission with direct hospital costs for adults with serious illness?
In this meta-analysis of 6 studies, hospital costs were lower for patients seen by a palliative care consultation team than for patients who did not receive this care. The estimated association was greater for those with a primary diagnosis of cancer and those with more comorbidities compared with those with a noncancer diagnosis and those with fewer comorbidities.
The estimated association of palliative care consultation with hospital costs varies according to baseline clinical factors; prioritizing current staff to patients with a high illness burden and increasing capacity may reduce hospital costs for a population with high policy importance.
Economics of care for adults with serious illness is a policy priority worldwide. Palliative care may lower costs for hospitalized adults, but the evidence has important limitations.
To estimate the association of palliative care consultation (PCC) with direct hospital costs for adults with serious illness.
Systematic searches of the Embase, PsycINFO, CENTRAL, PubMed, CINAHL, and EconLit databases were performed for English-language journal articles using keywords in the domains of palliative care (eg, palliative, terminal) and economics (eg, cost, utilization), with limiters for hospital and consultation. For Embase, PsycINFO, and CENTRAL, we searched without a time limitation. For PubMed, CINAHL, and EconLit, we searched for articles published after August 1, 2013. Data analysis was performed from April 8, 2017, to September 16, 2017.
Economic evaluations of interdisciplinary PCC for hospitalized adults with at least 1 of 7 illnesses (cancer; heart, liver, or kidney failure; chronic obstructive pulmonary disease; AIDS/HIV; or selected neurodegenerative conditions) in the hospital inpatient setting vs usual care only, controlling for a minimum list of confounders.
Data Extraction and Synthesis
Eight eligible studies were identified, all cohort studies, of which 6 provided sufficient information for inclusion. The study estimated the association of PCC within 3 days of admission with direct hospital costs for each sample and for subsamples defined by primary diagnoses and number of comorbidities at admission, controlling for confounding with an instrumental variable when available and otherwise propensity score weighting. Treatment effect estimates were pooled in the meta-analysis.
Main Outcomes and Measures
Total direct hospital costs.
This study included 6 samples with a total 133 118 patients (range, 1020-82 273), of whom 93.2% were discharged alive (range, 89.0%-98.4%), 40.8% had a primary diagnosis of cancer (range, 15.7%-100.0%), and 3.6% received a PCC (range, 2.2%-22.3%). Mean Elixhauser index scores ranged from 2.2 to 3.5 among the studies. When patients were pooled irrespective of diagnosis, there was a statistically significant reduction in costs (−$3237; 95% CI, −$3581 to −$2893; P < .001). In the stratified analyses, there was a reduction in costs for the cancer (−$4251; 95% CI, −$4664 to −$3837; P < .001) and noncancer (−$2105; 95% CI, −$2698 to −$1511; P < .001) subsamples. The reduction in cost was greater in those with 4 or more comorbidities than for those with 2 or fewer.
Conclusions and Relevance
The estimated association of early hospital PCC with hospital costs may vary according to baseline clinical factors. Estimates may be larger for primary diagnosis of cancer and more comorbidities compared with primary diagnosis of noncancer and fewer comorbidities. Increasing palliative care capacity to meet national guidelines may reduce costs for hospitalized adults with serious and complex illnesses.
People with serious and complex medical illnesses account disproportionately for health care utilization, but this expenditure yields poor value.1,2 A quarter of Medicare beneficiaries die in acute care hospitals, and many experience intensive care unit admission and health care transitions in the last weeks of life, indicating high-intensity care inconsistent with patient preferences.3 Costs are increasing because of not only increasing prevalence of serious chronic disease4-6 but also increasing unit costs of medical care, including hospital care.7,8 Reforming a system originally designed to provide acute, episodic care is essential for its long-term sustainability.9-11
Palliative care is an interdisciplinary specialty focused on improving quality of life for seriously ill patients and their families through symptom management, communication, and patient autonomy.12,13 A review of the literature to July 31, 2013, found that palliative care consultation (PCC) teams are associated with reduced hospital costs but that heterogeneity of methods precluded meta-analysis.14,15 Reported methods may also have biased treatment effect estimates and disguised important associations.15,16 Previous studies have found that estimating the association of PCC with hospital costs requires incorporating time from admission to consultation16,17 and consideration of patient characteristics that may influence the magnitude of the estimate, including primary diagnosis and number of comorbidities.18,19 Few PCC programs meet national staffing guidelines in the United States,20,21 and there is a need worldwide for improved evidence on this intervention.22 Such evidence could inform allocation of existing palliative care capacity, efforts to address staff shortages, and future evaluations of care to populations with serious and complex medical illness.
The aims of this analysis were to estimate the association of PCC with total direct cost of hospital care for adults with serious illness and to examine whether this estimated association varies by primary diagnosis and number of comorbidities. Prior reviews found that meta-analysis of published estimates is not possible because of heterogeneity14,23 and limitations in methods.15,16 Therefore, the aim was pursued through 5 sequential objectives: (1) literature search to identify all economic evaluations of PCC for adult hospital inpatients to August 31, 2017; (2) study selection to appraise all evaluations and identify those that were suitable for methods that incorporate intervention timing,16,17 controlling appropriately for confounding,15 and stratification by clinical factors18,19; (3) data access to approach each suitable study’s lead author and invite collaboration through data sharing; (4) estimation of the association of PCC with total direct costs within each collaborating study data set and for subsamples defined by primary diagnoses and number of comorbidities at admission; and (5) meta-analysis to pool these estimates to address our primary aim.
We hypothesized that PCC is estimated to reduce hospital costs, consistent with prior studies14,24 that have identified lower intensity of hospital treatment and shorter length of stay for patients receiving PCC. We hypothesized that this estimated association would be greater for patients with primary diagnosis of cancer than those with a primary noncancer diagnosis because inpatients with cancer are typically receiving more aggressive care that palliative care may be more able to influence.9 In addition, we hypothesized that the estimated association would be greater for patients with more comorbidities on the basis that palliative care’s interdisciplinary approach may have a greater change in treatments for patients with complex needs (eg, polypharmacy) than for those for whom single disease–focused treatment remains appropriate.19
All studies included in this meta-analysis received relevant institutional review board approval from their sites at the time of the original study. Appropriate data-sharing protocols have been followed in reanalyzing the data for this study: where permitted, principal investigators (PIs) made available anonymized patient-level data to one of us (P.M.); where data were not permitted to be shared, the PI conducted analyses under the direction of one of us (P.M.).
Systematic searches of the Embase, PsycINFO, CENTRAL, PubMed, CINAHL, and EconLit databases were performed for English-language journal articles using keywords in the domains of palliative care (eg, palliative, terminal) and economics (eg, cost, utilization), with limiters for hospital and consultation. For Embase, PsycINFO, and CENTRAL, we searched without a time limitation. For PubMed, CINAHL, and EconLit, we search for articles published after August 1, 2013, because earlier studies were identified by a previous literature review.14 All articles from that earlier review were added to our returned studies by hand. Two independent investigators (P.M., C.N., R.M., and/or C.A.M.) applied the same eligibility criteria as the prior review: economic evaluation (any study design) of a PCC for adult inpatients in acute care hospitals, estimating the association of costs, charges, or cost-effectiveness against a usual care comparator. Conflicts were resolved in discussion between reviewers and another coauthor (R.S.M.). Only English-language peer-reviewed journal articles were considered. One author (R.S.M.) contacted investigators known to be active in this field to ask whether other data sets, including unpublished data sets, existed and were available for analysis. Articles were reviewed using Covidence software.25 Data analysis was performed from January 3, 2017, to January 2, 2018. Details are available in the eMethods in the Supplement.
All economic evaluations identified were appraised by 2 independent reviewers (P.M. and C.N.) for suitability for reanalysis to our methods incorporating PCC timing and matching treatment groups on a minimum set of confounders. We emphasized data that would allow reliable and useful treatment effect estimation: a sample of inpatients with serious illness, baseline data on important confounders and details on PCC provision, and direct costs as an outcome of interest (direct costs are the most reliable indicator of short-term hospital resource26). The authors followed PRISMA in study selection. Details are available in the eMethods in the Supplement.
For each suitable study, we invited collaboration. In the first instance, we approached the corresponding author using contact details published with their article and sought approval from the PI. When this approach did not yield contact, we pursued other avenues to contact the corresponding author (eg, searching online) and by approaching coauthors. All studies for which PIs agreed to collaborate were included in the meta-analysis; all studies for which the PIs refused or were unable to participate or did not acknowledge our inquiries were excluded.
For each study, we retained individuals only if they were recorded as having at least 1 of 7 conditions: cancer; heart, liver, or kidney failure; chronic obstructive pulmonary disease; AIDS/HIV; or selected neurodegenerative conditions. Patients who were admitted for trauma or received an organ transplant were excluded. All diagnoses, trauma, and transplants were identified using hospital International Classification of Diseases, Ninth Revision (ICD-9) codes (eMethods in the Supplement). The number of comorbidities was calculated according to the Elixhauser index27 using ICD-9 codes.
For analyses stratifying by primary diagnosis, within each study, we separated all those with a primary diagnosis of cancer and noncancer. Therefore, noncancer groups included a small number of patients with a secondary diagnosis of cancer because primary diagnosis was the principal reason for admission.
For analyses that stratified samples by comorbidity, we anticipated that the mean and distribution of comorbidity counts would vary among the studies. Therefore, we waited until all data for analysis had been obtained with the intention to compare comorbidity distributions across samples and use the largest number of strata that could support balance of treatment and comparison groups. Details are available in the eMethods in the Supplement.
Our dependent variable was total direct cost of hospital stay. Direct costs are taken from the accounting database of each hospital site and are traceable to specific staffing, equipment, pharmaceuticals, and procedures during an inpatient stay.26
Our primary independent variable was a binary exposure variable: did the individuals receive a PCC within 3 days of admission? Incorporating intervention timing in economic evaluation of PCC increases the accuracy and usefulness of treatment effect estimates and reduces the risk of a false-negative result.15-17 There are different possible approaches to controlling for timing.16 For this meta-analysis, we examined the distributions of consultation timing in all included data sets and selected within 3 days of admission as the specification that best balances competing considerations in defining the exposure variable by timing in observational studies. Details are available in the eMethods in the Supplement.
Additional independent variables were all factors collected at admission that we hypothesized could be associated with treatment and outcome. Uncontrolled-for proximity to death is a particular concern in observational studies of end-of-life populations, but controlling explicitly for mortality raises endogeneity concerns because this is associated with our exposure variable (individuals with higher proximity to death are more likely to receive palliative care) and our outcome of interest (proximity to death is associated with increasing costs).15 We used ICD codes to generate 2 comorbidity measures: the Elixhauser index,27 an additive count of the presence of 31 serious conditions to act as a measure of illness burden, and the van Walraven index,28 a weighted count designed specifically to predict in-hospital mortality. In addition, we performed sensitivity analysis with hospital decedents removed (eMethods in the Supplement).
In a meta-analysis29 of seriously ill populations, observational designs dominate, treatment and comparison groups often differ on observed factors (eg, higher illness burden among palliative care populations), and there may also be a risk of bias through unobserved differences (eg, physician attitude to palliative care). Two main strategies are available in trying to minimize bias: instrumental variables30 and propensity scores.31,32 Instrumental variables allow for control of unobserved confounding, whereas propensity scores control only for observed baseline confounders.33 However, a valid instrument is often hard to identify, especially in routinely collected data, and propensity scores are the most widely used approach to managing confounding.14,15
To control for confounding in observational studies, we used an instrumental variable if available.34 When no instrument was available, we balanced treatment groups in each study on all baseline variables using the covariate balancing propensity score method.31,33,35 We created inverse probability of treatment weights from the estimated propensity score for analyses. Before estimating treatment effects, we evaluated the balance between treatment groups for the overall sample, performing balance diagnostics and taking a 10% mean standardized difference between the treatment group and comparison group to constitute sufficient support.36 We also checked balance across the distribution of the propensity score.37
For each estimate, we regressed total direct costs on our treatment variable and other independent predictors in a weighted sample using generalized linear models with a γ distribution and a log link selected following model evaluation.38,39 We estimated the average treatment effect on the treated (ATET), the typical treatment effect in those who received PCC, holding all other values constant. All regressions were calculated with bootstrapped SEs (1000 replications).40 Where the sample or treatment group was redefined, new propensity scores were calculated.41 No patient in our final analytic sample had missing data in the dependent or independent variables or in receipt and timing of palliative care. Propensity scores were calculated in R,35 and regressions estimating treatment effects were performed with Stata software, version 12 (StataCorp).42 For all tests, P < .05 was deemed to be statistically significant.
Regression output (sample sizes, mean estimated ATETs, and associated 95% CIs) were combined in the meta-analysis to calculate pooled ATETs and CIs. Differences between ATETs for cancer and noncancer samples were assessed using unpaired t tests. Differences between ATETs for samples defined by the number of comorbidities were assessed using 1-way analysis of variance (ANOVA) and posthoc Tukey honest significant difference tests when the ANOVA test statistic was significant at P < .05.
The literature review identified 17 economic evaluations of PCC. Of these 17, we assessed 8 as suitable for the meta-analysis, and these PIs were invited to collaborate.17,18,43-48 Of these 8, 6 accepted the invitation.17,18,43-45,48 One PI was unable to participate because the data were no longer available,46 and one PI and their team did not respond to multiple attempts at communication.47 Therefore, 6 samples were included in the study, with a total 133 118 patients (range, 1020-82 273), of whom 93.2% were discharged alive (range, 89.0%-98.4%), 40.8% had a primary diagnosis of cancer (range, 15.7%-100.0%), and 3.6% received a PCC (range, 2.2%-22.3%) (Figure).
An overview of the 6 data sets is provided in Table 1. The earliest study collected data from May 24, 2001, to December 28, 2004 and the most recent from February 24, 2010, to October 1, 2015. All are from the United States, recording costs in US dollars, which we standardized to 2015, the final year of data collection, using the Consumer Price Index.49 There was wide variation in the proportion of participants receiving PCC (3%-22%), proportion with a primary diagnosis of cancer (16%-100%), and sample size (1020-82 273). Mean Elixhauser indexes ranged from 2.2 to 3.5 among the studies. Data sources varied according to study design. Secondary cohort studies used routine administrative data, extracting baseline and outcome variables from the hospital databases. One primary cohort study complemented access to routinely collected data with original data collection from patient-reported measures and medical record review, and this therefore has the greatest number of baseline variables. In each study, hospitals recorded the presence of conditions using ICD-9 and receipt of PCC was recorded by dedicated databases operated by the palliative medicine service.
One study44 used an instrumental variable to manage confounding; the other 5 studies17,18,43,45,48 had sufficient sample size and baseline characteristics to support good propensity score balance in analysis of treatment effect in all individuals within the sample. Five of 6 studies18,43-45,48 supported stratification by cancer or noncancer primary diagnosis; the exception was a cancer-only study.17 Five of 6 studies17,18,43,45,48 supported stratification by comorbidity count using the Elixhauser index; the exception was the study by Penrod et al44 (eResults in the Supplement).
The estimated association of PCC within 3 days of admission with total direct costs for patients of all diagnoses and stratified by primary diagnosis of cancer or noncancer is presented in Table 2. When patients were pooled irrespective of diagnosis, the meta-analysis estimate suggests a statistically significant reduction in costs (−$3237; 95% CI, −$3581 to −$2893; P < .001). In the stratified analyses, the pooled meta-analysis estimate suggests a statistically significant reduction for both the cancer (−$4251; 95% CI, −$4664 to −$3837; P < .001) and noncancer (−$2105; 95% CI, −$2698 to −$1511; P < .001) subsamples. For comparison of cancer and noncancer, the t test values were −1756 (P < .001) for the study by Morrison et al,43 −8381 (P = .04) for the study by Penrod et al,44 −5365 (P = .15) for the study by Morrison et al,45 −6286 (P < .001) for the study by McCarthy et al,18 4373 (P = .02) for the study by May et al,48 and −2146 (P < .001) for the pooled results.
The meta-analysis treatment effect estimates for samples stratified by primary diagnosis and number of comorbidities (Elixhauser index) are given in Table 3. For all patients irrespective of primary diagnosis, the magnitude of the estimated treatment effect was larger for subsamples with higher numbers of comorbidities. In posthoc analyses, the differences were significant for comparisons of the samples with 4 or more comorbidities with those with 2 or fewer comorbidities and in comparisons of those with 3 comorbidities and those with 1 or 0 comorbidities. The same results were found when equivalent analyses were run for participants with a primary diagnosis of cancer. Among patients without cancer, the magnitude of estimated treatment effect was greater for those with 4 or more comorbidities than for those with 2 or fewer.
Our results suggest that PCC within 3 days of admission reduces direct costs for hospitalized adults with life-limiting illness and that there are important differences in the estimated treatment effect according to clinical factors. Treatment effect estimates were significantly larger for individuals with a cancer diagnosis than for those with a noncancer diagnosis and for individuals with 4 or more comorbidities compared with those with 2 or fewer. These results were derived using a single methodologic approach that addresses identified weaknesses in prior work, and they therefore reflect the state of the science of economic studies of PCC. Of importance, the correlation between estimated cost effect and the comorbidity total of the sample is the reverse of prior research that assumed that long-stay, high-cost hospitalized patients could not have their care trajectories affected by palliative care.15
Current palliative care provision in the United States is characterized by widespread understaffing.20 Our results suggest that acute care hospitals may be able to reduce costs for this population by increasing palliative care capacity to meet national guidelines.21 Generalizability of results from the United States to other societies and health care systems is a concern, and only by exploring similar questions in other settings can the international relevance of these results be fully understood. However, the general pattern of cost-saving observed in the United States is also reported in economic studies50,51 of hospital palliative care in other countries, and although some factors, such as access and incentives, may be specific to the United States, other challenges, such as appropriate decision making for individuals with complex illness in acute care settings, are universal.
Further research is required to confirm these results with randomized clinical trials and prospective observational studies and to assess whether our hypothesized drivers of differences are accurate. Further examination of treatment effect heterogeneity offers the potential to reduce bias in these types of analyses and to identify populations for whom interventions such as PCC are more or less effective.52 Future research must also expand beyond the hospital setting and perspective to estimate cost effects across settings and practitioners for all payers from initial consultation to death.
Our analyses represent a new standard in economic evaluation of the PCC intervention, and the identified weaknesses constitute a framework for further research. In addition to our methods that incorporate intervention timing and minimize observed confounding, a key strength of our study is the use of data from multiple sites, which increases confidence that our findings are generalizable to palliative care practice in different settings in the United States.
All participating studies were identified through already published journal articles, which may be subject to publication bias, although detecting this is difficult because of the small number of studies. We inquired among authors in the field for relevant unpublished data, and none were identified. Eligibility criteria included articles in English, which may have excluded relevant material. For each included study, the PI approved data access and participated as an author on this meta-analysis, contributing to interpretation of the data. All studies used observational designs; thus, causation cannot be claimed, and 5 of 6 studies were retrospective studies that used routinely collected data only; our multimorbidity findings are consistent with a study17 that was conducted prospectively and included original primary data collection.
Our definition of the treatment as receiving palliative care within 3 days of admission was made using the available data rather than a priori, which would be optimal. However, no definitive guidelines defining the appropriate timing of palliative care exist.53 We settled on our 3-day cutoff before estimating any treatment effects on the basis of maximizing sample sizes (and thus power) and minimizing risk of a type I error. Details on defining treatment by timing are available in the eMethods in the Supplement.
Potential sources of unobserved confounding include preferences and treatment rationales; PCC is an intervention that may be provided to a broad range of clinical populations for a variety of reasons,54,55 and in all but one study17 a few eligible individuals received palliative care, which may indicate that our criteria are insufficiently specific in defining palliative care need (and thus bias our results). However, the low numbers of palliative care receipt are consistent with data on widespread unmet need for patients with serious illness (and particularly persons with noncancer diagnoses), and our key conclusions hold in sensitivity analyses that control for discharge status.
Our analyses were concerned only with costs within a single index hospital admission, which has important limitations for policy given that the optimal viewpoint in economic evaluation is the societal perspective, incorporating postdischarge use and out-of-pocket costs.14,24 For example, cost savings observed in hospital settings may be passed to other parts of the health system or onto patients and their families, and we did not incorporate patient outcomes, meaning that lower costs are only beneficial on an assumption that outcomes for palliative care are at least as good as for usual care only (and magnitude of any association does not differ significantly among compared subsamples).56,57 Total direct costs were heterogeneous because each institution uses different cost-accounting methods; however, cost pooling from different hospitals has been performed in multiple prior studies.43,45,58
Stratified analyses may continue to disguise important effect differences. Specific cancer diagnoses and nonmalignant conditions will have different treatment options and prognoses, which may increase or decrease the scope of PCC to change patterns of care. Similarly, the multimorbidity analyses do not take account of specific additional conditions or combinations of conditions, which may have important implications for treatment options, including polypharmacy.
Distributions of ATETs may be sensitive to choice of modeling approach59; however, we compared different model performance before regression analyses, and sensitivity analysis with standard regression did not substantively differ from our main results (eMethods in the Supplement). Finally, this meta-analysis did not weight according to study quality or year of publication, but changes in practice and attitudes to palliative care during the time frame of available data (2002-2015) suggest that more recent data may be more reflective of current practice.
Palliative care consultation within 3 days of hospital admission is estimated to reduce cost of care for hospitalized adults with life-limiting illness, and this reduction in cost was larger for patients with cancer than for those with a noncancer diagnosis and for those with 4 or more comorbidities than for those with 2 or fewer. These results suggest that palliative care is more effective in changing patterns of care for patients with higher illness burden and that it may be possible for acute care hospitals to reduce costs by expanding palliative care capacity.
Accepted for Publication: January 27, 2018.
Corresponding Author: Peter May, PhD, Centre for Health Policy and Management, Trinity College Dublin, 3-4 Foster Place, Dublin 2, Ireland (firstname.lastname@example.org).
Published Online: April 30, 2018. doi:10.1001/jamainternmed.2018.0750
Author Contributions: Drs May and R. S. Morrison had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: May, Normand, Cassel, C. Morrison, R. Morrison.
Acquisition, analysis, or interpretation of data: May, Cassel, Del Fabbro, Fine, Menz, Penrod, Robinson, R. Morrison.
Drafting of the manuscript: May, Normand, Penrod, R. Morrison.
Critical revision of the manuscript for important intellectual content: May, Cassel, Del Fabbro, Fine, Menz, C. Morrison, Robinson, R. Morrison.
Statistical analysis: May, Normand, Robinson.
Obtained funding: Penrod, R. Morrison.
Administrative, technical, or material support: Normand, Cassel, Del Fabbro, Robinson.
Study supervision: Normand, C. Morrison, R. Morrison.
Conflict of Interest Disclosures: None reported.
Funding/Support: Dr May was supported by the International Access, Rights and Empowerment Fellowship Program, which is funded by grant 24611 from The Atlantic Philanthropies.
Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs.
Additional Contributions: This work follows both the original research studies that inform it and data management at participating sites in the meta-analysis. The research team acknowledge the valued contributions of Diane Meier, MD, Melissa Garrido, PhD, Robert Arnold, MD, Phil Santa Emma, MD, Mary Beth Happ, PhD, Thomas J. Smith, MD, and David Weissman, MD on the Palliative Care for Cancer project; Ian M. McCarthy, PhD, Sakib Huq, BS, Martha Philastre, MBA, Gerald Ogola, PhD, and Purvi Prajapati, BS, at Baylor, Scott and & White Health, Dallas, Texas; and Nevena Skoro, MPH, and Danielle Noreika, MD, at Virginia Commonwealth University, Richmond. These individuals were not compensated for their contribution to this project.