Association Between Sponsorship and Findings of Medical Home Evaluations

Published evaluations of medical home interventions have reported mixed results.^1,2 Some evaluators have been employed or funded by the sponsors of the interventions that they evaluated, while others were independent. Relationships between evaluators and sponsors of medical home interventions might influence evaluation designs and decisions about whether to publish evaluation findings.^3,4 If such influence exists, evaluators employed or funded by intervention sponsors might be more likely to publish favorable findings (or suppress unfavorable findings), compared with evaluators who are independent—thus potentially biasing the medical home literature toward favorable effect estimates.

We searched PubMed, Cumulative Index to Nursing and Allied Health Literature, and Cochrane databases through June 2016 using the key words “medical home,” “health-care home,” “comprehensive primary care,” “advanced primary care,” and “patient aligned care team” to identify 45 evaluations of 23 unique medical home interventions² that compared participating practices with nonparticipating practices on measures of patient experience, technical quality, cost, or utilization of care.

After masking sponsor information, we calculated the proportion of measures on which each article reported statistically significant favorable quantitative results associated with the medical home intervention (eg, higher quality and/or lower costs at P < .05). Two authors (A.O., G.R.M.) independently rated the tone of each article’s conclusion as favorable (better patient experience or quality, lower costs or utilization), neutral (limited, modest, or potential improvements in patient experience or quality; no associations with costs or utilization), or unfavorable (worse patient experience or quality, higher costs, increased utilization) (κ = 0.90). Disagreements were resolved by consensus. We then unmasked sponsor information and categorized evaluations as “not independent” if the intervention sponsor funded the evaluation or employed 1 or more authors; otherwise, evaluations were “independent.” We used Fisher exact tests to assess relationships between evaluation independence, choice of evaluation measures, proportion of statistically significant favorable results, and tone of article conclusion.

Thirty-two of the 45 articles presented findings from nonindependent evaluations (Table). Independent evaluations were less likely than nonindependent evaluations to report patient experience (n = 0 vs n = 10 [31%]; P = .04) and more likely to report technical quality measures (n = 11 [85%] vs n = 16 [50%]; P = .04). The proportion of favorable quantitative results did not statistically significantly differ between independent and nonindependent evaluations. The conclusions of independent evaluations were less likely than nonindependent evaluations to have favorable tone (n = 6 [46%] vs n = 25 [78%]; P = .04). Results were similar when we separately analyzed nonindependence due to evaluation funding and nonindependence due to evaluator employment.

The majority of medical home evaluations have been funded by or included employees of the sponsors of the interventions that they evaluated. We found no statistically significant differences between the quantitative results of independent and nonindependent medical home evaluations. However, the tone of narrative conclusions was more likely to be positive when evaluations were nonindependent. Such narrative conclusions matter because medical home evaluations frequently include multiple quantitative measures, allowing authors leeway in crafting a takeaway message when findings are mixed (eg, in deciding how much relative emphasis to put on favorable vs unfavorable findings).

Our study has limitations. We could not analyze evaluations that lacked published sponsorship information; our literature search identified 3 such evaluations. Although article conclusions were rated after blinding sponsor information and with high interrater agreement, these ratings were subjective. The sample size was modest.

We recommend that policy makers and other stakeholders focus on the quantitative findings of medical home evaluations and consider evaluator independence when reading authors’ descriptions of medical home interventions’ overall effectiveness.

Corresponding Author: Anna Oh, MSN, MPH, RN, Division of Geriatrics, Department of Medicine, University of California–San Francisco, 4150 Clement St, VA181G, San Francisco, CA 94121 (anna.oh@ucsf.edu).

Accepted for Publication: April 29, 2017.

Published Online: July 31, 2017. doi:10.1001/jamainternmed.2017.3188

Author Contributions: Ms Oh and Dr Martsolf had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: All authors.

Acquisition, analysis, or interpretation of data: Oh, Martsolf.

Drafting of the manuscript: Oh, Martsolf.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Oh, Martsolf.

Obtained funding: Martsolf.

Administrative, technical, or material support: Oh, Martsolf.

Supervision: Friedberg.

Conflict of Interest Disclosures: Dr Martsolf has received funding to study medical home pilots from the Agency for Healthcare Research and Quality. Dr Friedberg has received compensation from the US Department of Veterans Affairs for consultation related to medical home implementation and research support from the Patient-Centered Outcomes Research Institute via subcontract to the National Committee for Quality Assurance. He also has received funding to evaluate medical home pilots from the Commonwealth Fund, the Centers for Medicare & Medicaid Services, the US Air Force, and North Carolina Innovation, LLC. No other disclosures are reported.