[Skip to Navigation]
Sign In
JAMA Forum
June 16, 2022

Evidence, Anecdotes, and Health Policy

Author Affiliations
  • 1Harris School of Public Policy, University of Chicago, Chicago, Illinois
JAMA Health Forum. 2022;3(6):e222427. doi:10.1001/jamahealthforum.2022.2427

It is often hard to comprehend the human toll of public health disasters. How can anyone possibly understand 1 million lives lost to COVID-19 in the US? Sometimes hearing the story of one life lost can be a window into the tragedy of countless others. The same is true of so many health policy decisions that affect millions, but may seem abstract; the stories of individuals can drive home the real importance of those decisions.

But there is a risk to drawing policy conclusions from stories, no matter how powerful. Most policies have complicated effects that are different for different people. Stories that are true can still be quite misleading because any individual’s experience may not be representative of the effect of the policy for most people. It is the best part of human nature to be moved by a fellow human being’s story, but it is a dangerous way to make policy.

Consider the effects of expanding health insurance. Having insurance can affect the health care that people use across many settings (such as office visits, screening tests, prescription drugs, and emergency department [ED] visits) and thus spending on health care. It can also affect financial security (such as debt and overdue bills) and health outcomes (such as mental health, management of chronic physical conditions, mortality, and overall well-being). Assessing the effect of insurance expansion thus requires incorporating these multifaceted outcomes. But those effects might also be quite different for different people.

For example, does health insurance increase or decrease ED use? One can imagine 2 equally plausible answers. On the one hand, by reducing the cost of an ED visit for patients, insurance might lead to more visits for those with insurance than those without insurance, all else being equal. On the other hand, by increasing access to preventive and primary care and potentially improving health, health insurance might lead to fewer ED visits for those with insurance than those without insurance. As with many questions in health policy, we need rigorous data and analyses to generate valid answers.

Getting a good answer first requires a way to separate out the effect of insurance itself from all of the other confounding factors that affect insurance status, health care use, and health outcomes. For example, jobs that offer health insurance may also tend to come with higher wages, and people in those jobs may also have better access to safe environments, housing, social supports, and high-quality health care. Separating those factors out requires thoughtful statistical analysis and the right setting (such as a randomized clinical trial1 or a quasi-experimental approach2,3). Within these careful designs, data analysis can yield insights about both overall effects and how those effects vary across people. It is worth noting that data do not have to be quantitative. Important lessons can be learned from systematic analysis of qualitative data, but that analysis does need to be systematic.

In the Oregon Health Insurance Experiment—a study on which I collaborated with Amy Finkelstein, PhD, Heidi Allen, PhD, MSW, Bill Wright, PhD, and others—we collected qualitative as well as quantitative information in the context of a lottery used to allocate limited slots in Oregon’s Medicaid program—a de facto randomized clinical trial. The stories we heard from participants about their experiences and their health were compelling and also illustrated how different those experiences could be.4

Consistent with the idea that covering uninsured individuals could reduce ED use, one uninsured respondent told us, “When I was uninsured, unless something happened where I had to go to the hospital, then I’d just go to the emergency room and deal with it. Emergency rooms, from what I understand, they can never turn you away.” But consistent with the idea that insurance would increase the use of the ED, an insured respondent told us, “Without coverage I wouldn’t have gone to ER those nights I was in crisis because I was already in crisis, and [the bills] just would have been too much for me to take on mentally or financially.”

Both of these stories convey important lived experiences. But which story is representative of the experience more people are likely to have?

Our analysis showed that insured persons went to the ED 40% more than uninsured individuals.5 This finding surprised many people. Was it because Medicaid did not actually enable access to high-quality primary care? Some patient stories suggested this might be the case. One respondent on Medicaid reported that “They [Medicaid] have no doctors that are actually seeing new patients,” and another reported that the clinic that the new enrollee went to “said they would contact me and then they never did.” But others reported that Medicaid gave them new access to the care they needed. One noted, “I love my doctor that I see with [Medicaid]. He calls me all the time to check up on me.” Another said, “I thought maybe [with state insurance] you just get the minimum care, they’re not really going to go further, but I was totally wrong…I felt like I got the full care that I needed.” The net result was that Medicaid increased physician office visits and ED visits; insurance did not lead to substitution of office visits for ED care.6,7

This information is crucial for policy makers to have. Many have argued that expanding insurance will save money because previously uninsured individuals will get preventive and primary care that will stave off more expensive ED visits. We did not find that to be the case. This finding does not mean that insurance should not be expanded. Rather, it helps paint a realistic picture of the costs to be weighed against the benefits.

This is just one illustration. In fact, Amy Finkelstein and I created a quiz that people can try for themselves (although a couple of the questions are already answered above). This quiz has a series of paired stories relaying very different experiences. Absent systematic analysis, each seems potentially persuasive and could be marshaled as evidence in support of a particular policy choice, making such human stories both extremely valuable and potentially dangerously misleading. Someone in favor of expanding Medicaid can find many true stories of people for whom the insurance policy kept them out of the ED and reduced the overall spending on their health care, yet the overall effect of the Medicaid expansion we studied was to increase ED use.

We ought not to ignore the personal stories that lend meaning and generate empathy and understanding in the name of evidence-based policy. Putting a human face on policy choices is crucial to internalizing the implications of these high-impact decisions. But policy makers need rigorous analysis to gauge the effect of policies that can have enormous consequences for so many.

Back to top
Article Information

Published: June 16, 2022. doi:10.1001/jamahealthforum.2022.2427

Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2022 Baicker K. JAMA Health Forum.

Corresponding Author: Katherine Baicker, PhD, University of Chicago, Harris School of Public Policy, 1307 E 60th St, Chicago, IL 60637 (kbaicker@uchicago.edu).

Conflict of Interest Disclosures: Dr Baicker reported receiving grants from the National Institute on Aging and serving on the board of directors for Eli Lilly and Mayo Clinic.

Finkelstein  A, Taubman  S.  Health care policy: randomize evaluations to improve health care delivery.   Science. 2015;347(6223):720-722. doi:10.1126/science.aaa2362PubMedGoogle ScholarCrossref
Khullar  D, Jena  AB.  “Natural experiments” in health care research.   JAMA Health Forum. 2021;2(6):e210290. doi:10.1001/jamahealthforum.2021.0290Google ScholarCrossref
Zaslavsky  AM.  Exploring potential causal inference through natural experiments.   JAMA Health Forum. 2021;2(6):e210289. doi:10.1001/jamahealthforum.2021.0289Google ScholarCrossref
Allen  HL. Personal narratives from Oregon’s Medicaid expansion. In: Lanford  D, ed.  Medicaid: Politics, Policy, and Key Issues. Nova Science Publishers; August 2020.
Taubman  SL, Allen  HL, Wright  BJ, Baicker  K, Finkelstein  AN.  Medicaid increases emergency-department use: evidence from Oregon’s health insurance experiment.   Science. 2014;343(6168):263-268. doi:10.1126/science.1246183PubMedGoogle ScholarCrossref
Baicker  K, Taubman  SL, Allen  HL,  et al; Oregon Health Study Group.  The Oregon experiment—effects of Medicaid on clinical outcomes.   N Engl J Med. 2013;368(18):1713-1722. doi:10.1056/NEJMsa1212321PubMedGoogle ScholarCrossref
Finkelstein  AN, Taubman  SL, Allen  HL, Wright  BJ, Baicker  K.  Effect of Medicaid coverage on ED use—further evidence from Oregon’s experiment.   N Engl J Med. 2016;375(16):1505-1507. doi:10.1056/NEJMp1609533PubMedGoogle ScholarCrossref
4 Comments for this article
Insurance and Emergency Department Use
Daniel Krell, M.D. | Retired PCP
It must be recognized that many medical practices (and, I expect, many more primary care practices than subspecialty practices) do not provide off-hours coverage for their patients. That leaves patients needing acute care in the position of either visiting an emergency department (ED) or quick-care facility. Many such practices have off-hours calls routed to a nurse, who can make some assessment and give some advice, or recommend either an ED or quick care visit, or contact with the office in the morning for an appointment (if the office schedule can accommodate a patient with a somewhat urgent issue). Insurance coverage of such ED or quick care visits is quite variable, some only covering the cost if the patient is admitted from the ED.

As a family practice resident who worked moonlighting in an ED. I would occasionally see people who did not really need emergency care but had Medicaid, nowhere else to go at that time, and had been unable to establish themselves with a practice in spite of looking for one. If I believed the person needed appropriate follow-up care and monitoring because of a significant issue, I would occasionally inform some of them of my schedule and invite them to return to the ED during one of my subsequent shifts. I did not want them to need preventable hospitalization or urgent care, again, or come to the ED and be seen by a clinician who would not provide them with what they really needed; I established my own, mini, primary care practice in the ED. Yes, that cost Medicaid more for the extra ED visits, but I'm sure I prevented enough hospital admissions to make up the difference. The article did not identify changes in rates of hospitalization among ED patients with or without insurance, so decreased rates of hospitalization could not be factored into the net changes in expenses.

Perhaps larger EDs could be paired with in-house quick care facilities that do not need to be as extensively/expensively staffed and equipped as are fully functional EDs. Patients could be triaged to the lower-intensity facility, saving money for insurance providers (private or government) and self-pay patients. I would not be surprised to find such facilities evolving from just acute care to acute and primary care providers, with patients being able to schedule visits and having continuity of care that is not afforded by EDs. The EDs would benefit by not having to provide services for patients who do not require their level of care.
An Experience Similar to Dr. Krell
Gerry Creager | Retired
A long time ago, I did moonlighting at a pediatric ER in a major medical center. Most (bordering on every) weekends we had an uninsured clinic between 0100 and 0600. More times than I can count, the first thing the parents said was along the lines of "I hope you can take care of my (baby|child) and I'm not taking you from a more important patient." Often, these families were undocumented, or so I suspected. The hospital was affiliated with a faith-based general hospital and funded by a charitable foundation; I was forbidden to ask status, even when it might have affected care. Many of these patients were working multiple jobs without benefits (a hint they might be undocumented). All too many of the kids I saw were admitted, and too many of those were direct-admits to ICU. Most of those admitted had been ill for several weeks and this was when the parents could get them in, but that was tied to the fact they'd become notably more ill and the parents were worried.

Situational conditions affect selection of care. If parents can't get off work without risking losing their jobs, to take a child to a pediatrician's appointment, what I found was they often tried to take care of their child as best they could. They sought crisis care when the condition changed... but still came in when they weren't working.

My wife, a retired CNM, noted that with Medicaid expansion she saw her panel expand in clinic, but she had to add an educational component to each visit, eg: if it's a minor ailment, like a headache, don't go to the ER over the weekend, but call the Clinic Nurse line, and come into an urgent-care appointment if needed on Monday. With Medicaid, they could see a provider in her clinic in a timely manner, a luxury they didn't have without any insurance.

I don't disagree with the findings of this article, but there are other considerations involved.
Hard Cases Make Bad Law
John Maher, MD | Providence, Little Company of Mary Hospital
Legal people say: "Hard cases make bad law." I agree. You can't generalize too well from a few unusual or difficult examples. I say this about scope-of-practice issues a lot. Folks bring up a few problems that allied health practitioners may have caused. It is not good evidence. You need numbers.
CONFLICT OF INTEREST: I teach at an Allied Health facility.
Scott Helmers, MD | Retired
What an excellent essay. It seems that in so many items of question we rush to find a simple answer. But complexity and nuance almost always lie beneath the surface. We can end up paralyzed in over-analysis, and yet rational thinking requires as much objective evidence as possible. Anecdotes can be wonderful illustrations while simultaneously distorting reality.