Patient Factors Associated With Appendectomy Within 30 Days of Initiating Antibiotic Treatment for Appendicitis

Key Points

Question  Which patient factors are associated with appendectomy after starting antibiotics for acute appendicitis?

Findings  This cohort study including 735 patients with appendicitis initially treated with antibiotics found that appendicitis severity was not associated with risk of appendectomy within 30 days, but presence of appendicolith was associated with a nearly 2-fold increased risk.

Meaning  These findings suggest that for patients considering antibiotics who want to better understand their chance of appendectomy, assessing appendicolith status may be informative.

Importance  Use of antibiotics for the treatment of appendicitis is safe and has been found to be noninferior to appendectomy based on self-reported health status at 30 days. Identifying patient characteristics associated with a greater likelihood of appendectomy within 30 days in those who initiate antibiotics could support more individualized decision-making.

Objective  To assess patient factors associated with undergoing appendectomy within 30 days of initiating antibiotics for appendicitis.

Design, Setting, and Participants  In this cohort study using data from the Comparison of Outcomes of Antibiotic Drugs and Appendectomy (CODA) randomized clinical trial, characteristics among patients who initiated antibiotics were compared between those who did and did not undergo appendectomy within 30 days. The study was conducted at 25 US medical centers; participants were enrolled between May 3, 2016, and February 5, 2020. A total of 1552 participants with acute appendicitis were randomized to antibiotics (776 participants) or appendectomy (776 participants). Data were analyzed from September 2020 to July 2021.

Exposures  Appendectomy vs antibiotics.

Main Outcomes and Measures  Conditional logistic regression models were fit to estimate associations between specific patient factors and the odds of undergoing appendectomy within 30 days after initiating antibiotics. A sensitivity analysis was performed excluding participants who underwent appendectomy within 30 days for nonclinical reasons.

Results  Of 776 participants initiating antibiotics (mean [SD] age, 38.3 [13.4] years; 286 [37%] women and 490 [63%] men), 735 participants had 30-day outcomes, including 154 participants (21%) who underwent appendectomy within 30 days. After adjustment for other factors, female sex (odds ratio [OR], 1.53; 95% CI, 1.01-2.31), radiographic finding of wider appendiceal diameter (OR per 1-mm increase, 1.09; 95% CI, 1.00-1.18), and presence of appendicolith (OR, 1.99; 95% CI, 1.28-3.10) were associated with increased odds of undergoing appendectomy within 30 days. Characteristics that are often associated with increased risk of complications (eg, advanced age, comorbid conditions) and those clinicians often use to describe appendicitis severity (eg, fever: OR, 1.28; 95% CI, 0.82-1.98) were not associated with odds of 30-day appendectomy. The sensitivity analysis limited to appendectomies performed for clinical reasons provided similar results regarding appendicolith (adjusted OR, 2.41; 95% CI, 1.49-3.91).

Conclusions and Relevance  This cohort study found that presence of an appendicolith was associated with a nearly 2-fold increased risk of undergoing appendectomy within 30 days of initiating antibiotics. Clinical characteristics often used to describe severity of appendicitis were not associated with odds of 30-day appendectomy. This information may help guide more individualized decision-making for people with appendicitis.

For more than 120 years, appendectomy has been the standard treatment for acute appendicitis. In the last 2 decades, 10 randomized clinical trials (RCTs)^1-10 in adults demonstrated that most patients who initiate antibiotic treatment can avoid appendectomy within 30 days. To support decision-making in acute appendicitis, understanding the associations between clinical and radiographic characteristics and the risk of 30-day appendectomy is critical.

While it has been suspected that more advanced appendicitis (eg, perforation) is associated with appendectomy after initiating antibiotics, the most commonly used staging system (American Association for the Surgery of Trauma [AAST]) is based on information that is neither available at the time of decision-making for antibiotics (ie, pathologic findings) nor very sensitive for perforation¹¹ (ie, radiographic information). Other scoring systems (eg, Alvarado score) were designed to diagnose appendicitis rather than estimate the response to antibiotics. It has been speculated that the presence of an appendicolith, ie, inspissated and mineralized stool, may be associated with failure of antibiotics. While appendicolith is associated with perforation and complications after appendectomy,¹² patients with appendicolith have been excluded from all but 1 prior RCT⁵ evaluating antibiotics. Thus, the association between appendicolith and avoidance of appendectomy in adults remains to be determined.

Among 776 participants randomized to antibiotics in the recently reported Comparison of Outcomes of Antibiotic Drugs and Appendectomy (CODA) trial (ClinicalTrials.gov identifier: NCT02800785),¹³ the 27% of participants with appendicolith had a higher rate of appendectomy by 90 days (time at which all participants had follow-up at the time of publication) compared with those without appendicolith (41% vs 25%). The degree to which appendicolith is a marker associated with more advanced disease or is an independent risk factor for appendectomy was not assessed. Using data from the antibiotic-assigned participants in the CODA trial, we describe demographic characteristics, clinical appendicitis stage information, and radiographic characteristics (AAST) among participants who underwent appendectomy 30 days after randomization to the antibiotics group compared with those who did not. Appendectomy at 30 days was selected as the end point of this analysis to focus on initial treatment response and exclude recurrence. In a sensitivity analysis, we focused on only patients who had an appendectomy for acute clinical reasons.

The trial protocol of the CODA trial was approved by institutional review boards at all 25 participating sites. All participants provided written informed consent. This study is reported following the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.

The CODA trial was a nonblinded, noninferiority randomized clinical trial designed to assess whether antibiotic treatment for appendicitis was noninferior to appendectomy. Patient stakeholders contributed to the design of the study, which is framed to allow for differences in how patients prioritize outcomes.^14,15 The trial was funded by the Patient-Centered Outcomes Research Institute. Complete details of the trial design and protocol have been published elsewhere.^13,16 To help support appendicitis-related decision-making during the COVID-19 pandemic, the primary outcome analysis was published in October 2020,¹³ earlier than planned, once all participants had 90-day follow-up information. In that report, 90-day appendectomy rates in the antibiotics-assigned group were presumed related to index appendicitis (≤30 days) or recurrence (>30 days). The focus of this secondary data analysis is on appendectomy at 30 days in the antibiotics group of the CODA trial.

The study was conducted at 25 US medical centers participating in the Comparative Effectiveness Research Translation Network, which is based at the University of Washington.¹⁷ Adult patients with a clinical diagnosis of uncomplicated appendicitis confirmed by imaging were approached consecutively in emergency departments by research coordinators. Study exclusion criteria have been described previously.^13,16 Participants were enrolled between May 3, 2016, and February 5, 2020. Both treatments in the CODA trial have been previously described.¹³

Appendectomy at 30 days is defined as appendectomy within 30 days of randomization (confirmed by participant survey). Participants were designated lost to follow-up (missing) if they did not respond to the 30-day and 90-day surveys, did not report an appendectomy on a prior survey, and did not have a record of appendectomy in their health record. When the participant reported having an appendectomy at a hospital other than the one from their index visit and did not provide a date of operation, the median time between the 2 most recent surveys was used to estimate the date of appendectomy. The reason for appendectomy was classified using information from the participant collected via scheduled surveys and from the participant’s health record review provided by the site. Reasons were grouped into 3 general categories: acute clinical (ie, development of diffuse peritonitis or worsening pain), nonacute clinical (eg, clinician concern for mucocele), and nonclinical (eg, participant concern for recurrence).

Our analysis considered key demographic, clinical, physiological, and radiologic variables (eTable 1 in Supplement 1). Factors were first characterized individually in their association with 30-day appendectomy and then grouped into 3 sets: physiologic, including demographic data (eg, age, sex, body mass index [BMI; calculated as weight in kilograms divided by height in meters squared], duration of symptoms, mean pain score in the previous 7 days, white blood cell count, fever, and nausea, vomiting, or anorexia); radiologic, including appendiceal diameter and abscess, perforation, or fat stranding; and presence or absence of an appendicolith. Race and ethnicity were determined by participant self-report, supplemented by the electronic medical record when missing, at the time of enrollment. Along with health literacy, poverty, and language, previous literature suggests race and ethnicity can be barriers to health care. Our focused list of individual factors reflects our effort to evaluate as many candidate factors as possible while recognizing both the limitations owing to the overall number of appendectomies and the missingness and frequency of responses for any factor. Furthermore, we also reduced repetition of measurements that reflect similar constructs and therefore may be collinear; for example, we included just 1 pain score from many available. We grouped the physiologic factors together, as these are readily available at presentation and may be associated with the appendectomy outcome through a biological pathway involving disease severity. We then explored the relative association of appendicolith, first evaluating physiologic factors, then including other radiologic features, such as appendiceal width and signs of perforation or severe phlegmon. The CODA data set for this analysis includes all information as of December 14, 2020.

Baseline demographic and clinical characteristics of participants randomized to antibiotics were described using mean and SD for continuous measures and number and percentage for categorical variables. Characteristics are also summarized overall and by outcome status: appendectomy at 30 days vs no appendectomy within 30 days. To assess variation in appendectomy across practice sites, we plotted the proportion of patients who underwent appendectomy at each site. The frequencies and percentages of each reason for appendectomy were also tabulated.

Given the observed variation across sites in the proportion of participants undergoing appendectomy, we used conditional logistic regression to focus on patient factors while addressing site-level variables in both univariate and multivariable models of appendectomy status (appendectomy at 30 days vs no appendectomy within 30 days). The largest contributing site recruited 15% of the cohort, and sites with fewer than 20 randomized participants were grouped together or with another geographically similar site if one was available. Four multivariable models were fit to assess the relative associations of appendicolith and other radiologic findings with appendectomy. The base model included age, sex, BMI, duration of symptoms, mean pain score, white blood cell count, fever, and report of nausea, vomiting, or anorexia. The base + A model added appendicolith to the base model. The base + R model included all variables in the base model and the radiologic features of appendiceal diameter and evidence of perforation, abscess, or fat stranding. The base + R + A model was the full model that included the physiologic variables from the base model, appendicolith, and all other radiologic variables. Nested models were compared using a multivariate Wald test,¹⁸ with α set at .05.

Missing data on appendectomy status at 30 days and any candidate factors were imputed using multivariate imputation by chained equations algorithms in R software versions 3.6.1 and 4.0.3 (R Project for Statistical Computing). All variables included in the full base + R + A model were included in the imputation process, as well as the outcome and other participant-reported variables and variables from health records collected at index. A full list of variables used for imputation is provided in eAppendix 1 in Supplement 1. Model coefficients were pooled across the 10 imputation sets and are shown with corresponding 95% CIs. A planned sensitivity analysis repeated the process of comparing 4 nested models but examined appendectomy for acute clinical reasons, excluding appendectomies that were performed for nonclinical or nonacute clinical reasons.

Finally, because conditional logistic regression adjusted for site without actually quantifying site associations, we performed a post hoc assessment of the association of site after controlling for all patient factors in the full model. To assess variability across sites, we fit a mixed-effects logistic regression model treating site as a random effect and performing a variance component test (ie, testing the SD of the random site effect equal to vs >0) pooling over imputed data sets.¹⁹ All analyses were performed in R statistical software versions 3.6.1 and 4.0.3. P values were 2-sided, and statistical significance was set at α = .05. Data were analyzed from September 2020 to July 2021.

Of 776 participants (mean [SD] age, 38.3 [13.4] years; 286 [37%] women and 490 [63%] men) randomized to antibiotics, 30-day appendectomy status was available for 735 participants (95%); 154 participants (21%) underwent appendectomy within 30 days. Among sites randomizing at least 20 participants, frequency of appendectomy at 30 days ranged from 8% to 35% (Figure). Participant sociodemographic and clinical characteristics at the time of enrollment are described in Table 1. Appendicolith was found in 212 participants (27%) overall, in 65 participants (42%) who underwent appendectomy within 30 days, and in 138 participants (24%) who did not have an operation.

The reported reason for appendectomy was most often an acute clinical reason (116 participants [77%]), with worsening signs and symptoms of appendicitis being the most common reason given for undergoing surgical treatment (Table 2). Among those who had an appendectomy for nonclinical reasons, participant worry or concern for recurrence and consultation with friend or family were the most frequent motivations for surgical treatment (Table 2). Negative appendectomy (without pathological evidence of acute appendicitis) was uncommon, occurring in 4% (95% CI, 2%-9%) of patients but occurred more often in women (8% [95% CI, 3%-18%] of women) than in men (1% [95% CI, 0%-6%] of men).

To create a more parsimonious multivariable model, since BMIs of 25 to 30 and 30 to 35 had similar associations with 30-day appendectomy (Table 1), we grouped BMI 25 to 35 together. In univariate models (Table 3), BMI of 25 to 35 vs less than 25 was associated with increased odds of appendectomy (odds ratio [OR], 1.77; 95% CI, 1.14-2.75). BMI greater than 35 was associated with lower odds of appendectomy compared with BMI less than 25 (OR, 0.75; 95% CI, 0.42-1.34). Appendiceal diameter (OR per 1-mm increase, 1.14; 95% CI, 1.06-1.22) and presence of appendicolith (OR, 2.56; 95% CI, 1.73-3.79) were also associated with increased odds of appendectomy.

Model comparison of base + A vs base was statistically significant (pooled Wald test = 17.1; P < .001), indicating that the inclusion of appendicolith status improved model fit (eTable 1 in Supplement 1). The full model (base + R + A) had improved model fit compared with the model that included all variables except appendicolith (base + R, pooled Wald test = 9.5; P = .003), suggesting that appendicolith status provides more information about who was at risk of an appendectomy in 30 days beyond that already accounted for by appendiceal diameter and evidence of perforation, abscess, or fat stranding (Table 3 and eTable 1 in Supplement 1).

In the full model, several factors were associated with increased odds of 30-day appendectomy after controlling for other physiologic and radiologic factors: female sex (OR, 1.53; 95% CI, 1.01-2.31), increased appendiceal diameter (OR per 1-mm increase, 1.09; 95% CI, 1.00-1.18), and presence of appendicolith (OR, 1.99; 95% CI, 1.28-3.10) (Table 3). Characteristics that are often associated with increased risk of complications (eg, advanced age, comorbid conditions) and those clinicians often use to describe appendicitis severity (eg, fever: OR, 1.28; 95% CI, 0.82-1.98) were not associated with odds of 30-day appendectomy. The ORs for BMI 25 to 35 vs less than 25 and mean pain score in the previous 7 days were greater than 1.00, but 95% CI lower bounds were less than 1.00 (BMI: OR, 1.60; 95% CI, 0.99-1.60; pain score: OR, 1.03; 95% CI, 0.99-1.14). ORs from the base, base + A, and base + R models are shown in eTable 2 in Supplement 1. We performed an exploratory, post hoc analysis that added an interaction term between appendiceal diameter and appendicolith to the full model. This coefficient was not statistically significant (OR, 0.96; 95% CI, 0.82-1.11).

The planned sensitivity analysis of appendectomy performed for clinical reasons vs no appendectomy within 30 days produced similar results (eTable 3 in Supplement 1). Notably, the association of appendicolith status with 30-day appendectomy strengthened (OR, 2.41; 95% CI, 1.49-3.91). Both model comparisons remained statistically significant (base + A vs base, pooled Wald test = 23.1; P < .001; base + R + A vs base + R, pooled Wald test = 13.0; P < .001) (eTable 1 in Supplement 1). Finally, the variance component test from the mixed-effects model was statistically significant (D₂, 7.9; P = .003), indicating that, after controlling for physiologic and radiologic patient factors, the odds of appendectomy still varied across sites.

This cohort study using data from the CODA trial found a significant and persistent association between radiographic evidence of an appendicolith and 30-day appendectomy in participants initiating antibiotics (a nearly 2-fold increase in odds), even after controlling for physiologic characteristics and other radiographic findings. Of note, several factors often presumed by clinicians to be consistent with more severe appendicitis (ie, fever, higher white blood cell count, and radiographic evidence of perforation) were not independently associated with appendectomy, but ORs were in the expected direction. For more than a century,²⁰ surgeons have looked for patient characteristics to help guide the treatment of appendicitis. Increasing evidence that antibiotics can be used to treat appendicitis has made the search for such factors even more relevant. Understanding which patients are at higher risk of appendectomy after starting antibiotics might be one way to guide treatment decisions.

An emerging theory proposes that appendicitis has 2 phenotypes,²¹ a simple type and a severe type, and that the first does not necessarily evolve to the second if left untreated. The simple type is thought likely to respond to antibiotics²² and may even be self-limiting.^23-25 The severe type is thought to be associated with perforation on presentation and less likely to resolve successfully with antibiotics. Given the limited exclusion criteria in the CODA trial, it is likely that patients with both types of appendicitis were included in the study. Unfortunately, while the 2-phenotype theory may have merit, we did not identify any disease severity characteristics that were associated with appendectomy at 30 days.

Appendicolith was the main factor we found to be associated with 30-day appendectomy. While we found that the odds of appendectomy were approximately 2-fold higher for those with an appendicolith compared with those without (unadjusted rates of 42% vs 24%), approximately 6 of 10 patients with appendicolith did not undergo appendectomy within 30 days. For many individuals, appendicoliths are incidental findings, identified by computed tomography (CT) and autopsy series in a variable proportion (4%-25%) of those without appendicitis.^26-28 In other patients, appendicolith may be a marker or even a cause of more severe appendicitis.^29,30 Most of what we know about the success of antibiotics in patients with appendicolith comes from the pediatric population. Although patients with appendicolith are often excluded from pediatric trials,³¹ a few RCTs have included pediatric patients with appendicolith, and a 2017 meta-analysis of these patients³² found a much lower likelihood of response. Among adults, only 1 RCT of antibiotics and appendectomy, by Vons et al,⁵ included patients with appendicolith in the antibiotics-assigned group. In that study, 6 of 19 patients (32%) did not have resolution of appendicitis within a month and underwent appendectomy, a rate very similar to that seen in CODA participants with an appendicolith (31%).¹³ Another challenge in assessing the role of appendicolith and appendectomy is that surgeons were not blinded in the CODA trial or the study by Vons et al⁵; their beliefs about appendicolith may also play a role in the greater use of appendectomy in that group.

The nearly 2-fold increased odds of appendectomy among people with an appendicolith suggest that, at least for patients interested in the antibiotic treatment option, obtaining information about appendicolith status may be important. For those patients, given the presumed role of imaging in identifying appendicolith, using CT instead of ultrasonography as part of the diagnostic evaluation for appendicitis may be a better alternative.³³ While CT was used for diagnosis in 97% of participants in the CODA trial, ultrasonography may be more widely in use across the United States and globally.³⁴ The use of CT for this purpose needs to be balanced against the low sensitivity of CT in identifying appendicolith (55%) found in a study by Singh and Mariadason,¹² as well as increased radiation exposure.³⁵ Furthermore, knowledge of an appendicolith may or may not influence decision-making. When determining treatment in light of an appendicolith, clinicians and patients should consider the risk of appendectomy in the context of several outcomes, such as overall well-being, time until relief of symptoms, time in health care, safety events, and time away from work—all parameters for which appendicolith was associated with worse outcomes, albeit with low frequencies of events.^13,36 For some patients, this added risk may make antibiotics a less attractive option, while for others, these risks may be outweighed by the benefits of potentially avoiding an urgent operation. Interestingly, patient advisor feedback during the design phase of the CODA trial indicated that some patients would find antibiotics to be a preferable alternative even if they were in a subgroup with an up to 75% chance of appendectomy after antibiotics.

An unexpected finding in the adjusted analysis was an association of female sex and appendectomy, an observation not identified on univariate analysis or in the sensitivity analysis limited to appendectomies with clinical indication. In considering associations between sex and clinical outcomes,³⁷ there is robust debate about potential pathways, such as biological, social, or other, that may be involved. The CODA trial did not include data that might inform this question; therefore, we caution against applying this finding in clinical decision-making. Women were more likely to have a negative appendectomy. This finding may also be associated with factors that we could not or did not study, such as misdiagnosis at index admission, adequacy of pain control, concerns about the impact of appendicitis on fertility, or prior experiences with the health care system that could have influenced the 5 participants’ decision to undergo appendectomy. Underrecognition and undertreatment of acute abdominal pain in women³⁸ may also have been a factor.

After adjusting for physiologic and radiologic factors, there remained important variation across enrollment sites in the rate of appendectomy after initiating antibiotic treatment. The reasons for this variation are unclear and may be related to process (eg, pain control, encouragement from staff and clinicians) as well as structural issues (eg, ease of reentering the health care system with concerns or complaints, availability of telephone-based support); these highlight the relative complexity of the antibiotic treatment intervention. Individual bias from clinicians and patients and experience and comfort with nonoperative treatment that varies by site may also play a role for this observation. To counter the risk of clinician bias and the potential for undue influence,³⁹ evidence-based shared decision-making processes have emerged as a standard.⁴⁰

This study had several limitations. Most critically, we were not able to discern which appendectomies were a consequence of a failure of antibiotics vs other unmeasured reasons, such as insufficient pain control. While our sensitivity analysis attempted to focus on appendectomies owing to physiologic failure, participants often voiced multiple reasons for undergoing appendectomy, and it was difficult to determine if the participant was requesting the treatment solely owing to their symptoms or if they were also influenced by the preferences of their treating clinician. Similarly, we chose a 30-day window rather than a 90-day window to focus on the index case of appendicitis; however, we cannot rule out recurrence within this timeframe. Generalizability of findings is another limitation: this study may only be relevant to those without CODA exclusion criteria (eg, abscess and severe phlegmon) and those who undergo a CT scan to determine appendicolith status. We also did not power this study to assess this outcome or these specific factors. Owing to low prevalence, we combined perforation, abscess, and phlegmon into 1 variable, which may have obscured the individual risk associated with 1 or more of these features. Furthermore, while we did select candidate factors with the intention of avoiding correlated variables, multicollinearity may still be attenuating the strength of some findings. We chose not to use a dimension-reduction procedure, such as factor analysis, as this would limit our ability to see the risk associated with each individual characteristic; however, doing so would have allowed the inclusion of more factors. We also did not include demographic variables in the nested model analysis that could be seen as barriers to health care, as it was impossible to know if the barrier was preventing a participant from getting a needed appendectomy or pushing them to get an appendectomy that was not required or desired by the participant. We instead focused on likely biological pathways that might be associated with a patient undergoing appendectomy. Additionally, these are exploratory models that should not be used for any given patient. Cohorts with different inclusion and exclusion criteria and more detailed information on the reason for appendectomy are needed to confirm these findings.

This cohort study found that the presence of appendicolith was associated with the greatest risk for undergoing appendectomy within 30 days of initiating antibiotics, among examined factors. Multiple factors that clinicians associate with disease severity were not independently associated with appendectomy. Beyond these factors, site characteristics, potentially including physician decision-making and unmeasured processes of care and structural barriers, may also be involved in determining which patients get an appendectomy. These findings may be helpful in developing patient-facing tools to support informed decision-making in the treatment of appendicitis.

Accepted for Publication: October 24, 2021.

Published Online: January 12, 2022. doi:10.1001/jamasurg.2021.6900

Corresponding Author: David R. Flum, MD, MPH, University of Washington, Box 356410, 1959 NE Pacific St, Seattle, WA 98195 (daveflum@uw.edu).

Writing committee of the CODA Collaborative: Sarah E. Monsell, MS; Emily C. Voldal, BA; Giana H. Davidson, MD, MPH; Katherine Fischkoff, MD; Natasha Coleman, MD; Bonnie Bizzell, MEd, MBA; Thea Price, MD; Mayur Narayan, MD, MPH, MBA, MHPE; Nicole Siparsky, MD; Callie M. Thompson, MD; Patricia Ayoung-Chee, MD, MPH; Stephen R. Odom, MD; Sabrina Sanchez, MD, MPH; F. Thurston Drake, MD, MPH; Jeffrey Johnson, MD; Joseph Cuschieri, MD; Heather L. Evans, MD, MS; Mike K. Liang, MD; Karen McGrane, MD; Quinton Hatch, MD; Jesse Victory, DO; Jon Wisler, MD; Matthew Salzberg, MD, MBA; Lisa Ferrigno, MD, MPH; Amy Kaji, MD, PhD; Daniel A. DeUgarte, MD; Melinda Maggard Gibbons, MD; Hasan B. Alam, MD; John Scott, MD; Lillian S. Kao, MD, MS; Wesley H. Self, MD, MPH; Robert J. Winchell, MD; Cassandra M. Villegas, MD, MPH; David A. Talan, MD; Larry G. Kessler, ScD; Danielle C. Lavallee, PharmD, PhD; Anusha Krishnadasan, PhD; Sarah O. Lawrence, MS; Bryan Comstock, MS; Erin Fannon, BA; David R. Flum, MD, MPH; Patrick J. Heagerty, PhD.

Affiliations of Writing committee of the CODA Collaborative: University of Washington, Seattle (Monsell, Voldal, Davidson, Bizzell, Kessler, Lavallee, Lawrence, Comstock, Fannon, Flum, Heagerty); Columbia University Medical Center, New York, New York (Fischkoff, Coleman); Rush University Medical Center, Chicago, Illinois (Price, Siparsky); Weill Cornell Medical Center, New York, New York (Narayan, Winchell, Villegas); Vanderbilt University Medical Center, Nashville, Tennessee (Thompson, Self); University of Utah, Salt Lake City (Thompson); Tisch Hospital NYU Langone Medical Center, New York, New York (Ayoung-Chee); Morehouse School of Medicine, Atlanta, Georgia (Ayoung-Chee); Beth Israel Deaconess Medical Center, Boston, Massachusetts (Odom); Boston University Medical Center, Boston, Massachusetts (Sanchez, Drake); Henry Ford Health System, Detroit, Michigan (Johnson); Harborview Medical Center, UW Medicine, Seattle, Washington (Cuschieri, Evans); University of California, San Francisco (Cuschieri); University of South Carolina, Charleston (Evans); Lyndon B. Johnson General Hospital, University of Texas, Houston (Liang); University of Houston, HCA Healthcare, Kingwood, Texas (Liang); Madigan Army Medical Center, Tacoma, Washington (McGrane, Hatch); Bellevue Hospital Center, NYU School of Medicine, New York, New York (Victory); Ohio State University Medical Center, Columbus (Wisler); UCHealth University of Colorado Hospital, Denver (Salzberg, Ferrigno); Harbor–UCLA Medical Center, West Carson, California (Kaji, DeUgarte); Olive View–UCLA Medical Center, Los Angeles, California (Gibbons, Talan, Krishnadasan); University of Michigan Medical Center, Ann Arbor (Alam, Scott); Northwestern University, Evanston, Illinois (Alam); McGovern Medical School, The University of Texas Health Science Center at Houston, Houston (Kao); Ronald Reagan UCLA Medical Center, Los Angeles, California (Talan); BC Academic Health Science Network, Vancouver, Canada (Lavallee).

Author Contributions: Mss Monsell and Voldal had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Monsell, Voldal, Davidson, Bizzell, Price, Evans, Liang, Victory, Kaji, DeUgarte, Talan, Kessler, Lavallee, Krishnadasan, Comstock, Fannon, Flum, Heagerty.

Acquisition, analysis, or interpretation of data: Monsell, Voldal, Davidson, Fischkoff, Coleman, Price, Narayan, Siparsky, Thompson, Ayoung-Chee, Odom, Sanchez, Drake, Johnson, Cuschieri, Evans, Liang, McGrane, Hatch, Victory, Wisler, Salzberg, Ferrigno, Kaji, DeUgarte, Maggard Gibbons, Alam, Scott, Kao, Self, Winchell, Villegas, Talan, Kessler, Krishnadasan, Lawrence, Comstock, Fannon, Flum, Heagerty.

Drafting of the manuscript: Monsell, Voldal, Davidson, Coleman, Price, Narayan, Siparsky, Thompson, Liang, Victory, Wisler, Salzberg, Lavallee, Fannon, Flum.

Critical revision of the manuscript for important intellectual content: Monsell, Voldal, Davidson, Fischkoff, Bizzell, Price, Narayan, Siparsky, Thompson, Ayoung-Chee, Odom, Sanchez, Drake, Johnson, Cuschieri, Evans, Liang, McGrane, Hatch, Victory, Wisler, Salzberg, Ferrigno, Kaji, DeUgarte, Maggard Gibbons, Alam, Scott, Kao, Self, Winchell, Villegas, Talan, Kessler, Krishnadasan, Lawrence, Comstock, Fannon, Flum, Heagerty.

Statistical analysis: Monsell, Voldal, Davidson, Odom, Kaji, Comstock, Heagerty.

Obtained funding: Davidson, Kessler, Comstock, Fannon, Flum.

Administrative, technical, or material support: Davidson, Bizzell, Price, Thompson, Ayoung-Chee, Drake, Johnson, Liang, McGrane, Hatch, Victory, Wisler, Salzberg, Ferrigno, DeUgarte, Alam, Scott, Self, Villegas, Talan, Kessler, Lavallee, Krishnadasan, Lawrence, Fannon, Flum.

Supervision: Monsell, Davidson, Fischkoff, Bizzell, Sanchez, Victory, Maggard Gibbons, Self, Villegas, Talan, Kessler, Flum.

Conflict of Interest Disclosures: Dr Price reported serving as a consultant for Kerecis, Acera, and Medline. Dr Thompson reported serving as a reviewer for the Shriner’s Research Fund and receiving personal fees from UpToDate. Dr Evans reported receiving personal fees from Tetraphase Pharmaceuticals outside the submitted work. Dr Winchell reported receiving personal fees from Stryker. No other disclosures were reported.

Funding/Support: This study was supported by grant No. 1409-240099 from the Patient-Centered Outcomes Research Institute.

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

CODA Collaborative Members: The CODA Collaborative members are listed in eAppendix 2 in Supplement 1 and Supplement 2.

Disclaimer: The views presented in this work are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its board of governors, or its methodology committee.

Additional Contributions: Margo Godersky, MPH, coordinated the development of this manuscript, and Caroline Shevrin, MS, edited it. Catalina Gibbons, BA; Heather Harris, BS; Madison Hollcroft, BS; Mariam Hantouli, MD; Sandra Mata-Diaz, BS; Stephanie Benipal, BA; Stephanie Herrera, BA; Susan Robles, BA; Solange Mecham, BA; Jenney Lee, MA; Alejandra Silva Hernandez, MPH; Sarah Rinehart, BS; Addy Borges, BA; Jennifer Cox, MS; Jeanette Yang, MS; Maianna Dematteis, BS; and Alex Ruiz, BS (University of Washington), served on the research staff for this study. All were compensated.