eMethods. Instrumental Variable and Modeling Specifications
eTable 1. Diagnostic and Procedure Codes for Cohort Identification
eTable 2. Covariate Balance for Pooled Sample and Distance Categories
Customize your JAMA Network experience by selecting one or more topics from the list below.
Scales CD, Lai JC, Dick AW, et al. Comparative Effectiveness of Shock Wave Lithotripsy and Ureteroscopy for Treating Patients With Kidney Stones. JAMA Surg. 2014;149(7):648–653. doi:10.1001/jamasurg.2014.336
Shock wave lithotripsy (SWL) and ureteroscopy (URS) account for more than 90% of procedural interventions for kidney stones, which affect 1 in 11 persons in the United States. Efficacy data for SWL are more than 20 years old. Advances in URS, along with emerging evidence of reduced efficacy of modern lithotripters, have created uncertainty regarding the comparative effectiveness of these 2 treatment options.
To compare the effectiveness of SWL and URS to fragment or remove urinary stones in a large private payer cohort.
Design, Setting, and Participants
We performed a retrospective cohort study of privately insured beneficiaries who had an emergency department visit for a kidney stone and subsequently underwent SWL or URS. Using an instrumental variable approach to control for observed and unobserved differences between the 2 groups, we created a bivariate probit model to estimate the probability of repeat intervention following an initial procedure.
Main Outcomes and Measures
A second procedure (SWL or URS) within 120 days of an initial intervention to fragment or remove or a kidney stone.
Following an acute care visit for a kidney stone, 21 937 patients (45.8%) underwent SWL and 25 914 patients (54.2%) underwent URS to fragment or remove the stone. After the initial URS, 4852 patients (18.7%) underwent an additional fragmentation or removal procedure compared with 5186 patients (23.6%) after the initial SWL (P < .001). After adjusting for observed and unobserved variables, the estimated probabilities of repeat intervention were 11.0% (95% CI, 10.9-11.1) following SWL and 0.3% (95% CI, 0.325-0.329) following URS.
Conclusions and Relevance
Among privately insured beneficiaries requiring procedural intervention to remove a symptomatic stone, repeat intervention is more likely following SWL. For the marginal patient (as opposed to the average patient), the probability of repeat intervention is substantially higher.
Kidney stones impose a significant burden of disease in the United States. The prevalence of kidney stones has increased dramatically since 1976, a change that is likely driven by the obesity epidemic.1,2 Stone disease affects approximately 1 of every 11 persons in the United States, a prevalence similar to that of diabetes mellitus.3 Up to 50% of individuals forming stones will have a recurrence within 5 years.4 Dietary and lifestyle factors contribute substantially to the risk of stone disease, and emerging physiological data suggest that stone disease should be considered a metabolic disorder, punctuated by attacks of periodically formed, symptomatic kidney stones.5-8
The costs to society of kidney stones are significant. In terms of aggregate annual medical expenditures, kidney stones represent one of the most costly urologic conditions, with greater than $10 billion in expenditures in 2006 for treating patients with kidney stones.9,10 Indirect costs such as work loss and temporary disability are also an important contribution to the disease burden, particularly because kidney stones affect a largely working-age population.11 Among the important cost drivers of kidney stone disease are procedural interventions to treat patients with symptomatic stones.
Fragmentation and removal of symptomatic stones relieves pain and prevents harm to renal function from chronic obstruction. Two dominant modalities exist for procedural intervention, namely, shock wave lithotripsy (SWL) and ureteroscopy (URS). Shock wave lithotripsy is noninvasive and uses high-energy acoustic waves to fragment stones. Ureteroscopy is a minimally invasive endoscopic technique that can access all parts of the ureter and renal collecting system, typically using a laser to fragment stones. Together, these 2 modalities represent more than 90% of the procedures performed to remove renal and ureteral stones in the United States.12 Both are considered first-line options for the management of symptomatic stones, although SWL may have been slightly less efficient than URS in published series.13,14 More recently developed lithotripters seem to be less effective than the original device (modified HM3 lithotripter; Dornier MedTech), which was used to generate most of the clinical trial data comparing URS and SWL.15,16 In addition, ureteroscopic technology has experienced important advances during the past decade.17-19
Reducing waste and avoiding retreatment are important factors to drive value in the provision of health care.20,21 Prior analyses of clinical trial data suggest that a trade-off for the completely noninvasive nature of SWL is a greater need for retreatment.13,14,22 In addition, claims data suggest that unadjusted retreatment rates for SWL can be up to triple those of URS.12 However, that analysis did not account for important clinical factors such as stone size or location, nor did it incorporate other potential confounders such as patient preferences.
Given that both technologies are well established and only incrementally evolving, it is unlikely that new randomized clinical trials comparing the 2 will occur. In addition, creating a large registry of URS and SWL procedures, along with data regarding patient preferences, to understand the comparative effectiveness of these technologies would be time-consuming and expensive. Therefore, we performed an administrative claims–based analysis using econometric techniques to compare repeat interventions among patients initially treated with SWL or URS after adjusting for important unmeasured confounders such as stone size and location and patient preferences. The use of appropriate econometric techniques such as instrumental variable analysis seeks to balance observed and unobserved confounders among treatment groups using naturally occurring variation in observational data.23-25 Through the use of a bivariate probit model in a large cross-section of commercially insured beneficiaries, we sought to compare the effectiveness of SWL vs URS in the treatment of patients with urinary stones.
The institutional review board of the RAND Corporation determined that the study design was exempt from the requirement for review given the nature of the data. To identify patients who were likely to have a symptomatic stone, we used data from the MarketScan Commercial Claims and Encounters data set (http://www.bridgetodata.org/node/987) to select beneficiaries with an emergency department or urgent care center clinical encounter (the index encounter) for a kidney or ureteral stone between January 1, 2002, and December 31, 2010. The MarketScan data include encounter-level health claims for clinical services and pharmacy claims. Although they are not nationally representative, these data represent a convenience sample of commercially insured beneficiaries in the United States.
We further restricted our cohort of patients receiving emergent or urgent care to those who had no claim for any clinic visit or procedure for urinary stones within 12 months before the index encounter and who had at least 8 months of continuous enrollment following the index encounter. Finally, we limited our cohort to those beneficiaries who underwent URS or SWL within 4 months of the index encounter. These exclusion criteria resulted in a study sample of 47 851 patients who underwent treatment to fragment or remove a presumably symptomatic urinary stone.
We identified procedures for treating patients with kidney or ureteral stones using Current Procedural Terminology codes (eTable 1 in the Supplement).12 We created a binary treatment variable for SWL vs URS.
Demographic characteristics for each patient were identified from MarketScan data. These included age and sex; patient race/ethnicity is not included in this data set. The year of treatment was included to address secular trends in practice during the 9-year study period. Preexisting comorbid conditions were summarized by calculating the Charlson Comorbidity Index using claims submitted within 365 days before the index encounter. Geographic variation in the epidemiology of stone disease exists, as well as variation in practice patterns. Therefore, we included census regions, as reported in the MarketScan data.
The primary outcome was a second procedure to fragment or remove stones within 120 days of the initial intervention. We elected a window of 120 days to capture potential second procedures performed outside of the global period for these procedures. A second URS or SWL was considered an outcome event, regardless of which initial procedure the patient underwent.
We used the χ2 test and t test, as appropriate, to compare patient-level covariates between procedures. The χ2 test was also used to compare the proportions of patients undergoing a second procedure between SWL and URS.
An important consideration in any observational analysis is the potential for estimation bias due to unmeasured variables that can confound the results and limit causal inference. In particular, if unobserved factors that affect the treatment decision also influence the outcomes, standard estimates of treatment effects might be biased. For example, the size and location of urinary stones are important determinants of treatment (SWL vs URS) and outcomes.13 Therefore, treatment effects may be biased by the lack of equivalence of the unobserved factors. Previous claims-based comparisons of SWL and URS have failed to address this potential confounding.12 We adopted an alternative approach in which we specify and estimate a simultaneous equations model that addresses this concern. Our approach, which is analogous to instrumental variable analysis, used an econometric technique designed to balance the effect of measured and unmeasured confounders among treatment groups23 and, if specified correctly, allows for causal inference of treatment effects.
The validity of our approach is contingent on the availability of instrumental variables that satisfy 2 conditions. First, they are strongly correlated with treatment decisions. Second, conditional on other observable factors, they are only related to the outcome (in our case, a second intervention to remove stones) through their influence on treatments.23,25
We identified 3 potential instruments for the analysis. We hypothesized that the distance to a hospital with SWL vs URS capability would influence the probability of patients receiving one of the treatments and that this is the only mechanism by which the distance to a hospital with SWL vs URS capability would influence the receipt of a second procedure within 120 days of the initial intervention. Similar distances have been used effectively in other instrumental variable analyses.24,26 We hypothesized that the per capita density of urologists and surgeons would reflect the availability of surgical services at the local area level, as well as the intensity of potential competition among providers, and that these factors would influence the probability of patients receiving one of the treatments (eMethods and eTable2 in the Supplement).
Given that the regression coefficients of probit models are not intuitive to interpret, we then calculated model-derived probabilities of repeat treatment for ease of interpretation. As a sensitivity analysis, we constructed separate models for patients treated from 2002 to 2005 and from 2006 to 2010. Statistical testing was 2-sided, with the type I error rate set at 5%. Analyses were performed using commercial software (SAS version 9.2; SAS Institute Inc and STATA version 11.0; StataCorp LP).
The cohort included 47 851 individuals who underwent an initial procedure for fragmentation or removal of a urinary stone. Among these, 25 914 (54.2%) underwent URS as the first procedure and 21 937 (45.8%) underwent SWL as the first procedure. The individuals undergoing SWL were slightly older than those undergoing URS (Table 1). Women were more likely to undergo URS as the first procedure (41.3% in the URS group vs 37.6% in the SWL group, P < .001). Minor regional variation in the use of the 2 procedures existed. Most patients had a Charlson Comorbidity Index of 0.
Within 120 days of the initial procedure, approximately 1 in 5 individuals (10 038 of 47 851 [21.0%]) underwent an additional procedure to fragment or remove urinary stones. After the initial URS, 4852 patients (18.7%) underwent an additional fragmentation or removal procedure compared with 5186 patients (23.6%) after the initial SWL (P < .001). Those undergoing initial SWL were more likely to require a second procedure than those undergoing initial URS (Table 2). The relative risk of a second procedure following SWL was approximately 25% higher than after URS (relative risk, 1.26; P < .001). On multivariate analysis (naive model), differences in the proportions of individuals undergoing additional fragmentation or removal procedures persisted after adjusting for observable confounders such as age, sex, and census region. Adjustment for these potential confounders did not substantively change the predicted probability of additional treatments.
We then used a 2-equation bivariate probit model to adjust for observed and unobserved potential confounders. The likelihood ratio test of the 2-equation model demonstrated that the results were statistically significant (P < .001). The 2-equation model substantially reduced the predicted probability of additional treatments for SWL and URS (Table 2). After adjusting for observed and unobserved potential confounders, the predicted probability of an additional procedure for the marginal patient (as opposed to the average patient) undergoing SWL was 11.0%. In contrast, after adjusting for observed and unobserved potential confounders, the predicted probability of an additional procedure for the marginal patient undergoing URS was 0.3%.
In this novel analysis, we use an econometric approach to compare the effectiveness of SWL and URS. Our findings demonstrate that, on an unadjusted basis and after adjusting for observable potential confounders, the relative risk of an additional stone fragmentation or removal procedure is approximately 25% higher following SWL compared with URS. After using a bivariate probit model to address unobserved potential confounders (eg, stone size), the probability of additional procedures decreases for SWL and URS, but the marginal patient undergoing SWL is substantially more likely to experience an additional procedure to remove a previously treated stone compared with the same patient undergoing URS.
The efficacy of SWL and URS for fragmenting and removing urinary stones is well established.13,27 For ureteral stones, SWL and URS are considered first-line therapy.13 However, for ureteral stones exceeding 10 mm, ureteroscopic fragmentation generally results in higher stone-free rates with fewer procedures.13,28-30 For renal stones 10 mm or less, SWL and URS have similar efficacy in clinical trials.31
In contrast to abundant efficacy data, the comparative effectiveness of SWL and URS outside of controlled clinical trials is poorly documented. One claims-based cost analysis suggested that the mean number of procedures per patient is slightly higher for SWL (1.22 vs 1.12 for URS).11 Prior work from the Urologic Diseases in America Project12 examined practice patterns for urinary stone fragmentation and removal procedures among unselected Medicare beneficiaries. In a longitudinal cohort, 38% of Medicare beneficiaries undergoing SWL had an additional stone fragmentation or removal procedure within 120 days compared with 12% of those undergoing URS. Most important, neither of these analyses adjusted for important covariates such as stone size and location, nor did they focus on patients with symptomatic stones (as opposed to patients potentially treated for incidentally detected stones).
Our findings suggest that, for the marginal patient with a symptomatic stone amenable to SWL or URS, first-line treatment with SWL would result in approximately 11% of patients undergoing an additional stone removal procedure. By comparison, the model results suggest that the same patient undergoing first-line treatment with URS faces less than a 1% chance of an additional stone removal procedure. Most important, our analytic approach should control for observable factors (eg, patient age, sex, comorbid conditions, and census region) and factors that we could not observe in our analysis (eg, patient obesity, race/ethnicity, size and location of the targeted stone, ureteral stent placement, bilateral stones, and patient preference). This approach strengthens the causal inference of our findings.23
The contrast between efficacy data and comparative effectiveness data for these technologies is noteworthy. Some of this difference could be due to evolving technical capabilities of the 2 procedures. Much of the efficacy data for SWL was generated using the original lithotripter (modified HM3) from more than 20 years ago.32,33 Newer lithotripter models have smaller focal zones and deliver shock waves at faster rates, which is believed to decrease the efficiency of stone fragmentation.15 At the same time, substantial technical progress has been made with endoscopic technology, which has improved the capability of URS.17-19 Technical limitations of modern lithotripters have stimulated intense interest in new, more efficient designs.34,35 Our findings provide additional support for the importance of engineering research to improve the technical capabilities of SWL.
Our results have important implications for counseling patients regarding treatment options. Kidney stones affect approximately 9% of the United States population,1 and up to half of the patients who develop a kidney stone will experience recurrence within 5 years.4 Many of these patients will undergo procedures to fragment or remove stones. A substantial proportion of patients undergoing SWL will require a second procedure to fragment or remove stones. However, SWL may be preferred by some patients because it is a noninvasive procedure.29 Given this trade-off, physicians should formally assess patient-centered outcomes and preferences and ideally develop a tool to facilitate shared decision making for those patients who require procedural intervention.
These findings also have important policy implications regarding payment structures for health care delivery. Under the current fee-for-service structure, the intervention most likely to require additional treatment (SWL) is reimbursed at a higher rate than the technology most likely to remove a stone in a single session (URS).36 Even when surgeon fees are limited by a global payment period, substantial ancillary costs (ie, anesthesia and facility payments) still pertain. This misaligned economic structure would seem to exacerbate the costs of treating patients with kidney stones. Not only are these costs growing rapidly, but kidney stones are one of the most costly urologic conditions from the perspective of aggregate health care expenditures.9,10 Shifting toward a payment structure based on an episode of care would more closely align payer and provider incentives and potentially reduce the overall costs of treating patients with kidney stones.
Our results should be considered in the context of several limitations. The study population was young, generally healthy, and privately insured. Therefore, our results may not generalize to older populations (eg, those covered by Medicare) or others without private insurance. The claims-based nature of the analysis prevented direct examination of the influence of important clinical factors such as obesity and stone size and location. Similarly, we were unable to incorporate patient preferences for SWL or URS into our investigation. However, assuming that the instrumental variables function appropriately, our analytic approach should balance these and other unobserved confounders. Finally, the study findings pertain to the marginal patient, rather than the outcomes of treatment for the average patient. Clinical experience likely represents an average picture of patient outcomes. Therefore, even when expressed as the probability of repeat treatment, the findings may not be intuitive from a clinical perspective. Conceptually, patients undergoing URS or SWL for stones comprise 3 groups, namely, those who always undergo URS (eg, for a 6-mm distal ureteral stone), those who always undergo SWL (eg, for a 10-mm intermittently symptomatic renal stone treated by an SWL specialist), and those who might undergo either procedure. The average outcome (ie, clinical experience) likely represents the comparison of all patients undergoing URS vs all patients undergoing SWL. In contrast, the marginal patient represents the comparison of only those individuals who might undergo either procedure.37
These limitations notwithstanding, our findings retain significant validity and have important policy implications. Observed differences in the comparative effectiveness suggest that further exploration of additional outcomes (eg, unplanned postprocedural care, complication rates, and costs) will inform the comparison of outcomes for these procedures. Formal assessment of patient preferences, incorporating potential trade-offs in the nature and outcomes of these procedures, could reduce decisional conflict and improve patient satisfaction. Finally, efforts to improve the efficiency of modern lithotripters should continue to be supported.
Accepted for Publication: January 16, 2014.
Corresponding Author: Charles D. Scales Jr, MD, MSHS, Duke Clinical Research Institute, Division of Urologic Surgery, Duke University Medical Center, 40 Medicine Cir, White Zone, Ste 1570, Durham, NC 27710 (email@example.com).
Published Online: May 16, 2014. doi:10.1001/jamasurg.2014.336.
Author Contributions: Dr Scales had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: All authors.
Acquisition, analysis, or interpretation of data: All authors.
Drafting of the manuscript: Scales.
Critical revision of the manuscript for important intellectual content: Lai, Dick, Hanley, van Meijgaard, Setodji, Saigal.
Statistical analysis: Scales, Lai, Dick, Hanley, Setodji.
Obtained funding: Scales, Saigal.
Administrative, technical, or material support: Saigal.
Study supervision: Dick, van Meijgaard, Setodji, Saigal.
Conflict of Interest Disclosures: None reported.
Funding/Support: This study was supported by the Robert Wood Johnson Foundation Clinical Scholars Program and the US Department of Veterans Affairs (Dr Scales). The Urologic Diseases in America Project is supported by grant HHSN276201200016C from the National Institute of Diabetes and Digestive and Kidney Diseases and the National Library of Medicine.
Role of the Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation or review of the manuscript; or decision to submit for publication. The Robert Wood Johnson Foundation and US Department of Veterans Affairs had no role in the approval of the manuscript. The Executive Committee of the Urologic Diseases in America Project includes representatives from the National Institute of Diabetes and Digestive and Kidney Diseases; the Executive Committee approves project manuscripts based on technical considerations alone but otherwise has no role in the study.
Disclaimer: The views expressed in this article are those of the authors and do not necessarily reflect the position and policy of the US Federal Government or the Department of Veterans Affairs. No official endorsement should be inferred.
Group Information: The members of the Urologic Diseases in America Project are Mark S. Litwin, MD, MPH, Department of Urology, David Geffen School of Medicine, Fielding School of Public Health, University of California, Los Angeles, and RAND Corporation, Santa Monica, California; Christopher S. Saigal, MD, MPH, Department of Urology, David Geffen School of Medicine, University of California, Los Angeles, and RAND Corporation, Santa Monica, California; Paul Eggers, PhD, National Institute of Diabetes and Digestive and Kidney Diseases, Washington, DC; Tamara Bavendam, MD, National Institute of Diabetes and Digestive and Kidney Diseases, Washington DC; and Ziya Kirkali, MD, National Institute of Diabetes and Digestive and Kidney Diseases, Washington, DC.