Comparison of the Harms, Advantages, and Costs Associated With Alternative Guidelines for the Evaluation of Hematuria

Key Points

Question  What are the harms, advantages, and costs associated with alternative guidelines for examining patients with hematuria?

Findings  In a microsimulation modeling study of a hypothetical cohort of 100 000 adults with hematuria, uniform computed tomography scanning appeared to be associated with more than 500 secondary cancers from imaging-associated radiation exposure and was approximately twice the cost of alternative approaches.

Meaning  The balance of harms, advantages, and costs of hematuria evaluation may be optimized with risk stratification and more selective application of diagnostic testing in general and computed tomography imaging in particular.

Importance  Existing recommendations for the diagnostic testing of hematuria range from uniform evaluation of varying intensity to patient-level risk stratification. Concerns have been raised about not only the costs and advantages of computed tomography (CT) scans but also the potential harms of CT radiation exposure.

Objective  To compare the advantages, harms, and costs associated with 5 guidelines for hematuria evaluation.

Design, Setting, and Participants  A microsimulation model was developed to assess each of the following guidelines (listed in order of increasing intensity) for initial evaluation of hematuria: Dutch, Canadian Urological Association (CUA), Kaiser Permanente (KP), Hematuria Risk Index (HRI), and American Urological Association (AUA). Participants comprised a hypothetical cohort of patients (n = 100 000) with hematuria aged 35 years or older. This study was conducted from August 2017 through November 2018.

Exposures  Under the Dutch and CUA guidelines, patients received cystoscopy and ultrasonography if they were 50 years or older (Dutch) or 40 years or older (CUA). Under the KP and HRI guidelines, patients received different combinations of cystoscopy, ultrasonography, and CT urography or no evaluation on the basis of risk factors. Under the AUA guidelines, all patients 35 years or older received cystoscopy and CT urography.

Main Outcomes and Measures  Urinary tract cancer detection rates, radiation-induced secondary cancers (from CT radiation exposure), procedural complications, false-positive rates per 100 000 patients, and incremental cost per additional urinary tract cancer detected.

Results  The simulated cohort included 100 000 patients with hematuria, aged 35 years or older. A total of 3514 patients had urinary tract cancers (estimated prevalence, 3.5%; 95% CI, 3.0%-4.0%). The AUA guidelines missed detection for the fewest number of cancers (82 [2.3%]) compared with the detection rate of the HRI (116 [3.3%]) and KP (130 [3.7%]) guidelines. However, the simulation model projected 108 (95% CI, 34-201) radiation-induced cancers under the KP guidelines, 136 (95% CI, 62-229) under the HRI guidelines, and 575 (95% CI, 184-1069) under the AUA guidelines per 100 000 patients. The CUA and Dutch guidelines missed detection for a larger number of cancers (172 [4.9%] and 251 [7.1%]) but had 0 radiation-induced secondary cancers. The AUA guidelines cost approximately double the other 4 guidelines ($939/person vs $443/person for Dutch guidelines), with an incremental cost of $1 034 374 per urinary tract cancer detected compared with that of the HRI guidelines.

Conclusions and Relevance  In this simulation study, uniform CT imaging for patients with hematuria was associated with increased costs and harms of secondary cancers, procedural complications, and false positives, with only a marginal increase in cancer detection. Risk stratification may optimize the balance of advantages, harms, and costs of CT.

Long recognized as a transformative innovation,¹ advanced imaging has come under increased scrutiny. Payments for imaging have seen disproportionate growth,² and up to half of all imaging studies ordered in the United States may be unnecessary.³ Reflecting this tension and despite the intentionally broad scope of the initiative, most of the initial Choosing Wisely campaign’s recommendations involved medical imaging.⁴ Furthermore, the long-term harms of radiation exposure are gaining more recognition, given evidence of the association between doses in the range of common computed tomography (CT) scans and increased risk for developing cancer in the future.^5,6 Compounding the tripling volume of CT procedures in the United States during the past 2 decades to more than 85 million in 2011^2,7 are the inconsistency in radiation dose per examination and the facility-level variation in dose up to 13-fold.^8-10

Recommendations for the diagnostic evaluation of hematuria provide an instructive frame of reference within which to consider the advantages, harms, and costs associated with CT. Hematuria is prevalent,¹¹ and nearly 2 million Americans annually are referred to urologists for this finding.¹² Current guidelines emphasize a structured evaluation that includes cystoscopy and imaging to rule out urinary tract malignancy, although the threshold for referral and the recommended imaging modality remain uncertain given the limitations in the evidence.¹³ Given the prevalence, differing recommendations may result in considerable variation in population-level costs¹⁴ and patient burden from downstream sequelae.¹⁵

Previous cost-effectiveness analyses have suggested that substantial incremental costs and minimal advantages are associated with the substitution of CT for ultrasonography.^14,16 Although provocative, these studies have important limitations. First, asymptomatic microscopic hematuria, the focus of these analyses, has a relatively low pretest probability for cancer. Previous publications, including one that serves as a basis for these other studies’ estimates,¹⁷ have demonstrated that nearly 20% of referred patients may have a history of gross hematuria, which is associated with higher cancer risk by an order of magnitude. Second, one study modeled renal cell carcinoma (RCC) and upper-tract urothelial carcinoma (UTUC) as a composite outcome¹⁴ despite the differences in CT and ultrasonography for these diagnoses, and the other study assumed perfect sensitivity for CT.¹⁶ Third, although the authors of these studies have expressed concerns about the potential harms of radiation exposure from CT scans, as noted elsewhere,^18-20 real-world variation in dose has not been considered. Multiphase CT imaging of the abdomen and pelvis has been associated with not only the highest median dose but also the broadest range of dose and, in turn, the highest adjusted lifetime-attributable risk for cancer among common CT protocols.⁸

Against this backdrop, we used microsimulation modeling to compare the advantages, harms, and costs associated with different guidelines for hematuria evaluation. We included in the simulated population patients with both gross and microscopic hematuria, considering the detection of upper-tract cancers as discrete entities and accounting for the variation of CT dose in real-world practice.

We developed a patient-level microsimulation model using TreeAge Pro (TreeAge Software Inc) to compare 5 guidelines for the initial diagnostic evaluation of hematuria. We projected downstream outcomes and costs for a hypothetical cohort of 100 000 adults with hematuria. A detailed schematic of the model is presented in eFigure 1A to 1G in the Supplement. We used test-specific characteristics to compare cancer detection rates, false-positive results, radiation-induced secondary cancers, procedural complications, and costs per patient. This study was conducted from August 2017 through November 2018. Approval from an institutional review board was not needed because this study was a computer simulation based on published estimates in the literature.

We modeled data on patients’ age, sex, and presence and type of urinary tract cancer (bladder, RCC, or UTUC) from the 2 largest prospective hematuria cohort studies (1 from the United States,¹⁷ and 1 from the United Kingdom²¹) involving a total of 6363 patients (Table 1). We obtained distributions of other risk factors (smoking status and history of gross hematuria) from the US study (n = 4414).¹⁷ We constrained the simulation to exclude patients with a history of contrast allergy or renal impairment, as these conditions may be a factor in the selection of imaging modality.²² We validated model performance by comparing model-estimated cancer rates with those from a meta-analysis of 32 hematuria evaluation studies.²²

We simulated the clinical encounter and sequelae for the model cohort (n = 100 000) under each of 5 evaluation guidelines (eTable 1 in the Supplement). Under the Dutch and Canadian (Canadian Urological Association [CUA]) guidelines, eligible patients (50 years or older or 40 years or older, respectively) received cystoscopy and ultrasonography.^18,19 Under the guidelines of Kaiser Permanente (KP), a large integrated health care system in the United States, only patients with a history of gross hematuria received CT and cystoscopy; smokers, males, and anyone 50 years or older received cystoscopy and ultrasonography, whereas nonsmoking female patients younger than 50 years did not undergo any evaluation.²³ Under the Hematuria Risk Index (HRI) guidelines, we calculated HRI scores (eMethods in the Supplement) for each patient to determine their evaluation method (none for low-risk, cystoscopy and ultrasonography for moderate-risk, and cystoscopy and CT for high-risk patients).¹⁷ Under the American Urological Association (AUA) guidelines, all patients 35 years or older received cystoscopy and CT.²²

Diagnostic accuracy (sensitivity and specificity) of cystoscopy, ultrasonography, and CT for each cancer type (bladder, RCC, or UTUC) as well as rates of test-specific complications (urinary tract infection from cystoscopy, CT contrast-induced nephropathy, and contrast allergy) were obtained from published sources (Table 1).^17,21,24-35

We modeled the lifetime-attributable risks of cancer from CT radiation exposure as a function of radiation dose and age at exposure.^6,36 Each examination was modeled as a factor in radiation exposure of 6.4 to 90 millisieverts (mSv), reflecting the observed range of effective dose for the multiphase abdomen and pelvis CT imaging.⁸ We performed a sensitivity analysis around effective dose to characterize the plausible range using community-based data (eFigure 2 in the Supplement).³⁷ Biological outcomes of the one-time CT radiation exposure were modeled over the patients’ lifetime using a study of radiation carcinogenesis that found increased risks after exposure in middle or older age.³⁶ We used the reported age-specific estimates of excess lifetime risk of radiation-induced solid cancers and leukemia normalized for a dose of 0.1 gray (Gy) per 100 000 patients.^6,36 Next, we adjusted the cancer risks by the mean effective dose (in mSv) for CT after converting it to absorbed dose (in Gy) using a radiation weighing factor of 1 for x-rays.^36,38 The age-specific, dose-adjusted lifetime risks of radiation-induced cancer are reported in Table 1 and eTables 2 to 4 in the Supplement.

Costs were estimated from the US national payer (Medicare) perspective using 2017 US dollars and were obtained from the Centers for Medicare & Medicaid Services Hospital Outpatient Prospective Payment System for 2017, which reports the national mean outpatient payments based on the Ambulatory Payment Classification system (Table 1).³⁹ We used the Healthcare Common Procedure Coding System and Current Procedural Terminology codes to identify costs for each test. False-positive costs were estimated as costs of the initial diagnostic evaluation and did not include costs associated with downstream testing. Because we only included one-time costs associated with initial workup, we did not discount costs.

We ranked the strategies in the 5 guidelines from least to most expensive and compared them sequentially in terms of cost per urinary tract cancer cases detected. We calculated the incremental cost-effectiveness ratio (ICER) of each strategy as the additional cost divided by the additional number of cancer cases detected compared with the next less expensive alternative, removing any dominated strategies from the next sequential comparison.⁴⁰

We altered all input variables simultaneously in probabilistic sensitivity analysis with 10 000 iterations to assess the implication of uncertainty for the model using CIs, ranges, or SDs from the published literature (Table 1).^{6,8,17,21,24-36,39} When such values were unavailable, we evaluated uncertainty by setting plausible bounds to ±5% of base case values. We used β distribution for binomial data, including probabilities and test characteristics (sensitivity and specificity); γ distribution for costs; Dirichlet distribution for multinomial data; and triangular distribution for the effective radiation dose (Table 1), following best practices.⁴¹

The simulated cohort included 100 000 patients. A total of 3514 (3.5%) of these patients had urinary tract cancers (95% CI, 3.0%-4.0%), an estimated prevalence rate similar to the malignancy rate of 4.0% from a meta-analysis of 32 studies of initial hematuria evaluation.²² Of these cancers, 2978 (84.7%; 95% CI, 2490-3500) were bladder cancer, 443 (12.6%; 95% CI, 280-630) were RCC, and 93 (2.6%; 95% CI, 30-180) were UTUC.

Table 2 shows the expected health and economic outcomes associated with each guideline (sex- and age-specific results are in eTable 5 in the Supplement). The Dutch guidelines had the lowest cancer detection rate of 92.9% (n = 3263). Cancer detection rates increased in parallel with greater intensity of evaluation to 95.1% (n = 3343) for the CUA guidelines, 96.3% (n = 3385) for the KP guidelines, 96.7% (n = 3399) for the HRI guidelines, and 97.7% (n = 3432) for the AUA guidelines.

The CUA and Dutch guidelines had 0 radiation-associated cancers from CT because both recommend against CT in first-line evaluation. The model projected 108 (95% CI, 34-201) radiation-induced cancers under the KP guidelines, 136 (95% CI, 62-229) under the HRI guidelines, and 575 (95% CI, 184-1069) under the AUA guidelines per 100 000 patients. Sensitivity analyses that explored a broader range (3.5-144 mSv) of observed multiphase CT doses from the community³⁷ were associated with an estimate of 782 per 100 000 radiation-induced cancers for the uniform approach (eFigure 2 in the Supplement). The age breakdown of secondary solid cancers and leukemias is detailed in eTable 5 in the Supplement.

Additional harms associated with more intensive workup included more false-positive cases: 22 189 (95% CI, 17 520-27 370) for the AUA guidelines compared with 13 811 (95% CI, 10 800-17 170) for the HRI, 9099 (95% CI, 6270-12 450) for the KP, 6740 (95% CI, 4220-9820) for the CUA, and 6452 (95% CI, 4040-9410) for the Dutch guidelines. Total short-term procedural complications (CT allergy, contrast nephropathy, dysuria, and urinary tract infection) were also greatest for the AUA guidelines at 17 637 compared with 9709 for the HRI, 9582 for the KP, 8344 for the CUA, and 7999 for the Dutch guidelines.

The Dutch guidelines were associated with the lowest ($443) and the AUA guidelines with the highest ($939) per-person costs. The relative costs were approximately twice as high using the AUA guidelines compared with all other strategies. Results from the base case cost-effectiveness analysis are presented in Table 3 and Figure 1. Decreasing the eligible age for evaluation with cystoscopy plus ultrasonography from 50 years or older (Dutch) to 40 years or older (CUA) had an incremental cost of $23 864 per additional cancer detected. Compared with the CUA guidelines, the KP guidelines had an ICER of $137 063 per additional cancer detected. The HRI guidelines had an ICER of $559 378 per additional cancer detected compared with the KP ICER. The AUA guidelines had an ICER of $1 034 374 per additional cancer detected compared with HRI.

The results from the probabilistic sensitivity analysis using 10 000 Monte Carlo simulations are presented as cost-effectiveness acceptability curves (Figure 2), which show the probability that each set of guidelines would be considered cost-effective for various willingness-to-pay thresholds.⁴¹ Given that the health outcomes of interest are in natural units (cancers detected) rather than utility values that can be used to calculate quality-adjusted life-years, no commonly accepted cost-effectiveness threshold exists against which to compare each set of guidelines.^40,42 The probability of the AUA guidelines being cost-effective gradually increased with the willingness-to-pay threshold but remained lower than for other strategies across a wide range of willingness-to-pay thresholds up to $250 000 per additional cancer detected.

Hematuria is a common clinical finding for which the optimal approach is uncertain, particularly with regard to the role of CT imaging.¹³ A health technology assessment identified 79 hematuria diagnostic algorithms, none of which had been formally evaluated in terms of the algorithm’s association with patient outcomes.⁴³ In this study, we examined the trade-offs of advantages, harms, and costs across 5 representative guidelines for hematuria evaluation. The findings are strengthened by the use of a microsimulation model to capture the heterogeneity of the target population. Specifically, we made the assumption of a subgroup with gross hematuria, reflecting real-world practice,¹⁷ in contrast to previous studies that focused on the lower-risk subset of patients with asymptomatic microhematuria.^14,16 Although most clinicians view microscopic and gross hematuria differently and patients with microhematuria gain less advantage from testing, much of the evidence supporting current guidelines conflates these entities.⁴⁴ The approach we developed permits the assessment of the relative advantages, harms, and costs of different testing strategies within and across these patient subgroups (Table 2). Differences in bladder cancer diagnostic yield reflect a variety of recommended age thresholds for uniform evaluation by cystoscopy (Dutch, CUA, and AUA) and strategies omitting cystoscopy for lowest-risk patients, such as nonsmoking female patients younger than 50 years with microhematuria (KP and HRI, consistent with recommendations from other organizations⁴⁵).

With regard to imaging, we examined comparators in which all patients, even those with gross hematuria, received ultrasonography (CUA, 40 years or older; Dutch, 50 years or older). The remaining comparators illustrated trade-offs of advantages, harms, and costs associated with the omission of imaging for the lowest-risk patients (KP and HRI) and substitution of CT for ultrasonography for those with gross hematuria (KP) or multiple risk factors, including gross hematuria (HRI), and a comparator in which all patients, even those with microhematuria, received CT (AUA). Recommendations against uniform CT in this context explicitly cite concerns about costs and harms, in particular radiation exposure.^18-20 The findings support these concerns, as the strategy of CT for all patients was associated with not only substantially greater costs but also a larger number of false-positive results, procedural complications, and—considering the wide variation in CT dose in the real world—an estimated risk of radiation-induced future cancer for 1 in 174 patients tested.

These findings support the concepts that most patients are at below-average risk and may have less-than-average advantage from treatment⁴⁶ and that as intensity increases beyond an optimal level, the growth in advantages slows while harms and costs rise rapidly.⁴⁷ Given the imperfect sensitivity of diagnostic testing, no guidelines detected all cancers. The most intensive guidelines missed 82 (2.3%) of 3514 cases, with an estimated 575 additional future cancers associated with radiation from CT. Although our sources for variable estimation did not permit modeling of cancer stage and grade by site and risk group, and the advantage of early detection in this context was uncertain, most of the incremental additional cases detected with CT imaging were RCCs (Table 2). Although RCC was classically associated with the triad of flank pain, palpable mass, and gross hematuria,⁴⁸ it is increasingly an incidental diagnosis,⁴⁹ and concerns have been raised about overdiagnosis.^50,51 The rate of RCC in this simulated population (443 [0.44%] of 100 000), drawn from the 2 largest series of hematuria evaluation, is comparable to the rate of renal procedures (0.57%) and nephrectomy (0.44%) in a recently published study of US Medicare patients undergoing thoracoabdominal CT imaging for all indications.⁵²

This study has several limitations. Our aim was to compare the trade-offs between representative contemporary recommendations, and we assumed 100% fidelity to a given set of guidelines in each simulation. Given the complex reality of clinical practice, these simulations should be considered a stylized comparison of different approaches. For example, the AUA guidelines recommend CT for all adults 18 years or older with 3 or more red blood cells per high-powered field on a single microscopic urinalysis,²² whereas other strategies not only have higher age thresholds for evaluation but also suggest a referral after multiple urinalyses with positive results.^17-19,23,53 Our comparisons focused on the top-line recommendations of different guidelines, but future work might consider alternative diagnostic pathways, including magnetic resonance imaging, urinary cytological testing, or other optional adjuncts in this setting. Furthermore, although nearly 2 million US patients are referred to urologists annually for evaluation,¹² the total population with hematuria is much larger. Multiple observational studies suggest that most patients with hematuria, including those with high-risk features,⁵⁴ do not undergo timely evaluation with cystoscopy and imaging.^55-58 Improving the value of care in this context may require addressing potentially larger problems of misallocation of testing resources, which can have greater welfare costs than overuse.⁵⁹

In the United States, medically associated exposure to ionizing radiation has increased 6-fold in the past 3 decades,⁶⁰ mostly owing to the rapid increase in CT imaging.⁵ The general consensus is that radiation from medical imaging is carcinogenic, but the exact magnitude of iatrogenic cancer risk remains uncertain. The most accepted model is the no-threshold linear risk model established by the Biologic Effects of Ionizing Radiation report.⁶ This model states that each increase in effective dose of radiation is accompanied by a linearly proportional increase in the risk of later malignancy and that no threshold exists at which the risk of cancer rises dramatically. We estimated 575 of 100 000 radiation-induced cancers in the uniform CT approach. These estimates were higher than those obtained in related work (149 of 100 000),¹⁶ which did not account for the wide dose variation in real-world practice. The implication of dose variation is underscored in our sensitivity analyses that explored the plausible range of variation in the community³⁷ (eFigure 2 in the Supplement). A framework of stewardship strategies⁶¹ could mitigate these risks, including restricting the population for whom CT imaging is recommended as well as developing interventions⁶² supporting proficient and consistent execution of best practices to reduce dose variation.

We estimated a large number of false-positive cases on the basis of test specificity. Our estimates for false-positive costs reflected only the testing in the initial encounter and not the additional downstream testing or procedures, accounting in part for the lower cost estimates in the model compared with estimates in previous related work.¹⁴ False positives are rarely reported in the hematuria diagnostic testing literature,¹⁵ and there is a paucity of data on the practice patterns in that context. We did not include estimates for incidental findings, which occurred in more than 30% of this type of CT imaging.⁶³ Such findings may be associated with potentially unnecessary anxiety, patient and family burden,⁶⁴ physical risks, and costs¹⁵ associated with additional testing or procedures.¹³ As such, the cost and harm estimates here should be viewed as conservative. Treatment for secondary cancers from radiation exposure, which have a case fatality rate of 50% or more,⁶ also carries harms and costs not included in this model; therefore, the total burden of risks attributable to CT is likely to be substantially greater.

Well-intentioned efforts may lead to the widespread dissemination of clinical practices before their safety and effectiveness are clearly understood. This model-based comparison of 5 different guidelines for the diagnostic evaluation of hematuria suggests that, in addition to its substantial costs, the potential harms of the intensive application of uniform CT urography may outweigh the advantages of early diagnosis of urinary tract malignant neoplasms.

Accepted for Publication: May 7, 2019.

Corresponding Author: Matthew E. Nielsen, MD, MS, Department of Urology, University of North Carolina School of Medicine, 2107 Physicians Office Building, Campus Box 7235, Chapel Hill, NC 27599 (mnielsen@med.unc.edu).

Published Online: July 29, 2019. doi:10.1001/jamainternmed.2019.2280

Author Contributions: Drs Georgieva and Nielsen had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Georgieva, Wheeler, Erim, Smith-Bindman, Loo, Ng, Raynor, Nielsen.

Acquisition, analysis, or interpretation of data: Georgieva, Wheeler, Erim, Smith-Bindman, Garg, Raynor, Nielsen.

Drafting of the manuscript: Georgieva, Erim, Smith-Bindman, Nielsen.

Critical revision of the manuscript for important intellectual content: Wheeler, Erim, Smith-Bindman, Loo, Ng, Garg, Raynor, Nielsen.

Statistical analysis: Georgieva, Erim, Smith-Bindman, Nielsen.

Administrative, technical, or material support: Wheeler, Erim, Smith-Bindman, Loo, Ng, Raynor.

Supervision: Wheeler, Erim, Smith-Bindman, Nielsen.

Conflict of Interest Disclosures: Dr Wheeler reported receiving grants from Pfizer outside of the submitted work. Dr Raynor reported being a paid consultant for Intuitive Surgical and Terumo Medical outside of the submitted work. Dr Nielsen reported receiving grants from the National Institutes of Health, Patient-Centered Outcomes Research Institute, Agency for Healthcare Research and Quality, and American Cancer Society; being a paid consultant for the American College of Physicians High Value Care Task Force; and serving on the medical advisory board of Grand Rounds, all outside of the submitted work. No other disclosures were reported.

Funding/Support: Drs Georgieva and Nielsen were supported in part by the University Cancer Research Fund, a resource provided by the State of North Carolina via the University of North Carolina Lineberger Comprehensive Cancer Center.

Role of the Funder/Sponsor: The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.