Accuracy of Practitioner Estimates of Probability of Diagnosis Before and After Testing

Key Points Question Do practitioners understand the probability of common clinical diagnoses? Findings In this survey study of 553 practitioners performing primary care, respondents overestimated the probability of diagnosis before and after testing. This posttest overestimation was associated with consistent overestimates of pretest probability and overestimates of disease after specific diagnostic test results. Meaning These findings suggest that many practitioners are unaccustomed to using probability in diagnosis and clinical practice. Widespread overestimates of the probability of disease likely contribute to overdiagnosis and overuse.

1. Data was first sought from high-quality recent systematic reviews, meta-analyses, and/or guidelines.
2. If only older systematic reviews, meta-analyses, and/or guidelines were available with newer high-impact studies after publication, we considered data from both (attempting to understand most accurate numbers for current technology/practice) 3. If no systematic reviews, meta-analyses, and/or guidelines were available, we used data from commonly cited studies based on citations in recent guidelines and creating weighted averages by consensus. The expert panel of physicians overseeing the study was presented with best evidence identified and settled on evidence-based answers presented in results.

a. How likely is it that Ms. Smith has pneumonia based on this information? (pretest probability)
35-yo woman, fatigue, cough, SOB, fevers 102°F/38.9°C, tachycardia There are no systematic reviews/meta-analyses for pretest probability. The closest thing we could identify was Metlay et al. 1 Prevalence-starting pretest, 5% of all patients visiting primary care physicians for cough diagnosed as having pneumonia.Heckerling 2 references the National Health Survey; 3% of people with respiratory infections have PNA.

Prediction Rules
All prediction rules are developed compared to their ability to identify infiltrates on CXR.
Pretest probability is best determined by pneumonia prediction algorithms. There are a few: • Heckerling criteria 2 -based on the absence of asthma, temp >100°F, HR >100, decreased BS (this patient was missing one variable of crackles) she has a 25% chance of pneumonia. o Article complicated but has a nomogram. Assuming 5% prevalence of PNA in primary care, 4 RF = 25% probability of pneumonia.
• Diehr criteria 3 -sputum, temp >100 (3 points) Score of +3 = +LR 14-with same 5% prevalence = 42% We looked for more recent publications and the only one identified was Tse et al. 4 This isn't as well developed and is confusing to interpret. Making some assumptions about duration of fever >3 days, it would predict ~33% pretest probability of pneumonia, so wouldn't change the conclusion from the Heckerling & Diehr criteria.

In summary, pretest probability 25-42%
There are no systematic reviews/meta-analyses for sensitivity & specificity of a chest x-ray for a clinical diagnosis of pneumonia. The best systematic review was Ye et al. 5 , which was primarily focused on using lung ultrasound for the diagnosis of CAP. It provides sensitivity & specificity, but this is vs. hospital ICD code, which is likely very non-specific for pneumonia.
The most informative paper for determining sensitivity and specificity was Claessens et al. 6 This was a prospective cohort study performed in the ER.
• 319 patients with suspected CAP prospectively enrolled from ER.
• Classified as definite, probable/possible or excluded clinically for CAP  had immediate CXR then CT. Report rates of changing diagnoses. A similar article that was reviewed but ultimately not referred to is: Self et al 7 This was a convenience sample of patients evaluated for PE or other non PNA reason who had a CXR and CT scan, we don't think this is as helpful as Claessens and haven't included the data.
We should note that the threshold for action for treatment of suspected pneumonia is often a relatively low pretest probability (or positive predictive value). You are seeing Mrs. Jones, a 43-year-old premenopausal woman with atypical chest pain and a normal ECG. She has no risk factors and normal vital signs/examination. She has no particular preference for testing and wants your advice.

a. How likely is Mrs. Jones' to have cardiac ischemia based on this information? (Pretest Probability)
We identified a systematic review in JAMA Rational Clinical Examination series by Fanaroff et al in 2015. 12 This review discussed clinical prediction rules. This was focused on the ER where the overall prevalence of acute coronary syndrome was estimated to be 13% (relatively high, and not the setting for this patient). However, we believe this is worth discussing. This patient would be categorized as LOW risk by all tools discussed, although some rules require obtaining a troponin test to categorize patients. The risk of a patient like this, considering 13% prevalence of ACS in ER would be 2.9% to 4.4%. Using prevalence of acute coronary syndrome in primary care with these prediction rules would lead to estimates of 1-2.7% pretest probability. 13 For a general discussion of calculators, see DiCarli et al. 14 Some cohorts are discussed for pretest probability by UpToDate but they only include those who had angiography and therefore have higher average pretest probability.
The best performing model seems to be the CAD consortium score 15 . This is the European CAD consortium score that was shown to be better than Diamond & Forrester. 16,17 For this model, apparently the best, the basic model gives a 3% risk whereas the more nuanced clinical model provides a 1% pretest probability.
Other scores reviewed: PreTest Consult score 18 was used by Victor Montori in a randomized trial; 19 it is simple and attributes 0.6% risk for this patient.