The Know-Do Gap in Quality of Health Care for Childhood Diarrhea and Pneumonia in Rural India

Importance  In rural India, as in many developing countries, childhood mortality remains high and the quality of health care available is low. Improving care in such settings, where most health care practitioners do not have formal training, requires an assessment of the practitioners’ knowledge of appropriate care and the actual care delivered (the know-do gap).

Objective  To assess the knowledge of local health care practitioners and the quality of care provided by them for childhood diarrhea and pneumonia in rural Bihar, India.

Design, Setting, and Participants  We conducted an observational, cross-sectional study of the knowledge and practice of 340 health care practitioners concerning the diagnosis and treatment of childhood diarrhea and pneumonia in Bihar, India, from June 29 through September 8, 2012. We used data from vignette interviews and unannounced standardized patients (SPs).

Main Outcomes and Measures  For SPs and vignettes, practitioner performance was measured using the numbers of key diagnostic questions asked and examinations conducted. The know-do gap was calculated by comparing fractions of practitioners asking key diagnostic questions on each method. Multivariable regressions examined the relation among diagnostic performance, prescription of potentially harmful treatments, and the practitioners’ characteristics. We also examined correct treatment recommended by practitioners with both methods.

Results  Practitioners asked a mean of 2.9 diagnostic questions and suggested a mean of 0.3 examinations in the diarrhea vignette; mean numbers were 1.4 and 0.8, respectively, for the pneumonia vignette. Although oral rehydration salts, the correct treatment for diarrhea, are commonly available, only 3.5% of practitioners offered them in the diarrhea vignette. With SPs, no practitioner offered the correct treatment for diarrhea, and 13.0% of practitioners offered the correct treatment for pneumonia. Diarrhea treatment has a large know-do gap; practitioners asked diagnostic questions more frequently in vignettes than for SPs. Although only 20.9% of practitioners prescribed treatments that were potentially harmful in the diarrhea vignettes, 71.9% offered them to SPs (P < .001). Unqualified practitioners were more likely to prescribe potentially harmful treatments for diarrhea (adjusted odds ratio, 5.11 [95% CI, 1.24-21.13]). Higher knowledge scores were associated with better performance for treating diarrhea but not pneumonia.

Conclusions and Relevance  Practitioners performed poorly with vignettes and SPs, with large know-do gaps, especially for childhood diarrhea. Efforts to improve health care for major causes of childhood mortality should emphasize strategies that encourage pediatric health care practitioners to diagnose and manage these conditions correctly through better monitoring and incentives in addition to practitioner training initiatives.

Diarrhea and pneumonia remain leading drivers of mortality among children worldwide, causing 2 million deaths in 2011, including 24% of deaths among children aged 1 to 4 years.^1,2 Reducing these deaths requires investments in improved prevention, accurate diagnosis, and appropriate treatment.^3,4 Therefore, assessment of the capability of health care delivery systems is critical for the correct diagnosis and management of these conditions.¹ We herein examine the knowledge of rural pediatric health care practitioners, many of whom have no formal training, and assess the quality of care they deliver for childhood diarrhea and pneumonia in Bihar, India.

For many of Bihar’s 100 million inhabitants, only low-quality pediatric health care is accessible, contributing to the highest infant mortality rate in India (55 per 1000 live births),^5,6 preventable morbidity, and esclating antibiotic resistance.⁷ Documented deficiencies include low levels of knowledge and even lower levels of observed performance among these health care practitioners.^8-11 Low quality of care can occur even when health care practitioners have appropriate knowledge because of the know-do gap, whereby practitioners’ actions diverge from what they know they should do.^10,12,13 Understanding know-do gaps of health care practitioners is a critical step toward developing effective, practical strategies to improve delivery of necessary health care. For example, recent empirical evidence on pay-for-performance programs suggests the potential of incentive contracts to improve health outcomes, even in low-resource settings.^14-18 With further knowledge concerning know-do gaps, appropriate training and monitoring programs to improve the quality of health care delivery can be developed, and effective regulations and incentives can then support such efforts.¹⁹

This study aimed to disentangle the low levels of health care practitioner knowledge from poor effort and delivery of care. We present, to our knowledge, one of the first estimates of know-do gaps in the context of health care systems in developing countries that uses a standardized patient (SP) method for rigorously measuring the performance of health care practitioners.^13,20,21 We analyzed data from interviews with health care practitioners using vignettes to estimate competence in terms of knowledge as to what care these practitioners would provide for a hypothetical patient²² and compared the responses with data from SP-based assessments that accurately describe what practitioners did when they were presented with the same case. The know-do gap is the difference between the 2 measures.

The study protocol of the Bihar Evaluation of Social Franchising and Telemedicine (BEST) project was approved by Duke University (approval No. 29755) and India’s Health Ministry Steering Committee (No.12/2008/30-HMSC/4). Three hundred forty health care practitioners provided oral informed consent. We assessed the quality of health care provided for childhood diarrhea and pneumonia using data collected from June 29 through September 8, 2012, as part of the BEST project evaluation of a large-scale telemedicine program.²³ Among 360 clusters in the BEST project, 80 were selected at random for the study described herein. Each cluster consists of a group of geographically contiguous villages within 11 districts of Bihar. We generated a list of all health care practitioners visited in the past 6 months—regardless of medical qualifications—from interviews with 64 randomly selected households with children younger than 5 years. Although the study included the 5 most frequently visited primary health care practitioners in each cluster, our analytical sample is restricted to 340 practitioners for whom we had complete data from the vignettes and SPs. Using this restricted sample yields a consistent sample size for our analyses. All of our results remain consistent regardless of the sample used (justification and sample comparison are available in the eMethods and eTables 1 and 2 in the Supplement).

The study used 3 data sources collected before implementation of the telemedicine intervention in the BEST project. Surveys captured information on the characteristics of the health care practitioners, including age, educational level, medical training, experience, and types of illnesses treated. Vignettes and SPs were used to measure the knowledge of the practitioners and the quality of care. The health care practitioners consented to vignette interviews and visits by unannounced SPs within 2 months; 178 practitioners were randomized to receive SPs presenting with childhood diarrhea, and 162 were randomized to receive SPs presenting with childhood pneumonia.

The vignettes estimate the clinical knowledge of health care practitioners by presenting a hypothetical case in an interview setting administered by 2 interviewers.^9,22,24 In the diarrhea vignette, health care practitioners are told that a father seeks treatment for his 2-year-old son who has had loose stools for 2 days. While interviewer 1 records questions asked by the health care practitioners, interviewer 2 reads scripted responses aloud. For the pneumonia vignette, the treating practitioner is informed that the child has had a fever and a cough for 5 days and appears to have trouble breathing (vignettes are available at http://cohesiveindia.org/publications-downloads.html).

Although vignettes measure the knowledge of practitioners, they do not measure actual care delivered. Previous studies of know-do gaps have typically used methods whereby practitioners are observed by interviewers, but this method is vulnerable to Hawthorne effects.^13,25 In addition, limitations from case mix and self-selection of patients make comparisons across practitioners difficult.^9,12,25 The SP method is considered the criterion standard for practitioner performance measurement because it presents a well-defined incognito case in a clinically accurate and consistent manner to all practitioners.^13,20,26

Following the methods of Das et al,¹³ we used a proxy SP case in which a father seeks treatment for his ill 2-year-old child; the child is not present for the interaction. This pattern of health care, in which a family member seeks care on behalf of the sick patient, is common in India²⁷ and enables use of SP methods without putting a child at risk. The first case is a child with diarrhea (likely caused by rotavirus infection) but no clinical signs of dehydration for whom the only medical therapy indicated, aside from food-based fluids (eg, soup) or clean water, consists of oral rehydration salts (ORSs) to prevent dehydration.²⁸ The second proxy case is a child with pneumonia (as defined by a cough and rapid breathing) who requires antibiotic treatment; given that the father also reports signs of respiratory distress, this child meets the criteria for severe pneumonia and needs an urgent referral for hospitalization.²⁹ Immediately after the interaction, SPs were debriefed using exit interviews that recorded details of the interaction.

For diarrhea, correct treatment was defined to include ORSs with or without zinc supplements, with no prescription of unnecessary or potentially harmful drugs according to the 2005 World Health Organization guidelines.²⁸ The 2013 World Health Organization recommendations³⁰ include ORS and zinc supplementation. The correct treatment for severe pneumonia was defined to include appropriate antibiotics, absence of potentially harmful drugs, and referral to a hospital.²⁹ Because our sample consisted exclusively of practitioners in outpatient settings, we do not have data on hospitalization and referrals and instead focus on the drugs prescribed. For the SPs and vignettes, performance was measured using the numbers of key diagnostic questions asked and examinations conducted. We used item response theory (IRT) to compute performance scores separately for SPs and vignettes following previously described methods.^9,31 All analyses, including IRT scores, relied on a subset of 12 essential questions that help to diagnose the cause and severity of disease (eTable 3 in the Supplement). We refer to IRT-based performance for vignettes as the knowledge score and for SPs as the performance score.

We tested for differences in characteristics between practitioners with and without medical qualifications using unpaired 2-tailed t tests and χ² tests of proportions. We then compared practitioners’ knowledge for vignettes and performance with SPs to characterize know-do gaps as the fraction of practitioners who asked key diagnostic questions on each method.

We used regression analyses to examine associations among practitioners’ observable characteristics, performance, and the know-do gap. We estimated multivariable regressions in which the outcome is practitioner performance measured as the percentage of diagnostic questions asked for diarrhea and pneumonia. We checked the robustness of our findings using fractional logit models that account for the outcome having values ranging from 0 to 1. We conducted logistic regressions for the prescription of potentially harmful treatments (yes/no) for each case. All regressions control for age; medical qualification; practitioners’ work hours; patient volume; whether practitioners engage in public events, such as medical camps; whether the clinic is public or private; and clinic cleanliness as observed by the investigators. Analyses were adjusted for SEs for those practitioners who were sampled within clusters.

Of the 340 practitioners included in this study, 80.0% had no formal medical degrees in allopathy, Ayurveda, homeopathy, or Unani medicine, which is consistent with other studies of health care in rural India³² (Table 1). Qualified practitioners had a higher caseload, worked longer hours, and were more likely to work in public facilities. We found no significant differences in mean years of experience between qualified and unqualified practitioners. Qualified practitioners are more likely to spend time on skill-oriented activities, such as consultation and laboratory-related duties, and less time selling drugs. Approximately 90% of all practitioners reported frequent prescription of allopathic treatments. Almost all practitioners reported treating diarrhea (98.5% of those with qualifications and 96.7% of those without), whereas fewer practitioners without formal qualifications reported treating pneumonia (81.6% vs 92.6%). However, even practitioners who reported that they do not treat patients with a stated condition actually do provide treatment because they may not always provide the correct diagnosis for these conditions.

Practitioners demonstrated low levels of knowledge of key diagnostic questions and examinations (Table 2). The most commonly asked question for diarrhea concerned the nature of stools (60.6%); 46.2% asked about the frequency of stools. Only 32.9% asked questions that provide critical information about dehydration severity (eg, weakness, ability to take fluids, and urinary frequency). Most practitioners (86.8%) failed to ask about blood in stools to distinguish the simple case of viral diarrhea from possible dysentery. Among the diagnostic questions and examinations for diarrhea listed in Table 2, practitioners asked a mean of 2.9 questions and suggested a mean of 0.3 examinations.

For pneumonia, 33.2% of practitioners asked questions about fever, but only 24.1% asked about rapid breathing, and 20.9% asked about visual signs of respiratory distress (drawing in of the chest and nasal flaring were present in this case). Although difficulty in breathing was voluntarily described in the presenting complaints, only 10.9% of practitioners said they would auscultate the child’s chest (Table 2). Practitioners asked a mean of 1.4 questions and performed a mean of 0.8 examinations for pneumonia.

Despite seeking little diagnostic information, almost all practitioners made a diagnosis and prescribed treatments. Among 99.1% of practitioners with a diagnosis for diarrhea, 74.5% were correct. This finding did not surprise us because the diagnosis in the local language is a term for loose stools. Although ORSs are a commonly available treatment, only 3.5% of practitioners offered the correct ORS treatment, whereas 20.9% prescribed unnecessary antibiotics, corticosteroids, and other potentially harmful drugs without ORSs or zinc. (The drug list is available at http://cohesiveindia.org/publications-downloads.html). Another 68.8% prescribed ORSs with other unnecessary treatments (Table 3). When asked at the end of the vignette if the drugs prescribed included antibiotics, 12.6% of practitioners reported not prescribing antibiotics when in fact they had.

For pneumonia, 59.0% of all practitioners made the correct diagnosis of pneumonia. Only 8.8% of practitioners prescribed the correct treatment. An additional 43.8% offered antibiotics with unnecessary, potentially harmful drugs, such as allergy medications and drugs for cardiac conditions. The relatively high share of practitioners prescribing antibiotics likely reflects the general overprescription of antibiotics irrespective of the patients’ conditions.^33,34 The most commonly prescribed incorrect treatments included corticosteroids or vitamin syrups without any antibiotics.

Practitioner effort, measured in their interactions with SPs, was low (Table 4). Practitioners spent a mean (SD) of 1.6 (1.7) minutes with the SP with diarrhea and 2.9 (3.8) minutes with the SP with pneumonia (eFigure 1 in the Supplement shows the distribution of time). Practitioners asked a mean of 2.7 essential questions for diarrhea and 2.8 for pneumonia, covering less than 30% of the questions needed to diagnose the cause and severity of the disease. Despite cursory consultations, practitioners prescribed a mean of 1.8 medicines for diarrhea and 2.2 medicines for pneumonia.

Practitioners asked the SPs even fewer questions about the severity of the disease than they did with the vignettes (Table 4). Although 45.5% of practitioners asked the SP with diarrhea about the quality of stools, only 29.8% asked about frequency, 2.2% about urination, and 2.8% about the child’s activity level. Similarly, only 27.2% asked the SP with pneumonia about rapid breathing, and 24.7% asked about signs of respiratory distress. Other than age, the most commonly asked question was about fever (61.1%), which was one of the presenting complaints described by the SP.

Practitioners participating in the SP method frequently offered treatment despite not seeing the child. Although 23.6% asked the SP to return with the child for diarrhea, 92.7% prescribed treatment (for the SP with pneumonia, 32.7% and 79.0%, respectively). The proportion of practitioners who prescribed the correct treatment to the SP was far lower for diarrhea (0) compared with pneumonia (13.0%).

We found a clear know-do gap for diarrhea: practitioners reported during the vignette that they would ask diagnostic questions far more often than they asked the SP (eFigure 2 in the Supplement). For example, although 46.2% of practitioners asked about the frequency of stools in the vignette, only 29.8% asked the SP. For pneumonia, practitioners’ generally poor performance was roughly comparable between the vignette and the SP.

Likewise, the know-do gap for treatments offered is larger for diarrhea than for pneumonia (Table 3). For the diarrhea vignette, 72.4% of practitioners reported that they would offer ORSs (often in combination with other drugs); only 17.4% actually offered this type of treatment to the SP. Another critical dimension of the large know-do gap for diarrhea is reflected in practitioners prescribing potentially harmful treatments to the SP and not in the vignette. Compared with 20.9% of practitioners who said they would prescribe only potentially harmful treatments without ORSs for the vignette, 71.9% offered such treatments to the SP (P < .001). For pneumonia, this gap is smaller. Although 11.8% of practitioners reported that they would not prescribe antibiotics (classified as the incorrect treatment), 14.8% did not prescribe antibiotics (P = .09) to the SP.

Practitioners asked the SP only 24.1% and 30.7% of the essential questions for diarrhea and pneumonia, respectively (Table 4). Table 5 shows the associations between practitioner characteristics and the percentage of diagnostic questions asked. Practitioners in public facilities asked the SPs significantly fewer questions (16 percentage points fewer for diarrhea [P = .006] and 25 percentage points fewer for pneumonia [P < .001]), controlling for measures of patient volume, type of practice, knowledge IRT scores, the practitioners’ age group, and the practitioners’ experience. Those practitioners with more working hours per week asked fewer diagnostic questions for the SPs with diarrhea (0.2 percentage points [P = .004]) and pneumonia (0.3 percentage points [P = .02]). Medical qualifications were associated with 8.4 percentage points fewer (P = .03) diagnostic questions for pneumonia. Practitioners with higher knowledge scores about diarrhea also asked more diagnostic questions (P = .005). Unlike the SPs, for the vignettes, we found no significant or large associations between other practitioner characteristics and the number of questions asked (eTable 4 in the Supplement). Results were highly similar when we used fractional logit regressions (eTable 5 in the Supplement).

Being an unqualified practitioner predicted a significantly higher likelihood of prescribing potentially harmful treatments (Table 5) for diarrhea compared with qualified practitioners (adjusted odds ratio, 5.11 [95% CI, 1.24-21.13]). For pneumonia, the adjusted odds ratio was 2.19 (95% CI, 0.88-5.45). Results were generally similar when performance IRT score was used as the outcome (eTable 6 in the Supplement).

In rural India, pediatric health care practitioners demonstrated low levels of knowledge during vignettes for childhood diarrhea and pneumonia. We also found a large know-do gap in the practitioners’ treatment of childhood diarrhea but not pneumonia. In these settings, medically qualified practitioners offered far fewer potentially harmful treatments compared with unqualified practitioners.

Although 72.4% of practitioners reported that they would prescribe ORS treatment in the diarrhea vignette, only 17.4% actually prescribed ORSs to the SP, with the remainder prescribing potentially harmful drugs instead. Understanding factors that lead to this large know-do gap is critical to reducing preventable deaths due to diarrhea. The know-do gap for diarrhea appears paradoxical because most practitioners in our sample are in private practice and thus should be able to benefit financially from better performance unless patients cannot differentiate practitioner quality (an asymmetric information problem).³⁵ The severity of this asymmetry may be seen in the lack of robust correlation between practitioner characteristics and performance with SPs; practitioner characteristics explain only 12% to 20% of the variation in performance.

The smaller know-do gap for pneumonia is striking. Among practitioners in our sample, 83.8% claimed to treat pneumonia (97.1% for diarrhea), suggesting that they might consider pneumonia to be more critical than diarrhea and might exert higher effort. However, because the technically correct treatment includes a referral for hospitalization, which we did not observe, the true rates of correct treatment are likely lower than the 13.0% that we report. Mean consultation time for pneumonia was almost twice that for diarrhea, yet was still short. Although the know-do gap for pneumonia was small, the level of practitioner knowledge during vignettes was very low. Our findings of low levels of practitioner knowledge and effort and that public practitioners had worse performance with SPs are consistent with previous research.^10,11,36,37

Although our SP methods represent substantial improvements over other methods for measuring the quality of health care practitioners deliver in developing countries, they have limitations. First, the study is restricted to cases for whom the interviewers did not face any risk. Second, we used proxy cases presented by a parent of a sick child, although this presentation is common in India. From previous studies, these limitations do not affect measurements of practitioner quality but do present challenges for implementing similar methods in other contexts. Furthermore, because the presentations of patients were standardized, we do not know how practitioners might perform given a different set of symptoms. Nonetheless, the cases we chose are simple, uncomplicated presentations of conditions seen commonly in primary care in rural areas, making the findings important on their own merit. Finally, the absence of data on referrals for hospitalization for pneumonia cases suggests that our estimated rate of correct treatment (13.0%) represents the upper bound.

Although medically qualified practitioners do less harm than their unqualified counterparts, most practitioners in rural areas are unqualified.^32,38,39 Although practitioners asked few questions and spent very little time with the patient, almost all practitioners, including the unqualified ones, prescribed treatments, none of which were correct for diarrhea and only 13.0% of which were correct for pneumonia.

Our findings highlight the need to better understand why pediatric practitioners in developing countries fail to correctly diagnose and manage the 2 leading causes of childhood mortality. The know-do gap we document supports the argument that more training focused on increasing knowledge alone is insufficient. Understanding the incentives faced by practitioners as well as the potential role of patients who do not have adequate information about practitioner performance and quality are critical. How such information can be appropriately communicated and targeted to patients in developing countries to counteract the know-do gap remains a major challenge.

Accepted for Publication: November 17, 2014.

Corresponding Author: Manoj Mohanan, PhD, Sanford School of Public Policy, Duke University, 302 Towerview Dr, 128 Rubenstein Hall, Durham, NC 27708 (manoj.mohanan@duke.edu).

Published Online: February 16, 2015. doi:10.1001/jamapediatrics.2014.3445.

Author Contributions: Dr Mohanan and Ms Giardili had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Mohanan, Vera-Hernández, Das, Goldhaber-Fiebert, Rabin, Schwartz.

Acquisition, analysis, or interpretation of data: Mohanan, Vera-Hernández, Giardili, Goldhaber-Fiebert, Rabin, Raj, Schwartz, Seth.

Drafting of the manuscript: Mohanan, Vera-Hernández, Goldhaber-Fiebert, Schwartz, Seth.

Critical revision of the manuscript for important intellectual content: Mohanan, Vera-Hernández, Das, Giardili, Goldhaber-Fiebert, Rabin, Raj, Schwartz.

Statistical analysis: Mohanan, Vera-Hernández, Giardili, Goldhaber-Fiebert.

Obtained funding: Mohanan, Vera-Hernández, Goldhaber-Fiebert.

Administrative, technical, or material support: Mohanan, Raj, Seth.

Study supervision: Mohanan, Vera-Hernández, Das, Raj.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported by the Bill and Melinda Gates Foundation.

Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Information: Data set and STATA Do files can be downloaded from http://cohesiveindia.org/publications-downloads.html.

Additional Contributions: Purshottam, MA, Charu Nanda, MA, Rajan Singh, MA, Simi Chaturvedi, MA, Geeta, MA, Devender, MCom, Zargham Sayed, MA, Krishan Yadav, BA, and Varun Kumar, BA, of the Institute of Socioeconomic Research for Development and Democracy (ISERDD) team and the standardized patients implemented the field work for this study. We thank interviewers from Sambodhi Research and Communications for conducting the practitioner surveys and vignette interviews. Joanna Maselko, ScD, Departments of Psychiatry and Behavioral Sciences and Global Health, Duke University, provided comments on the manuscript. Contributors from the ISERDD received compensation for their work.