[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.161.241.199. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure 1.
Flow of Participants Through the Randomized Clinical Trial
Flow of Participants Through the Randomized Clinical Trial
Figure 2.
Workflow of the Child Health Improvement Through Computer Automation Developmental Screening and Surveillance Module
Workflow of the Child Health Improvement Through Computer Automation Developmental Screening and Surveillance Module
Figure 3.
Record of Parent Assessment of Concerns About Their Child’s Development
Record of Parent Assessment of Concerns About Their Child’s Development
Table 1.  
Summary of Patient Characteristics for the Developmental Screening Groupa
Summary of Patient Characteristics for the Developmental Screening Groupa
Table 2.  
Summary of Patient Characteristics for the Developmental Surveillance Groupa
Summary of Patient Characteristics for the Developmental Surveillance Groupa
1.
Boyle  CA, Boulet  S, Schieve  LA,  et al.  Trends in the prevalence of developmental disabilities in US children, 1997-2008. Pediatrics. 2011;127(6):1034-1042.
PubMedArticle
2.
King  TM, Glascoe  FP.  Developmental surveillance of infants and young children in pediatric primary care. Curr Opin Pediatr. 2003;15(6):624-629.
PubMedArticle
3.
Glascoe  FP.  Screening for developmental and behavioral problems. Ment Retard Dev Disabil Res Rev. 2005;11(3):173-179.
PubMedArticle
4.
Rydz  D, Shevell  MI, Majnemer  A, Oskoui  M.  Developmental screening. J Child Neurol. 2005;20(1):4-21.
PubMedArticle
5.
Council on Children With Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee.  Identifying infants and young children with developmental disorders in the medical home. Pediatrics. 2006;118(1):405-420.
PubMedArticle
6.
 Developmental surveillance and screening of infants and young children. Pediatrics. 2001;108(1):192-196.
PubMedArticle
7.
Earls  MF, Hay  SS.  Setting the stage for success: implementation of developmental and behavioral screening and surveillance in primary care practice. Pediatrics. 2006;118(1):e183-e188.
PubMedArticle
8.
Bethell  C, Reuland  C, Schor  E, Abrahms  M, Halfon  N.  Rates of parent-centered developmental screening. Pediatrics. 2011;128(1):146-155.
PubMedArticle
9.
Palfrey  JS, Singer  JD, Walker  DK, Butler  JA.  Early identification of children’s special needs. J Pediatr. 1987;111(5):651-659.
PubMedArticle
10.
Sices  L, Feudtner  C, McLaughlin  J, Drotar  D, Williams  M.  How do primary care physicians identify young children with developmental delays? a national survey. J Dev Behav Pediatr. 2003;24(6):409-417.
PubMedArticle
11.
Sand  N, Silverstein  M, Glascoe  FP, Gupta  VB, Tonniges  TP, O’Connor  KG.  Pediatricians’ reported practices regarding developmental screening. Pediatrics. 2005;116(1):174-179.
PubMedArticle
12.
Pinto-Martin  JA, Dunkle  M, Earls  M, Fliedner  D, Landes  C.  Developmental stages of developmental screening. Am J Public Health. 2005;95(11):1928-1932.
PubMedArticle
13.
American Academy of Pediatrics. Developmental Surveillance and Screening Policy Implementation Project (D-PIP). http://www.medicalhomeinfo.org/how/clinical_care/developmental_screening/d-pip/. Accessed May 13, 2008.
14.
Trivedi  MH, Kern  JK, Marcee  A,  et al.  Development and implementation of computerized clinical guidelines. Methods Inf Med. 2002;41(5):435-442.
PubMed
15.
Carroll  AE, Biondich  P, Anand  V, Dugan  TM, Downs  SM.  A randomized controlled trial of screening for maternal depression with a clinical decision support system. J Am Med Inform Assoc. 2013;20(2):311-316.
PubMedArticle
16.
Biondich  PG, Downs  SM, Anand  V, Carroll  AE.  Automating the recognition and prioritization of needed preventive services: early results from the CHICA system. AMIA Annu Symp Proc. 2005:51-55.
PubMed
17.
Anand  V, Biondich  PG, Liu  G, Rosenman  M, Downs  SM.  Child Health Improvement through Computer Automation. Stud Health Technol Inform. 2004;107(pt 1):187-191.
PubMed
18.
Jenders  RA, Hripcsak  G, Sideli  RV,  et al.  Medical decision support: experience with implementing the Arden Syntax at the Columbia-Presbyterian Medical Center. Proc Annu Symp Comput Appl Med Care. 1995:169-173.
PubMed
19.
Hripcsak  G, Ludemann  P, Pryor  TA, Wigertz  OB, Clayton  PD.  Rationale for the Arden Syntax. Comput Biomed Res. 1994;27(4):291-324.
PubMedArticle
20.
Downs  SM, Uner  H.  Expected value prioritization of prompts and reminders. Proc AMIA Symp. 2002:215-219.
PubMed
21.
Biondich  PG, Overhage  JM, Dexter  PR, Downs  SM, Lemmon  L, McDonald  CJ.  A modern optical character recognition system in a real world clinical setting. Proc AMIA Symp. 2002:56-60.
PubMed
22.
Biondich  PG, Downs  SM, Carroll  AE,  et al.  Shortcomings in infant iron deficiency screening methods. Pediatrics. 2006;117(2):290-294.
PubMedArticle
23.
Biondich  PG, Anand  V, Downs  SM, McDonald  CJ.  Using adaptive turnaround documents to electronically acquire structured data in clinical settings. AMIA Annu Symp Proc. 2003:86-90.
PubMed
24.
Carroll  AE, Biondich  PG, Anand  V,  et al.  Targeted screening for pediatric conditions with the CHICA system. J Am Med Inform Assoc. 2011;18(4):485-490.
PubMedArticle
25.
Downs  SM, Anand  V, Dugan  TM, Carroll  AE.  You can lead a horse to water: physicians' responses to clinical reminders . AMIA Annu Symp Proc. 2010;2010:167-171.
PubMed
26.
Downs  SM, Biondich  PG, Anand  V, Zore  M, Carroll  AE.  Using Arden Syntax and adaptive turnaround documents to evaluate clinical guidelines. AMIA Annu Symp Proc. 2006:214-218.
PubMed
27.
Downs  SM, Carroll  AE, Anand  V, Biondich  PG.  Human and system errors, using adaptive turnaround documents to capture data in a busy practice. AMIA Annu Symp Proc. 2005:211-215.
PubMed
28.
Anand  V, Carroll  AE, Downs  SM.  Automated primary care screening in pediatric waiting rooms. Pediatrics. 2012;129(5):e1275-e1281.
PubMedArticle
29.
Squires  J, Twombly  E, Bricker  D, Potter  L. The ASQ-3 User's Guide. Baltimore, MD: Paul H. Brookes Publishing; 2009.
30.
Hix-Small  H, Marks  K, Squires  J, Nickel  R.  Impact of implementing developmental screening at 12 and 24 months in a pediatric practice. Pediatrics. 2007;120(2):381-389.
PubMedArticle
31.
Cabana  MD, Rand  CS, Powe  NR,  et al.  Why don’t physicians follow clinical practice guidelines? a framework for improvement. JAMA. 1999;282(15):1458-1465.
PubMedArticle
32.
Mangione-Smith  R, DeCristofaro  AH, Setodji  CM,  et al.  The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357(15):1515-1523.
PubMedArticle
33.
Carroll  AE, Bauer  NS, Dugan  TM, Anand  V, Saha  C, Downs  SM.  Use of a computerized decision aid for ADHD diagnosis: a randomized controlled trial. Pediatrics. 2013;132(3):e623-e629. doi: 10.1542/peds.2013-0933.
PubMedArticle
34.
Bauer  NS, Carroll  AE, Downs  SM.  Understanding the acceptability of a computer decision support system in pediatric primary care . J Am Med Inform Assoc. 2013;21(1):146-153.
PubMedArticle
35.
Bauer  NS, Strum  LA, Carroll  AE, Downs  SM.  Computer decision support to improve autism screening and care in community pediatric clinics. Infants Young Child. 2013;26(4):306-317.Article
Original Investigation
September 2014

Use of a Computerized Decision Aid for Developmental Surveillance and ScreeningA Randomized Clinical Trial

Author Affiliations
  • 1Children’s Health Services Research, Indiana University School of Medicine, Indianapolis
  • 2The Regenstrief Institute for Health Care, Indianapolis, Indiana
  • 3Department of Biostatistics, Indiana University School of Medicine and Richard M. Fairbanks School of Public Health, Indianapolis
JAMA Pediatr. 2014;168(9):815-821. doi:10.1001/jamapediatrics.2014.464
Abstract

Importance  Developmental delays and disabilities are common in children. Research has indicated that intervention during the early years of a child's life has a positive effect on cognitive development, social skills and behavior, and subsequent school performance.

Objective  To determine whether a computerized clinical decision support system is an effective approach to improve standardized developmental surveillance and screening (DSS) within primary care practices.

Design, Setting, and Participants  In this cluster randomized clinical trial performed in 4 pediatric clinics from June 1, 2010, through December 31, 2012, children younger than 66 months seen for primary care were studied.

Interventions  We compared surveillance and screening practices after adding a DSS module to an existing computer decision support system.

Main Outcomes and Measures  The rates at which children were screened for developmental delay.

Results  Medical records were reviewed for 360 children (180 each in the intervention and control groups) to compare rates of developmental screening at the 9-, 18-, or 30-month well-child care visits. The DSS module led to a significant increase in the percentage of patients screened with a standardized screening tool (85.0% vs 24.4%, P < .001). An additional 120 records (60 each in the intervention and control groups) were reviewed to examine surveillance rates at visits outside the screening windows. The DSS module led to a significant increase in the percentage of patients whose parents were assessed for concerns about their child's development (71.7% vs 41.7%, P = .04).

Conclusions and Relevance  Using a computerized clinical decision support system to automate the screening of children for developmental delay significantly increased the numbers of children screened at 9, 18, and 30 months of age. It also significantly improved surveillance at other visits. Moreover, it increased the number of children who ultimately were diagnosed as having developmental delay and who were referred for timely services at an earlier age.

Trial Registration  clinicaltrials.gov Identifier: NCT01351077

Introduction

Developmental delays and disabilities are common in children. Research has indicated that intervention during the early years of a child’s life has a positive effect on cognitive development, social skills and behavior, and subsequent school performance.13 To ensure early identification of developmental delays and disabilities, physician organizations, such as the American Academy of Pediatrics (AAP), have called on pediatricians to institute a standardized approach for the identification of developmental delays that includes developmental surveillance and screening (DSS).46

Developmental screening refers to the administration of a standardized tool to aid in the identification of children at risk for a developmental disorder.5 Developmental surveillance, on the other hand, is “an ongoing process of monitoring the status of a child by gathering information about the child’s development and behavior from multiple sources, including skillful direct observation of the child’s behavior and elicitation of concerns from parents and relevant professionals.”4(p 5)

Significant efforts have been made to address the implementation of DSS practices within primary care settings.7 Despite such efforts, these processes are still not routinely implemented, and studies5,811 have found that, when screening occurs, it often does not make use of standardized protocols or tools.

Numerous barriers exist to the successful implementation of DSS practices within the primary care setting, including lack of time and staff, logistical challenges in administering screening tools, inadequate reimbursement, and language barriers.1013 Computerized clinical decision support systems (CCDSS) are a promising strategy to overcome these barriers. Because CCDSS involve the use of integrated systems that routinely store and retrieve patient information, they can improve workflow by providing physicians with patient-specific recommendations based on integrated data at the time and place of a patient visit.14,15 Although previous attempts have been made to automate the process of developmental screening, these attempts have not proved successful because the decision support was not integrated with routine clinical care.14,15 The objective of this study was to determine whether CCDSS integrated with routine care could be an effective approach to improve standardized DSS within primary care practices.

Methods
Study Design

For the purposes of this study, we used an existing CCDSS designed for use in pediatric primary care practices: the Child Health Improvement Through Computer Automation (CHICA) system. However, we refined CHICA to include the DSS algorithm published by the AAP in 2006 by creating the CHICA DSS module.5

We conducted a randomized clinical trial in 4 pediatric clinics. Two of these clinics functioned as our control sites and used the traditional CHICA system that did not include the DSS module. The other 2 clinics functioned as our intervention sites and used the enhanced CHICA system that included the DSS module. We then compared surveillance, screening, and diagnosis of developmental disorders between the control and intervention clinics. This study was approved by the institutional review board of the Indiana University School of Medicine, and informed consent was waived.

Although our intervention was aimed primarily at physicians, the unit of randomization was the primary care clinic, and the unit of analysis was the individual patient. The 4 clinics were matched based on the number of physicians practicing at each of the clinics before randomization and then block randomized as pairs. We chose to randomize by clinic because contamination was a major concern.

Setting

This study was performed from June 1, 2010, through December 31, 2012, in 4 primary care pediatric clinics in the Eskenazi Medical Group, the largest safety-net health system in Indianapolis, Indiana. These 4 clinics conducted approximately 88 000 pediatric patient visits during this time frame, with 84% of the visits being supported by Medicaid.

Participants

Our intervention was aimed primarily at physicians. However, the outcomes of interest for our study are patient based. Therefore, patients younger than 66 months were automatically placed into the control or intervention group based on which of the 4 clinics they attended (Figure 1). There were no additional inclusion or exclusion criteria.

Intervention
The Traditional CHICA System

In 2004, CHICA was developed to support the provision of well-child care and the care of a number of common chronic conditions.16,17 It is unique in that, at each visit, the system uses a library of Arden Syntax18,19 rules that gather data from the Regenstrief Medical Record System and CHICA record systems to prioritize and then select 20 health questions that are printed on a paper questionnaire or an electronic tablet for that family to complete in the waiting room.20 The paper questionnaire, called the Patient Screening Form, is scannable, and the answers to the questions are stored in the electronic health record. After the information is stored, a tailored worksheet is generated for the physician to use during the visit. The worksheet includes up to 6 alerts, also prioritized, each with up to 6 check-box responses with which the physician can document his or her response. The comprehensiveness of what CHICA covers makes it useful at every clinic visit. Further details about the CHICA system have been described elsewhere.16,18,2028

The CHICA DSS Module

The CHICA system was operational at the 4 clinic sites before the initiation of this study. In June 2010, we added the DSS module to the CHICA system that was operating at the 2 intervention clinic sites. Before activating the DSS module in the intervention clinics, staff and physicians were informed that a new functionality was being added to the CHICA system, which is done any time new functionality is introduced to the system. Staff and physicians of the intervention and control clinics were not made aware that a clinical trial was under way.

The overall flow of the CHICA DSS module is shown in Figure 2 and mirrors the surveillance and screening algorithm for DSS recommended by the AAP in 2006.5 The CHICA DSS module had a number of components that we expected would improve the screening and diagnosis of developmental disorders that are not included in the traditional CHICA system:

  1. Universal Screening: The CHICA DSS module was designed to provide universal screening at 3 target well-child care visits (9-, 18-, and 30-month visits) through the use of the Ages and Stages Questionnaire 3 (ASQ-3) , which was automatically printed when the patient checked in for these target visits.29 The ASQ-3 is a brief parent report measure with 19 age-specific questions that span the age range of 4 to 60 months.5,30 The 4 clinic study sites were familiar with and had licenses to use the ASQ-3 in clinical care.

  2. Surveillance: On the patient screening form, parents were asked simple questions related to whether they had any concerns about their child’s development at every nontarget well-child care visit. If a parent responded affirmatively to any of the questions, the physician was notified on the physician worksheet, and a standardized screening tool was printed for use by the physician. The physician worksheet also provided a way for the physician to document the overall ratings of a child’s developmental status and specify developmental areas of concern.

  3. Reassessment: The CHICA DSS module automatically tracked those children whose parents had concerns or who had borderline results on a previous ASQ-3 screening. For these children, the CHICA DSS module would generate a new ASQ-3 and prompt the physician to rescreen at subsequent visits, consistent with AAP guidelines.

  4. Recommendations: On the basis of established guidelines,5 the CHICA DSS module prompted physicians to refer children with positive screening results for comprehensive evaluation and services.

Data Collection

Data collection began 6 months after the DSS module was activated at the intervention clinics. For the developmental screening portion of our study, 180 patient records were randomly pulled from the intervention clinics, and another 180 patient records were pulled from the control clinics, for a total of 360 patient records. These records were divided equally between the 9-, 18-, and 30-month well-child care visits. We made sure that patients selected for analysis at one visit were not included for the analyses at the other visits. An additional 120 patient records (60 from intervention clinics and 60 from control clinics) were randomly pulled from the 4 clinic sites for the developmental surveillance portion of the study. These records could be for any visit as long as it was not one of the target visits. Consequently, there was no overlap between the patient records reviewed for the developmental screening and developmental surveillance portions of the study.

Data were obtained from the electronic health record, the CHICA system, and manual paper record abstractions by trained research assistants. To assess the reliability of medical record abstraction, a random sample of 20% of the records were abstracted twice. The agreement on overall record abstraction was 89%, with a κ of 0.75.

Outcome Variables

For developmental screening, our primary outcome of interest was whether a standardized screening tool was administered at the target visits (ie, 9-, 18-, and 30-month visits). Other variables of interest for this portion of the study included the percentage of standardized screens with a positive result for a developmental delay at target visits, diagnosis of developmental delay or disorder at any point after target visit, and age at diagnosis.

For developmental surveillance, our primary outcome of interest was whether developmental concerns were elicited from parents at visits other than the target visits. Other data collected for this portion of the study included physician documentation of developmental concerns separate from parental concerns, diagnosis of developmental delay or disorder at any point after target visit, and age at diagnosis.

Sample Size Calculation

The primary outcome of interest was whether a child was screened for developmental delay at target visits. For sample size calculations, we used a rate of 25% in the control clinic. We wanted to have at least a 60% screening rate in the intervention clinics after implementation of the DSS module. Because patients were nested within clinics, we applied a conservative intracluster correlation estimate of 0.008, twice the expected intracluster correlation of 0.004. Although we did not anticipate much variability in the screening rate at the 4 study clinics, we used 4 different rates (12%, 15%, 18%, and 21%) to estimate intraclinic correlation. Using the χ2 test and setting α at .05, we found the probability of detecting a statistically significant difference in the proportion of children screened at the target visits (9-, 18-, and 6-month visits) between the intervention and the control group of 90%, with an effective sample size of 46 per group. However, the proposed sample size had more than 99% power to detect a 35% absolute difference in screening rates.

Statistical Analysis

Patient characteristics in the surveillance and screening components of the study were compared between the control and intervention groups using χ2, Fisher exact, and t tests. Logistic and exact logistic regression models were used to assess associations between dichotomous outcomes and intervention after controlling for sex, race, and type of insurance. Point estimates and 95% CIs on treatment effect were reported. A linear mixed-effects model was used to compare age at diagnosis after controlling for the same confounding factors as in the previous models. We assumed the parents’ assessment of the child’s development was independent of the clinic; therefore, the clinic was not considered a cluster in modeling this outcome. However, all other models included the clinic as a cluster to incorporate dependency among responses from the same clinic, and the clinic was used as a random effect in the models. The linear mixed-effects model used a logit link function to model the dichotomous outcomes. Finally, a linear mixed-effect model that included clinic as a cluster was used to model the age at diagnosis for developmental delay. SAS statistical software (SAS Institute Inc) was used for all analyses, and P < .05 was considered statistically significant.

Results

For the developmental screening portion of our study, the characteristics of the 360 study patients are given in Table 1. No significant differences were found between the intervention and control groups with respect to sex, type of insurance, and age. A significant difference was found between the 2 groups with respect to race based on differences in the clinic populations (P < .001). We controlled for race in all analyses.

The DSS module led to a significant increase in the percentage of patients screened with a standardized screening tool at the target visits (85.0% vs 24.4%, P < .001). The odds of being screened in the intervention group were 15.6 (95% CI, 6.9-35.7) times the odds for a child in the control group. If screening occurred, however, the rate of a positive screen result was similar between the groups (19.6% vs 18.2%, P = .57). This finding implies that the number of children at risk for developmental delay was similar between groups but that more children were picked up in the intervention group because of higher screening rates.

Although our study was not powered to detect differences in a full diagnosis of developmental delay, our results indicated that a diagnosis of developmental delay was not significantly more common in the intervention group (10.6% vs 6.7%, P = .52). However, the diagnosis of developmental delay was made earlier in the intervention group than in the control group (mean age at diagnosis, 17.2 vs 27.9 months; P < .001).

For the developmental surveillance portion of this study, characteristics of the 120 patients included are given in Table 2. Again, a significant difference was found in the distribution of race between the intervention and control groups (P < .001). We therefore controlled for race in all analyses.

The CHICA DSS module led to a significant increase in the percentage of parents who were asked about concerns regarding their children’s development outside the target visits (71.7% vs 41.7%, P = .04) (Figure 3). The odds of parents being asked about child development for children in the intervention group were 2.70 (95% CI, 1.05-6.84) times the odds for children in the control group. When concerns were assessed, more concern was noted in the control group than the intervention group, although the difference was not significant (9.3% vs 16.0%, P = .38). This finding suggests that parents were being assessed in the control group only when there was a higher likelihood of a positive concern. The intervention had no effect on whether physicians documented an assessment of developmental concerns (83.3% vs 81.7%, P = .52).

Once again, the study was not powered to detect differences in a full diagnosis of developmental delay. We found that a diagnosis of developmental delay occurred after positive surveillance in 20% of intervention children vs 8.3% of control children (P = .62). No significant difference was found between the 2 groups in terms of age at diagnosis.

Discussion

This study found that the CHICA DSS module more than tripled the use of standardized tools to conduct screening at target visits (ie, 9, 18, and 30 months of age) as recommended by the AAP guideline. Guidelines also recommend that surveillance occur at all other ages,5 and the DSS module notably increased the rates at which parents were asked about developmental concerns. The surveillance aspect led to an increase in the number of children diagnosed as having developmental delay. The CCDSS intervention led to earlier detection of developmental delay (mean of approximately 8 months earlier). Because optimal outcomes of developmental delay depend on early detection, this finding is a critically important finding, although our study was not designed to detect changes in clinical outcomes.

This study has limitations that warrant consideration. Previously existing practice differences could account for some differences, although we have no reason to believe that such differences existed. Use of the clinic as the basis of randomization could also lead to biases. However, the limitations and advantages of randomizing at the patient, physician, and clinic levels led us to decide that the clinic served as the best point of intervention. Generally, all informatics interventions have limitations as well. Some researchers believe they make physicians more reliant on external sources for quality care or leave them exposed to legal risk if they ignore prompts. Such things were outside the scope of this study, however. Generalizability of a specific CCDSS is always an issue. Although most electronic medical records are not as sophisticated as the CHICA system, many systems would allow programming with the algorithms we used in this study. In addition, the CHICA system has been created with open-source tools and is available for installation into other systems.

Adherence to existing clinical practice guidelines can be difficult for pediatricians who are often overwhelmed with patients and constrained by time.31 Because of this, it is possible for them to fail to properly and fully screen and document every condition.32 The CHICA DSS module is unique in that it permits us to insert guideline-based care subtly into existing clinic practices, and by prioritizing when certain modules are used, the system ensures that physicians and patients are not overwhelmed. A holistic CCDSS, such as the CHICA system, has much potential for introducing better evidence-based care and chronic care management into busy practices.28,3335

Our study found that a CCDSS, such as the CHICA system, that performs surveillance and screens patients for the presence of developmental delay with a standardized screening tool, coupled with personalized, evidence-based prompts to physicians, significantly improved the rates of proper screening and time to diagnosis of children with developmental delay. It also offered hope that management and referral of such patients could be improved. Future research should focus on determining whether these improvements are clinically significant and can be replicated elsewhere and whether these results can be applied to other chronic conditions.

Conclusions

Use of a CCDSS to automate the screening of children for developmental delay significantly increased the numbers of children screened at 9, 18, and 30 months of age. It also significantly improved consistent surveillance at other ages. Moreover, it increased the number of children who ultimately were diagnosed as having developmental delay and referred for timely services at an earlier age. More work is needed to determine whether this system translates into improved outcomes for children.

Back to top
Article Information

Accepted for Publication: March 5, 2014.

Corresponding Author: Aaron E. Carroll, MD, MS, Children’s Health Services Research, Indiana University School of Medicine, 410 W 10th St, Health Information and Translational Sciences, Ste 4099c, Indianapolis, IN 46202 (aaecarro@iupui.edu).

Author Contributions: Drs Carroll and Bauer had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Carroll, Anand, Downs.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Carroll, Saha, Downs.

Critical revision of the manuscript for important intellectual content: Bauer, Dugan, Anand, Saha, Downs.

Statistical analysis: Carroll, Saha, Downs.

Obtained funding: Carroll, Saha, Downs.

Administrative, technical, or material support: Carroll, Dugan, Anand, Downs.

Study supervision: Carroll, Anand, Saha, Downs.

Published Online: July 14, 2014. doi:10.1001/jamapediatrics.2014.464.

Conflict of Interest Disclosures: None reported.

Funding/Support: This project was supported by grant R01HS017939 from the Agency for Healthcare Research and Quality.

Role of the Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Additional Contributions: The Child Health Informatics Research and Development Lab and the Pediatric Research Network group at the Indiana University School of Medicine collected data for this study.

References
1.
Boyle  CA, Boulet  S, Schieve  LA,  et al.  Trends in the prevalence of developmental disabilities in US children, 1997-2008. Pediatrics. 2011;127(6):1034-1042.
PubMedArticle
2.
King  TM, Glascoe  FP.  Developmental surveillance of infants and young children in pediatric primary care. Curr Opin Pediatr. 2003;15(6):624-629.
PubMedArticle
3.
Glascoe  FP.  Screening for developmental and behavioral problems. Ment Retard Dev Disabil Res Rev. 2005;11(3):173-179.
PubMedArticle
4.
Rydz  D, Shevell  MI, Majnemer  A, Oskoui  M.  Developmental screening. J Child Neurol. 2005;20(1):4-21.
PubMedArticle
5.
Council on Children With Disabilities; Section on Developmental Behavioral Pediatrics; Bright Futures Steering Committee; Medical Home Initiatives for Children With Special Needs Project Advisory Committee.  Identifying infants and young children with developmental disorders in the medical home. Pediatrics. 2006;118(1):405-420.
PubMedArticle
6.
 Developmental surveillance and screening of infants and young children. Pediatrics. 2001;108(1):192-196.
PubMedArticle
7.
Earls  MF, Hay  SS.  Setting the stage for success: implementation of developmental and behavioral screening and surveillance in primary care practice. Pediatrics. 2006;118(1):e183-e188.
PubMedArticle
8.
Bethell  C, Reuland  C, Schor  E, Abrahms  M, Halfon  N.  Rates of parent-centered developmental screening. Pediatrics. 2011;128(1):146-155.
PubMedArticle
9.
Palfrey  JS, Singer  JD, Walker  DK, Butler  JA.  Early identification of children’s special needs. J Pediatr. 1987;111(5):651-659.
PubMedArticle
10.
Sices  L, Feudtner  C, McLaughlin  J, Drotar  D, Williams  M.  How do primary care physicians identify young children with developmental delays? a national survey. J Dev Behav Pediatr. 2003;24(6):409-417.
PubMedArticle
11.
Sand  N, Silverstein  M, Glascoe  FP, Gupta  VB, Tonniges  TP, O’Connor  KG.  Pediatricians’ reported practices regarding developmental screening. Pediatrics. 2005;116(1):174-179.
PubMedArticle
12.
Pinto-Martin  JA, Dunkle  M, Earls  M, Fliedner  D, Landes  C.  Developmental stages of developmental screening. Am J Public Health. 2005;95(11):1928-1932.
PubMedArticle
13.
American Academy of Pediatrics. Developmental Surveillance and Screening Policy Implementation Project (D-PIP). http://www.medicalhomeinfo.org/how/clinical_care/developmental_screening/d-pip/. Accessed May 13, 2008.
14.
Trivedi  MH, Kern  JK, Marcee  A,  et al.  Development and implementation of computerized clinical guidelines. Methods Inf Med. 2002;41(5):435-442.
PubMed
15.
Carroll  AE, Biondich  P, Anand  V, Dugan  TM, Downs  SM.  A randomized controlled trial of screening for maternal depression with a clinical decision support system. J Am Med Inform Assoc. 2013;20(2):311-316.
PubMedArticle
16.
Biondich  PG, Downs  SM, Anand  V, Carroll  AE.  Automating the recognition and prioritization of needed preventive services: early results from the CHICA system. AMIA Annu Symp Proc. 2005:51-55.
PubMed
17.
Anand  V, Biondich  PG, Liu  G, Rosenman  M, Downs  SM.  Child Health Improvement through Computer Automation. Stud Health Technol Inform. 2004;107(pt 1):187-191.
PubMed
18.
Jenders  RA, Hripcsak  G, Sideli  RV,  et al.  Medical decision support: experience with implementing the Arden Syntax at the Columbia-Presbyterian Medical Center. Proc Annu Symp Comput Appl Med Care. 1995:169-173.
PubMed
19.
Hripcsak  G, Ludemann  P, Pryor  TA, Wigertz  OB, Clayton  PD.  Rationale for the Arden Syntax. Comput Biomed Res. 1994;27(4):291-324.
PubMedArticle
20.
Downs  SM, Uner  H.  Expected value prioritization of prompts and reminders. Proc AMIA Symp. 2002:215-219.
PubMed
21.
Biondich  PG, Overhage  JM, Dexter  PR, Downs  SM, Lemmon  L, McDonald  CJ.  A modern optical character recognition system in a real world clinical setting. Proc AMIA Symp. 2002:56-60.
PubMed
22.
Biondich  PG, Downs  SM, Carroll  AE,  et al.  Shortcomings in infant iron deficiency screening methods. Pediatrics. 2006;117(2):290-294.
PubMedArticle
23.
Biondich  PG, Anand  V, Downs  SM, McDonald  CJ.  Using adaptive turnaround documents to electronically acquire structured data in clinical settings. AMIA Annu Symp Proc. 2003:86-90.
PubMed
24.
Carroll  AE, Biondich  PG, Anand  V,  et al.  Targeted screening for pediatric conditions with the CHICA system. J Am Med Inform Assoc. 2011;18(4):485-490.
PubMedArticle
25.
Downs  SM, Anand  V, Dugan  TM, Carroll  AE.  You can lead a horse to water: physicians' responses to clinical reminders . AMIA Annu Symp Proc. 2010;2010:167-171.
PubMed
26.
Downs  SM, Biondich  PG, Anand  V, Zore  M, Carroll  AE.  Using Arden Syntax and adaptive turnaround documents to evaluate clinical guidelines. AMIA Annu Symp Proc. 2006:214-218.
PubMed
27.
Downs  SM, Carroll  AE, Anand  V, Biondich  PG.  Human and system errors, using adaptive turnaround documents to capture data in a busy practice. AMIA Annu Symp Proc. 2005:211-215.
PubMed
28.
Anand  V, Carroll  AE, Downs  SM.  Automated primary care screening in pediatric waiting rooms. Pediatrics. 2012;129(5):e1275-e1281.
PubMedArticle
29.
Squires  J, Twombly  E, Bricker  D, Potter  L. The ASQ-3 User's Guide. Baltimore, MD: Paul H. Brookes Publishing; 2009.
30.
Hix-Small  H, Marks  K, Squires  J, Nickel  R.  Impact of implementing developmental screening at 12 and 24 months in a pediatric practice. Pediatrics. 2007;120(2):381-389.
PubMedArticle
31.
Cabana  MD, Rand  CS, Powe  NR,  et al.  Why don’t physicians follow clinical practice guidelines? a framework for improvement. JAMA. 1999;282(15):1458-1465.
PubMedArticle
32.
Mangione-Smith  R, DeCristofaro  AH, Setodji  CM,  et al.  The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357(15):1515-1523.
PubMedArticle
33.
Carroll  AE, Bauer  NS, Dugan  TM, Anand  V, Saha  C, Downs  SM.  Use of a computerized decision aid for ADHD diagnosis: a randomized controlled trial. Pediatrics. 2013;132(3):e623-e629. doi: 10.1542/peds.2013-0933.
PubMedArticle
34.
Bauer  NS, Carroll  AE, Downs  SM.  Understanding the acceptability of a computer decision support system in pediatric primary care . J Am Med Inform Assoc. 2013;21(1):146-153.
PubMedArticle
35.
Bauer  NS, Strum  LA, Carroll  AE, Downs  SM.  Computer decision support to improve autism screening and care in community pediatric clinics. Infants Young Child. 2013;26(4):306-317.Article
×