Copyright 2008 American Medical Association. All Rights Reserved. Applicable FARS/DFARS Restrictions Apply to Government Use.2008
To describe sleep-wake patterns in kindergarten children by measures derived from questionnaire, diary, and actigraphy and to report rates of agreement between methods according to Bland and Altman.
Cross-sectional study, data from 7 nights of actigraph recordings and sleep diary and from a questionnaire.
Children studied in their homes.
Fifty children, aged 4 to 7 years.
Main Outcome Measures
Sleep start, sleep end, assumed sleep, actual sleep time, and nocturnal wake time derived from different methods.
Differences between actigraphy and diary were ± 28
minutes for sleep start, ± 24 minutes for sleep end, and ± 32 minutes for assumed sleep, indicating satisfactory agreement between methods, whereas for actual sleep time and nocturnal wake time, agreement rates were not sufficient (± 106
minutes and ± 55 minutes, respectively). Agreement rates between actigraphy and questionnaire as well as between diary and questionnaire were insufficient for all variables. Sex and age of children and socioeconomic status did not influence the differences between methods for all variables.
Actigraphy and diary may be interchangeably used for the assessment of sleep start, sleep end, and assumed sleep but not for nocturnal wake times. The diary is a cost-effective and valid source of information about children's sleep-schedule times, while actigraphy may provide additional information about nocturnal wake times or may be used if parents are unable to report in detail. It is insufficient to collect information by a questionnaire or an interview asking about children's normal sleep patterns.
The evaluation of children's sleep-wake patterns is essential for the identification and management of sleep problems, which affect 20% to 30% of children 1 or more times during childhood.1 The assessment of sleep-wake patterns in children is also an important procedure in basic and clinical pediatric sleep research.
A frequent approach to assess the child's sleep-wake patterns is simply to ask the parents as part of the interview. Additional data may be collected either by questionnaires or sleep diaries, which visualize sleep patterns over time. Other methods include actigraphy,
providing a reliable and objective estimation of sleep times and interruptions over multiple days,2,3 and polysomnography, tracking the microstructure of sleep in the laboratory over 1 or 2 nights.
These most commonly used methods are different in terms of duration,
effort, cost, and source of information. Interview procedures, questionnaires,
and diaries are based on parental reports (subjective information),
whereas actigraphy records objective measures of the child's sleep-wake patterns derived from movements. The source of information (subjective vs objective) in the evaluation of sleep-wake patterns has been a topic in several studies with different populations (adults vs children,
normal vs clinical samples) and different devices.4- 8 It has been repeatedly demonstrated that subjective, compared with objective, reports are limited by the restricted and biased knowledge that parents have about children's sleep.7 However, parental reports remain a time- and cost-effective way of collecting data in research and clinical settings.
We note that previous investigations have primarily focused on infants9,10 and adolescents11 while attention has been less strong on school-aged children. In fact, to our knowledge, only a single study assessed sleep patterns using different methodological techniques in kindergarten children.8 However,
none of the previous reports provides the clinician or sleep researcher with information about the interchangeable use of different methods.12 Do parents report on their child's sleep accurately? How well do actigraphy, diary, and questionnaire data agree? Can these methods be used interchangeably?
Authors most often reported reliability estimates, comparisons of mean values, and correlations between methods, which do not provide appropriate information about agreement or disagreement between methods.
Correlations generally indicate whether 2 methods are measuring the same underlying quantity.13 In fact,
the validity of comparing measurements (ie, reflected by high correlations)
is an essential requirement to compare different methods. A more suitable approach is to calculate mean differences between measurements on the same subject estimated by t tests. However, t tests do not provide an interval in which 95%
of the differences between measurements are expected to lie (limits of agreement).13 Bland and Altman13 proposed such limits of agreement for the interchangeable use of methods (with a 5% probability of error). Whether these limits are acceptable for the interchangeable use of compared methods needs to be answered from a clinical point of view.
The aims of this study are to (1) describe sleep-wake patterns in a nonclinical sample of healthy kindergarten children by measures derived from questionnaire, diary, and actigraphy and (2) report rates of agreement between methods according to Bland and Altman.13,14 We hypothesized, according to the previous literature,7,8 that for sleep start, sleep end, and sleep duration actigraphy and diary data would agree well, while questionnaire data may differ substantially when compared with actigraphy but less so with diary data; we expected a lower agreement between methods in respect to nocturnal waking.
A total of 50 kindergarten children (aged 4-7 years) participated in the study. Children were recruited from 34 of 270 kindergarten schools (13%) in the greater Zurich, Switzerland, area. Overall, 660
families of kindergarten children were invited to participate in the study. All families received a letter including a description of the study and registration details. Sixty-eight families (10%) agreed to participate. Eighteen of the 68 families were dropped from the study for a variety of reasons, including insufficient language skills and deregistration (n = 9), child's refusal to wear the actigraph (n = 3), technical problems (n = 4),
and actimeter loss (n = 2). Fifty of the 68 children (74%)
were included in the final data analysis. Three of the 50 families had 2 kindergarten children; both children took part in the study,
but only 1 child was included in the data analysis. All children were recruited sequentially from May 2006 until July 2007 (winter, n = 9;
spring, n = 18; summer, n = 8; and autumn, n = 15).
Each child was home-monitored with an actigraph (Actiwatch Plus AW4; Cambridge Neurotechnology, Cambridge, England) for 6 to 8 consecutive nights and days (mean, 7). Data were analyzed in 1-minute epochs and translated into sleep measures by software (Actiware 5; Cambridge Neurotechnology), using the scoring procedure described by Acebo et al.4 The scoring interval was defined as 30 minutes before the reported bedtime and 30 minutes after the reported rising time. Data were evaluated by medium sensitivity. Actigraphic sleep measures for the agreement analysis included (1) “sleep start time,” defined as the first minute of at least 3 consecutive minutes of scored sleep within the scoring interval; (2) “sleep end time,” the last minute of at least 5 consecutive minutes of scored sleep just prior to the end of the scoring interval; (3)
“assumed sleep” (“nocturnal sleep period”),
the difference between sleep end time and sleep start time, (4) “actual wake time,” the amount of time scored as wake during the nocturnal sleep period; and (5) “actual sleep time” (“true sleep time”), the difference between assumed sleep and the amount of time scored as wake during the nocturnal sleep period. Children were the actigraph continuously on their nondominant wrist, removing it only during times when it could get wet. They were only monitored during school time, including 1 weekend, but not during school vacation.
Parents were requested to complete a detailed sleep diary (15-minute intervals) during the actigraph monitoring days and to indicate bedtime by a “greater than” sign and estimated sleep start and estimated sleep end by a continuous line (Figure 1). Wake phases during the nights were indicated by breaks in the continuous line. This diary has been in clinical use at our center for several years. Measures derived from the diary for the agreement analysis included (1) “sleep start,”
defined as the beginning of the continuous line in the evening; (2)
“sleep end,” the ending of the continuous line in the morning; (3) “assumed sleep” (“nocturnal sleep period”),
the difference between estimated sleep start and sleep end without excluding nocturnal wake time; (4) “nocturnal wake time,”
the time of indicated wake time during the nocturnal sleep period;
and (5) “actual sleep time,” the difference between assumed sleep and nocturnal wake time. Parents were also instructed to write down the kindergarten timetable, any kind of child's illness, and car drives during the child's nighttime sleep period.
Actigraphy (AC) and diary (DI) data of 1 child (female, aged 6.1 years).
To match the amount of days and nights, individual actigraphy and diary nights were discarded in both methods if the child was sick (1 of 362 nights), if the actigraph was off for all or parts of the night (3 of 362), if parents had forgotten to fill out the diary (5
of 362), and if the diary indicated unusual external motion that would mask sleep (eg, sleeping in the car, 2 of 362). Data for each actigraph and diary measure were averaged over all nights, weekdays, and weekend days. These individual weekly means were the units of analysis. All public vacation days were counted as weekend days; weekend days ranged from 1 to 3 (median, 2). Three children (6%) had only 1 weekend day,
which was taken as the unit of analysis. The total number of monitored and logged nights was 351.
Normal wake up time, get up time, bedtime, time of lights off,
sleep latency, and potential nap duration were obtained through a self-constructed questionnaire (SSTQ). The questions were phrased as follows: “When does your child normally wake up?”,
“When does your child normally get up?” “When is your child ready to fall asleep (lights turned off)?” “How long does your child take to fall asleep (after lights turned off)?”
and “For how long does your child nap?” For all questions no answer categories were presented and information was collected separately for weekdays and free days. This distinction has also been used in other studies and is considered an important differentiation.15 Measures derived from the SSTQ for the agreement analysis included (1) “sleep start,” defined as sleep latency added to the time of lights off, (2) “sleep end,”
defined as the time when children wake up, and (3) “assumed sleep,” (“nocturnal sleep period”) defined as the difference between sleep end and sleep start. Time awake during the night was not obtained.
Sleep-related variables were assessed by a standardized interview.
Parents were asked about different aspects of behavioral sleep problems (eg, struggles at bedtime, difficulties falling asleep, and night waking). The questions were phrased as follows: (1) “Does your child normally resist or delay going to bed?” (2) “Does your child normally have difficulties falling asleep?” and (3)
“Does your child normally wake up during the night?” Responses were rated by the interviewer as “no” if normally no sleep problems occurred during the week, “rarely” if sleep problems occurred once to twice per week, “sometimes” for 3 to 4 times per week, and “often” for 5 to 7 times per week.
Answers to these questions including “sometimes” and “often”
were defined as “poor-sleeping” children.
Additional information was obtained for the time of assessment and history of medication, physical complaints, and surgical interventions (yes/no). Child's sleep quality was rated by the parents on a 10-point Likert scale ranging from very bad (1) to very good (10).
Socioeconomic status (SES) was estimated by means of a sum score of 2 strictly defined 6-point scales of paternal occupation and maternal education. The scale of paternal occupation ranged from nonskilled (1) to highly skilled, including an occupation with a university degree (6). The scale of maternal education ranged from nonattendance of the obligatory school time (1) to university degree (6). The sum scores ranged from 2 (lowest SES) to 12 (highest SES). Three social classes were defined as follows: SES scores 2 to 5, lower class; SES scores 6 to 9, middle class; and SES scores 10 to 12, upper class. This measure has been used in previous studies and has been shown to be a reliable and valid indicator of SES in our community.16,17
The study was approved by the local research ethics committee and was performed according to the Declaration of Helsinki. Parents were informed about the study by letter and assessed by a standardized interview and questionnaires. Informed written consent was received by all parents. The SSTQ was answered before the interview. The interview was conducted by a psychologist (H.W.) in the parents' homes. Actigraphy and diary data were collected following the interview, and diaries and actigraphs were sent back by surface mail. Each family was rewarded with a voucher.
Descriptive results are presented as mean (SD). Differences between weekdays and weekend days were examined using t tests for dependant samples. Pearson and Spearman correlations were calculated to measure associations between child's sleep and various other variables. The degree of agreement between different methods was quantified using the 95% limits of agreement by Bland and Altman,13,14 estimated by mean difference ± 1.96 SD of differences. Simple regression and analysis of variance were used to describe the relationship between sleep and demographic variables (age, sex, and SES).
All analyses were performed with 2-sided tests and a value of P < .05 was considered significant.
SPSS (14.0J for Windows; SPSS Inc, Chicago, Illinois) was used for all statistical analyses.
Mean age at time of assessment was 5.9 years (range, 4.5 to 7.3 years). There were 28 boys (56%) and 22 girls (44%); sex was nearly equally distributed (χ2 = 0.40; P > .05). Parents reported bedtime struggles in 16% of the children, difficulties falling asleep in 14%, and night waking in 26% (answers including “often” and “sometimes”).
Parents of 15 children (30%) indicated having difficulties in only 1 of the 3 behaviors; parents of 5 children (10%), in 2; and parents of only 1 child (2%), in all 3 behaviors. Children's sleep quality ranged from 3 to 10 (mean [SD], 8.98 [1.5]), with 50% of the children reaching the highest score. Parents reported neither organic sleep disorders nor any sleep medication taken at the time of assessment or any surgical intervention in the last month that potentially could influence their child's sleep.
The children were mostly from 2-parent families (mean SES score,
9.6 [range, 6-12]), with 50% in middle- and 50% in upper-class SES;
the lower class was not represented. Forty-four children (88%) lived with both parents and only 6 children (12%), with only 1 parent. Additional sample characteristics are presented in Table 1.
Mean (SD) of sleep start, sleep end, assumed sleep, actual sleep time, and nocturnal wake time assessed by the different methods are presented in Table 2. Children delayed their sleep start and sleep end from weekdays to weekend days only about half an hour but did not significantly sleep longer on weekends, although some inconsistencies exist between the different measures.
Kindergarten started mainly between 8:15 AM and 8:30 AM (mean [SD], 8:24 [0:12] AM)
and was directly related to the children's sleep end on weekdays (P < .05 for all methods) (Table 3). Children woke up about 1 hour before kindergarten started. Fifty percent of the children were awakened by a family member; 46%, by self; and 1 child, by an alarm clock,
and data for 1 child were missing. Only 3 children napped on weekdays (2 on weekend days).
No significant effects of children's sex and SES were found for sleep start, sleep end, assumed sleep, actual sleep time, and nocturnal wake time consistently for all 3 methods. On the other hand,
there was a significant relation between age and sleep start (analysis of variance; F1 = 10.6; P = .001) and assumed sleep (F1 = 12.6; P = .002); older children had a later sleep start and a reduced sleep duration.
To assess the agreement between actigraphy and diary, limits of agreement according to Bland and Altman13,14 were obtained. We calculated mean (SD) of the differences AC − DI, where AC represents actigraphy and DI represents diary, over actigraphy monitored and logged days for each sleep measure (sleep start, sleep end, assumed sleep, actual sleep time, and nocturnal wake time). Then we calculated the mean difference ± (1.96×SD).
On the basis of the assumption of normally distributed differences,
we would expect 95% of the differences to lie between the limits.
Limits of agreement were calculated separately for all days, weekdays,
and weekend days (Table 4).
Bland-Altman plots13,14 of the difference (AC – DI) against the mean [(AC + DI)/2] were presented for our 5 parameters (Figure 2). The differences for sleep start and sleep end were not normally distributed,
but the deviations were not strong (Q-Q plots not presented). A priori we defined a satisfactory agreement if differences were smaller than 30 minutes. This requirement was essentially satisfied for 3 of our 5 parameters, for which differences were ± 28 minutes,
± 24 minutes, and ± 32 minutes for sleep start,
sleep end, and assumed sleep, respectively. This requirement was not satisfied for actual sleep time and nocturnal wake time, whose differences were ± 72 minutes and ± 55 minutes (all days).
For weekdays and weekend days, the differences were larger. Sex and age of the children, the age of parents, and SES did not significantly influence the differences between actigraphy and diary for all 5 parameters (P > .05).
Plot of difference against mean for sleep start (A), sleep end (B), assumed sleep (C), actual sleep time (D), and nocturnal wake time (E).
Limits of agreement were also calculated for actigraphy and questionnaire data (AC − QU) as indicated earlier and reported in Table 5. A prior defined satisfactory agreement of 30 minutes was not reached for any parameters. Again, there were no significant sex, age, or SES effects on the differences (P > .05).
Limits of agreement were also calculated for diary and questionnaire data (DI − QU), and results presented for sleep start, sleep end, and assumed sleep separately for weekdays and weekend days (Table 6). A prior defined satisfactory agreement of 30 minutes was not satisfied for any of the 3 parameters for weekdays or weekend days. Again, there were no significant sex,
age, or SES effects on the differences (P > .05).
This cross-sectional analysis of sleep-wake patterns presents agreement rates of the most commonly used methods in basic sleep research and clinical pediatric sleep medicine (questionnaire, diary, and actigraphy)
by the statistical approach proposed by Bland and Altman13,14 and describes various parameters within the different methods in healthy kindergarten children. Our study provides normative data for agreement rates in a nonclinical sample of middle- and upper-class children.
The results are consistent with our hypothesis, indicating that the agreement between actigraphy and diary regarding sleep start,
sleep end, and assumed sleep is satisfactory. On the basis of our clinical experience, we defined an acceptable agreement between the methods if differences between measures were smaller than 30 minutes (notably, the diary intervals were 15 minutes). Both actigraphy and diary are approximations of the true value of a child's sleep patterns and we believe that a mean disagreement of 30 minutes between methods is insignificant for diagnosing and evaluating children's sleep difficulties.
Thus, we conclude that actigraphy and diary can be interchangeably used for sleep start, sleep end, and assumed sleep, but not for variables related to nocturnal wake times.
The agreement for actual sleep time and nocturnal wake time between actigraphy and diary data was not sufficient. This result is entirely in line with previous other studies (also in different age groups3,7,18) and may be explained by the parents not always being aware of the sleep behavior of their child. For example, when the child does not signal during the night, parents may not realize that their child is awake. A child's signaling may lead to higher awareness of a child's nocturnal waking.19 Therefore, the lack of agreement in night waking in our study may be because children in the study were awake during the night and did not alert their parents.
The diary recording of sleep-wake pattern in 15-minute intervals could also have affected the findings. However, collecting diary data in 15-minute intervals was seen to be most practical for the families in terms of adherence. Another explanation for the disagreement between actual sleep time and nocturnal wake time is that actigraphy and the diary measured different things (ie, nocturnal motor activity and subjective recollections of sleep).6
The agreement rates between actigraphy and the questionnaire as well as between the diary and questionnaire were not sufficient for any measure variable. This may be because the questionnaire did not ask about the last few days that diary and actigraphy data had been collected. The agreement rates between the questionnaire and actigraphy or diary may be lower the less recent the period covered by the questionnaire is because the answers may be influenced by memories,
experiences, and expectations (recall bias). We also know that answers of parents are often vague and inaccurate, with statements such as “it depends on the situation” and with reporting to the nearest half hour or even full hour. Thus, we believe that exclusive information from a questionnaire or interview asking about general sleep times is insufficient. The validity of a questionnaire may be improved by asking about a recent period (eg, the last week).
Limitations of this study are the population including predominantly white, middle- and upper-class children in urban and suburban Zurich and the low participation rate, which may be explained by the time-consuming procedure. The issue whether parents from a lower SES class would less reliably fill out the diary cannot be answered by our study.
In our population, the agreement rates between the methods were not influenced by SES, sex, or age of the children or the parent. The finding that preschool-aged children already show a delay of their sleep phase with later sleep start and sleep end during weekends20 highlights the need for collecting data for weekdays and weekend days separately.
Overall, the sleep-wake diary is a valid and inexpensive source of information about a child's sleep-schedule variables. The diary is an important baseline tool for the health care professional when evaluating and attending to children with behavioral sleep problems and is also a valuable instrument for the pediatric sleep researcher.
Additional information about nocturnal wake times may be collected by actigraphy, which can also be used if parents are unable to report in detail.
We are not aware of any other study in the sleep field using the statistical approach proposed by Bland and Altman13,14 for comparing different assessment techniques of sleep variables. In contrast to previous reported correlations and comparisons of mean differences, the estimation of limits of agreement proposed by Bland and Altman allows a statement regarding the interchangeable use (with a 5% probability of error)
of different methods. We encourage others to apply this statistical method in other age groups and clinical populations to further answer the question of which methods may be interchangeably used. Such information will be very helpful for the clinician.
Correspondence: Oskar G. Jenni,
MD, Child Development Center, University Children's Hospital Zurich,
Steinwiesstrasse 75, CH-8032 Zurich, Switzerland (email@example.com).
Accepted for Publication: December 7, 2007.
Author Contributions:Study concept and design: Werner and Jenni. Acquisition of data: Werner and Jenni. Analysis and interpretation of data: Werner, Molinari, Guyer, and Jenni. Drafting of the manuscript: Werner, Guyer, and Jenni. Critical revision of the manuscript for important intellectual content: Werner, Molinari, Guyer, and Jenni. Statistical analysis: Werner, Molinari, and Jenni. Obtained funding: Jenni. Administrative,
technical, and material support: Werner, Molinari, Guyer, and Jenni. Study supervision: Jenni.
Financial Disclosure: None reported.
Funding/Support: This work was supported by research grants from the Claus Cramer Foundation and the Theodor and Ida Herzog-Egli Foundation.
Additional Contributions: We thank the children and parents who participated in this study. A special thanks goes to Monique LeBourgeois, PhD, for many discussions about the topic and comments on the manuscript.
Werner H, Molinari L, Guyer C, Jenni OG. Agreement Rates Between Actigraphy, Diary, and Questionnaire for Children's Sleep Patterns. Arch Pediatr Adolesc Med. 2008;162(4):350-358. doi:10.1001/archpedi.162.4.350