Free access

Research Article

23 November 2018

Validity and reliability of the Dundee difficult times of the day scale in Chinese children and adolescents with attention-deficit/hyperactivity disorder

Authors: Yasong Du, Yi Zheng, Xiaoyan Ke, Linyan Su [email protected], David Coghill, Yun Chen, Yanlei Zhang, Zheng Yuan, Yan Cheng, and Wendong ChenAuthor Info & Affiliations

Publication: J. Comp. Eff. Res.

Volume 8, Number 1

https://doi.org/10.2217/cer-2018-0091

PDF

Abstract

Methods: A cross-sectional study enrolled 200 children and adolescents with attention-deficit/hyperactivity disorder visiting four Chinese tertiary care psychiatric clinics to assess the validity and reliability of the Dundee difficult times of the day scale (D-DTODS), using the Weiss functional impairment rating scale for parents form (WFIRS-P) and the Swanson, Nolan and Pelham, version IV 26-item teacher and parent rating scale (SNAP-IV-26). Results: The calculated Cronbach's-α for the D-DTODS total score was 0.793. The calculated Spearman's correlation coefficients for D-DTODS versus WFRIS-P and Swanson, Nolan and Pelham, version IV 26-item teacher and parent rating scale (SNAP-IV-26) were 0.425 (p < 0.01) and 0.452 (p < 0.001), respectively. Conclusion: The D-DTODS was worth future test–retest confirmation regarding reliability and validity for assessing functional impairment associated with attention-deficit/hyperactivity disorder across different time periods of the day in Chinese children and adolescents.

Attention-deficit/hyperactivity disorder (ADHD) is the most common neurodevelopmental disorder [1]. Onset of ADHD usually occurs in childhood and is characterized by excessive activity, difficulty in paying attention and impulsivity, and is associated with a range of negative outcomes including challenges associated with developing interpersonal relationships [1]. The prevalence of ADHD in Chinese children and adolescents is around 6.3%, with attention deficit as the main ADHD subtype [2]. ADHD is associated with symptoms and functional impairment across the whole day, including during daytime at school and in the early mornings, late afternoon and evenings at home [3]. Because the tasks and demands on the child vary by setting, parent rating scales for ADHD play crucial roles in estimating symptom severity and assessing treatment response for optimal management of ADHD [4]. The most commonly used parent rating scales for ADHD in China include the Swanson, Nolan and Pelham, version IV 26-Item Teacher and Parent Rating Scale (SNAP-IV-26), a measure of ADHD symptoms, and the Weiss functional impairment rating scale for parents form (WFIRS-P), a measure of ADHD-specific impairment [5–7]. However, these two rating scales were designed for the general assessment of ADHD but not for disease assessment during specific time periods of the day. With the recognition of the significant clinical implications of ADHD-related symptoms and functional impairment across the day [8], all-day ADHD management, including medication to cover evenings and weekends when at home, has been recommended by the American Academy of Pediatrics clinical practice guideline to minimize the burden of ADHD on patients, their families and society in general [9]. However, well-established parent rating scales that can assess the burden of ADHD across specific time periods throughout the day are lacking in China.

The Dundee difficult times of the day scale (D-DTODS) is a clinical tool developed at the University of Dundee to aid communication between clinicians and parents of children with ADHD [10]. D-DTODS asks about ten specific time points during the day and asks parents to assess the difficulties associated with ADHD for children at home and at school during these time periods. Ratings are based on an average day over the last month. Thus, D-DTODS can be used as a parent rating scale to assess the fluctuation of problems across the day and to assist the clinician in identifying the clinical needs for all-day ADHD management. D-DTODS has been used as a measurement to assess the efficacy of interventions, including coaching [11] and pharmacotherapy [12], for ADHD across the day. However, the reliability and validity of D-DTODS have not been well established. By contrast to the SNAP-IV-26 and WFIRS-P, D-DTODS was specifically designed to measure the functional impairment associated with ADHD throughout the day and to meet the medical needs of monitoring ADHD and tailoring ADHD management throughout the day. Additionally, it takes less than 5 min to complete the D-DTODS, which has simple and friendly graphics that make it easy-to-use and increase the generalizability across different clinical settings and countries. These features make the D-DTODS potentially very suitable for busy clinic settings in China where the conventional rating scales are often seen as too long and complex for day-to-day use. As the D-DTODS has not thus far been evaluated with regards to validity and reliability, we conducted this study to assess these factors and determine the potential clinical utility of the D-DTODS for all-day ADHD assessment in Chinese children and adolescents.

Materials & methods

This study was a cross-sectional study recruiting children and adolescents with ADHD from well-established psychiatric clinics in four Chinese cities to assess the validity and reliability of D-DTODS. As there are no established Chinese rating scales for the assessment of ADHD across different time periods of the day, two validated ADHD rating scales (SNAP-IV-26 and WFIRS-P) for the general assessment of ADHD symptoms and functional impairments were selected as the reference scales to validate D-DTODS. All ratings were made by parents. This study was conducted under the ethics approval from the four study sites. Study participation consent was required from the parents of the enrolled children with ADHD.

Patient enrollment

Patients were enrolled from four child and adolescent psychiatry clinics at four tertiary care hospitals in Beijing, Shanghai, Nanjing and Changsha. The patient enrollment was conducted in routine clinical practice at these clinics. This study included patients aged between 6 and 18 years who had a clinical diagnosis of ADHD according to the Diagnostic and Statistical Manual of Mental Disorders, 5th Edition (DSM-5) criteria [3]. Patients were excluded if they had any of the following comorbidities: nervous system degenerative disease, brain trauma, cerebral palsy, epilepsy, bipolar disorder, schizophrenia, substance abuse, psychotic disorder or other severe psychiatric disorders. The parents of the enrolled children with ADHD signed the printed study consent to finalize patient enrolment.

Study package

A study package was developed to facilitate data collection. The study package consisted of a study-specific data collection form and the Chinese versions of D-DTODS, WFIRS-P and SNAP-IV-26. The data collection form was designed to collect the characteristics of the enrolled children and adolescents (demographics, school status, family environment, ADHD diagnosis and ongoing treatment, and mental comorbidities) and their parents (social economic status, lifestyle, pregnancy, birth delivery mode and infant feeding source). The study psychiatrists completed the data collection form by assessing the included children and adolescents with ADHD and interviewing their parents. The D-DTODS was translated to Chinese using a forward-and-back translation approach. Approval for use of the D-DTODS was obtained from the University of Dundee. The Chinese versions of WFIRS-P and SNAP-IV-26 were previously validated [6,7]. The three rating scales were completed by the parents of the included children and adolescents with ADHD. The collected data and completed rating scales in the study packages were transferred to an electronic database developed by EpiData software. All entered electronic data were verified by an independent quality assurance associate.

The rating scales

D-DTODS consisted of ten items that were rated by four scores from 1 to 4 (1: no difficulties; 4: the most difficulties) to assess the difficulty of activities for ‘morning before school’ (items 1–3), ‘in school’ (items 4–6), ‘after school’ (item 7), ‘school homework’ (item 8) and ‘bedtime’ (items 9 & 10). The total score of D-DTODS was calculated by the sum of the subscore of the 10 items. SNAP-IV 26 included three subsets to assess the severity of symptoms for inattention (items 1–9), hyperactivity/impulsivity (items 10–18) and oppositional criteria (items 19–26) [13]. Subscale scores on the SNAP-IV 26 are calculated by summing the scores of the items in the same subset. To specifically address overall symptoms of ADHD, the subsets of inattention and hyperactivity/impulsivity were combined to calculate a total score for ADHD. WFIRS-P included five domains including family, school, life skills, child's self-concept, social activities and risky activities [14]. Each domain of WFIRS-P consists of ten items. The subscale score for each domain was calculated by dividing the summed items scores by the number of items in the same domain.

Statistical analysis

A total of 200 patients were planned to be enrolled in order to make sure that the item subject ratio was >20 and an absolute minimum was >100 patient. Each study site was assigned to enrol 50 eligible patients. The characteristics of included children with ADHD and their parents were summarized using descriptive statistical methods. The internal consistencies of D-DTODS were measured by Cronbach's-α [15], which was calculated for all items and also the items within the same subset. The convergent validity of D-DTODS was assessed through the Spearman's correlations between overall or subscale scores associated with the two criteria rating scales. To explore the discrimination threshold of D-DTODS for determining ADHD severity, the receiver-operating characteristic (ROC) analysis method was used by using a binary classifier system with each D-DTODS cut-off score, which increased by five points from 0 to 40 to detect significant ADHD defined by SNAP-IV 26 mean score of item 1–18 (<1.67 for insignificant ADHD symptom vs ≥1.67 for significant ADHD symptom in top 5% of the average rating per item score in the patients with ADHD) [16] and WFIRS-P score (none of the question items scored 3 for insignificant function impairment vs at least one question item scored 3 for significant functional impairment) [17], respectively. The sensitivity and specificity associated with each cut-off score of D-DTODS for significant ADHD symptoms and function impairment were plotted to calculate the area under ROC (AUC). The data analyses were performed using SAS 9.2 (SAS Inc., NC, USA) and the statistical significance was defined as a two-sided p-value less than 0.05.

Results

The initial screening identified 270 children and adolescents with suspected ADHD at the four study sites. Further eligibility assessment excluded 40 patients who didn't meet the diagnosis criteria of ADHD. An additional 30 patients were excluded because their parents declined the study. A total of 200 children and adolescents with ADHD were enrolled and included in the final data analysis. The patient enrolment flowchart is illustrated in Figure 1.

**Figure 1.** Flowchart of patient enrollment.
ADHD: Attention-deficit/hyperactivity disorder; BAH: Beijing Anding Hospital; NBH: Nanjing Brain Hospital; SMHC: Shanghai Mental Health Center; SXH: The Second Xiangya Hospital.

Characteristics of the included patients with ADHD & their parents

Similar to the previously reported demographics of diagnosed patients with ADHD in China [18], the majority of the 200 included patients with ADHD were elementary school students (76.5%), boys (77.5%) and living in a single-child family (68.5%). This study cohort was associated with a relatively short mean length of time since the ADHD diagnosis was made of 62.6 ± 96.6 weeks and mainly included the two ADHD subtypes (inattention: 45.5%; combined inattention and hyperactivity/impulsivity: 49.0%). Ongoing treatment, mainly drug treatment, was reported in 65.5% of the included patients. The identified main neurodevelopmental and mental health comorbidities in this study cohort included learning disability (24.5%), oppositional defiant disorder (10.0%) and tic disorder (10.5%). Additionally, the parents of the included children with ADHD were characterized with high cesarean rate (49.2%) and low breastfeeding rate (42.7%). The detailed characteristics of the included patients with ADHD and their parents are summarized in Table 1.

Table 1. Summary of the characteristics of the included 200 patients with attention-deficit/hyperactivity disorder and their parents.

Study subjects	Variable	Mean ± SD (%)
	Demographics
	Age (years)	10.4 ± 2.66
	Male gender	77.5%
	Single-child family	68.5%
	School status
	Kindergarten	1.5%
	Elementary school	76.5%
	Middle school	17.5%
	High school or above	4.5%
	Family history of ADHD	9.0%
	Disease duration after diagnosis (weeks)	62.6 ± 96.6
	ADHD subtype
ADHD patients	Combination of inattention and hyperactivity/impulsivity	49.0%
	Inattention	45.5%
	Hyperactivity/impulsivity	5.5%
	Current ADHD treatment
	Any treatment	65.5%
	Medications	63.5%
	Behavior treatment	8.0%
	Parents training	5.0%
	School intervention	0.5%
	Mental comorbidities
	Learning disability	24.5%
	Tic disorder	10.5%
	Oppositional defiant disorder	10.0%
	Personality disorder	5.5%
	Stress	5.0%
	Birth delivery mode
	Vaginal delivery	50.8%
	Cesarean delivery	49.2%
Parents of the ADHD patients	Infant feeding score
	Breast milk	42.7%
	Breast milk substitutes	25.6%
	Mixed milk	31.7%

ADHD: Attention-deficit/hyperactivity disorder; SD: Standard deviation.

The rating scores of D-DTODS, SNAP-IV 26 & WFIRS-P

Of the rated ten items of D-DTODS, item 8 for completing school homework was associated with the highest score (2.93 ± 0.85) and item 5 for lunchtime was associated with the lowest score (1.59 ± 0.82). The average score per item of D-DTODS was 2.1 ± 0.53, indicating a certain degree of difficulty associated with daily activities in the study cohort. The rating results of the ten items of D-DTODS are summarized in Table 2. The average SNAP-IV 26 total score and mean score for the ADHD items (items 1–18) was 26.5 ± 9.1 and 1.5 ± 0.5, respectively. The rated subscale of SNAP-IV 26 for inattention subset (items 1–9) was numerically higher than that for hyperactivity/impulsivity (items 10–18) in the study cohort (15.6 ± 4.8 vs 10.9 ± 6.1). The mean scores per item associated with the six domains of WFIRS-P ranged from 0.34 (for risky activities) to 1.11 (for life skills). The average WFIRS-P score per item was 0.81, indicating modest function impairment associated with this ADHD cohort.

Table 2. The rating results of Dundee difficult times of the day scale from the 200 included patients with attention-deficit/hyperactivity disorder.

Testing item	Sample size	1 = Easy	2 = A little difficult	3 = Difficult	4 = Very difficult	Mean (SD)	Median
Before school
1. Waking up early in the morning	200	74 (37.0%)	61 (30.5%)	38 (19.0%)	27 (13.5%)	2.09 (1.05)	2
2. Getting up and getting ready for school	200	81 (40.5%)	65 (32.5%)	41 (20.5%)	13 (6.5%)	1.93 (0.93)	2
3. Getting to school	200	103 (51.5%)	60 (30.0%)	30 (15.0%)	7 (3.5%)	1.71 (0.85)	1
In school
4. Morning lesson	200	28 (14.0%)	56 (28.0%)	84 (42.0%)	32 (16.0%)	2.60 (0.92)	3
5. Lunchtime	200	116 (58.0%)	58 (29.0%)	18 (9.0%)	8 (4.0%)	1.59 (0.82)	1
6. Afternoon lessons	200	25 (12.5%)	67 (33.5%)	84 (42.0%)	24 (12.0%)	2.54 (0.86)	3
After school
7. Getting home from school	200	99 (49.5%)	67 (33.5%)	24 (12.0%)	10 (5.0%)	1.73 (0.86)	2
Evening
8. Completing school homework	200	7 (3.5%)	58 (29.0%)	77 (38.5%)	58 (29.0%)	2.93 (0.85)	3
Before bed
9. Getting ready for bed	200	76 (38.0%)	74 (37.0%)	33 (16.5%)	17 (8.5%)	1.96 (0.94)	2
10. Going to bed and sleeping	200	124 (62.0%)	44 (22.0%)	22 (11.0%)	10 (5.0%)	1.59 (0.88)	1
Overall	200	73.3 (36.7%)	61 (30.5%)	45.1 (22.6%)	20.6 (10.3%)	2.1 (0.53)	20

SD: Standard deviation.

Reliability of D-DTODS

The computed Cronbach's-α for D-DTODS (0.793) exceeded the generally accepted cut-off of 0.7 for reliability. Additionally, the computed Cronbach's-α for the D-DTODS subsets with two or more items were 0.721 for ‘morning before school’, 0.629 for ‘in school’ and 0.709 for ‘before bed’.

Validity of D-DTODS

The Spearman's correlation analyses observed that the D-DTODS total score, the sum of the ten item scores, was significantly correlated with both SNAP-IV 26 total score (coefficient 0.452; p < 0.01) and WFIRS-P average score (coefficient 0.425; p < 0.01; Table 3). The coefficients associated with five D-DTODS subsets for SNAP-IV 26 total score and WFIRS-P average score ranged from 0.256 to 0.377 and from 0.206 to 0.332, respectively. Further correlation analyses between the subsets of D-DTODS and SNAP-IV 26 observed the strongest correlation between D-DTODS ‘evening time’ subset and SNAP-IV 26 inattentive subset (coefficient 0.353; p < 0.01). The D-DTODS ‘evening time’ subset was associated with the strongest correlation with WFIRS-P family subset (coefficient 0.396; p < 0.01). Additionally, D-DTODS ‘morning before school’ subset had the weakest correlation with the SNAP-IV 26 total score (coefficient 0.256; p < 0.01) and WFIRS-P average score (coefficient 0.206; p < 0.01).

Table 3. Summary of Spearman's correlation coefficients for Dundee difficult times of the day scale versus Swanson, Nolan and Pelham, version IV 26-item teacher and parent rating scale and Weiss functional impairment rating scale for parents form.

Rating Scale	Subsets	D-DTODS subsets
		Total score				Morning before school				In school				After school				Evening time				Bedtime
		Coefficient	Lower limit of 95% CI	Upper limit of 95% CI	p-value	Coefficient	Lower limit of 95% CI	Upper limit of 95% CI	p-value	Coefficient	Lower limit of 95% CI	Upper limit of 95% CI	p-value	Coefficient	Lower limit of 95% CI	Upper limit of 95% CI	p-value	Coefficient	Lower limit of 95% CI	Upper limit of 95% CI	p-value	Coefficient	Lower limit of 95% CI	Upper limit of 95% CI	p-value
	Overall score	0.452	0.328	0.561	<0.01	0.256	0.119	0.383	<0.01	0.291	0.156	0.415	<0.01	0.377	0.247	0.494	<0.01	0.342	0.21	0.462	<0.01	0.313	0.179	0.436	<0.01
	Inattention	0.385	0.255	0.501	<0.01	0.205	0.067	0.336	<0.01	0.33	0.197	0.451	<0.01	0.273	0.137	0.399	<0.01	0.353	0.221	0.472	<0.01	0.216	0.078	0.346	<0.01
SNAP-IV 26	Hyperactivity/impulsivity	0.316	0.182	0.438	<0.01	0.144	0.005	0.278	0.042	0.179	0.04	0.311	0.011	0.329	0.196	0.45	<0.01	0.181	0.042	0.313	0.01	0.254	0.117	0.381	<0.01
	Oppositional criteria	0.372	0.241	0.489	<0.01	0.254	0.117	0.381	<0.01	0.199	0.061	0.33	<0.01	0.267	0.131	0.393	<0.01	0.256	0.119	0.383	<0.01	0.285	0.15	0.41	<0.01
	Average score	0.425	0.299	0.537	<0.01	0.206	0.068	0.336	<0.01	0.311	0.177	0.434	<0.01	0.311	0.177	0.434	<0.01	0.332	0.199	0.453	<0.01	0.317	0.183	0.439	<0.01
	Number of items scored 2 or 3	0.42	0.293	0.532	<0.01	0.218	0.08	0.348	<0.01	0.308	0.174	0.431	<0.01	0.268	0.132	0.394	<0.01	0.344	0.212	0.464	<0.01	0.303	0.168	0.427	<0.01
	Family	0.448	0.324	0.557	<0.01	0.243	0.106	0.371	<0.01	0.28	0.144	0.405	<0.01	0.315	0.181	0.438	<0.01	0.396	0.267	0.511	<0.01	0.366	0.235	0.484	<0.01
WFIRS-P	School	0.311	0.177	0.434	<0.01	0.172	0.033	0.304	<0.01	0.347	0.215	0.467	<0.01	0.196	0.058	0.327	<0.01	0.215	0.077	0.345	<0.01	0.088	-0.052	0.224	0.22
	Life skills	0.285	0.15	0.41	<0.01	0.134	-0.005	0.268	0.059	0.196	0.058	0.327	<0.01	0.186	0.047	0.318	<0.01	0.176	0.037	0.308	0.013	0.301	0.166	0.425	<0.01
	Child's self-concept	0.339	0.206	0.459	<0.01	0.209	0.071	0.339	<0.01	0.157	0.018	0.29	0.027	0.194	0.055	0.325	<0.01	0.28	0.144	0.405	<0.01	0.311	0.177	0.434	<0.01
	Social activity	0.302	0.167	0.426	<0.01	0.131	-0.008	0.265	0.065	0.276	0.14	0.402	<0.01	0.264	0.128	0.391	<0.01	0.14	0.001	0.274	0.048	0.195	0.057	0.326	<0.01
	Risky activity	0.258	0.121	0.385	<0.01	0.127	-0.013	0.262	0.074	0.163	0.024	0.296	0.021	0.232	0.095	0.361	<0.01	0.131	-0.008	0.265	0.064	0.204	0.066	0.335	<0.01*

CI: Confidence interval; D-DTODS: Dundee Difficult Times of the Day Scale; SNAP-IV 26: Swanson, Nolan and Pelham, version IV 26-Item Teacher and Parent Rating Scale; WFIRS-P: Weiss Functional Impairment Rating Scale for Parents form.

Sensitivity & specificity of D-DTODS to detect significant ADHD symptoms & impairment

When using SNAP-IV 26 total score to define significant ADHD symptoms, the cut-off point for the D-DTODS total score associated with the discrimination cut point was 25, with a sensitivity of 43.3% and specificity of 86.5%. The AUC of ROC for using D-DTODS to detect significant ADHD symptoms was 0.711. When using the WFIRS-P average score to define significant functional impairment associated with ADHD, the identified D-DTODS discrimination cut-off point was 20, with a sensitivity of 65.8% and specificity of 64.0%. The AUC of ROC for D-DTODS was 0.672 for significant functional impairment associated with ADHD. Based on this assessment, the discrimination cut-off score of D-DTODS for significant ADHD ranged from 20 to 25. The ROC associated with D-DTODS for the significant ADHD symptoms and functional impairment defined by the SNAP-IV 26 and WFIRS-P scores are illustrated in Figures 2A&B, respectively.

**Figure 2.** ROC curve.
Plotted ROC for sensitivity versus 1-specificity associated with D-DTODS total score for the diagnosis of significant ADHD based on the SNAP-IV 26 **(A)** and Weiss functional impairment rating scale for parents form **(B)**. **(A)** ROC associated with sensitivity versus 1-specificity of D-DTODS for significant ADHD defined by SNAP-IV 26. **(B)** ROC associated with sensitivity versus 1-specificity of D-DTODS for significant ADHD defined by Weiss functional impairment rating scale for parents form.
AUC: Area under ROC; D-DTODS: Dundee difficult times of the day scale; ROC: Receiver-operating characteristic; SNAP-IV 26: Swanson, Nolan and Pelham, version IV 26-item Teacher and Parent Rating Scale.

Discussion

The D-DTODS was originally developed to facilitate communication between parents of children with ADHD and clinicians to help understand difficulties associated with daily activities of children with ADHD and profile these across the day. The D-DTODS hasn't been previously validated for all-day ADHD management. Our study is the first study to assess validity and reliability of the D-DTODS against two well-established rating scales for the general assessment of ADHD symptoms and functional impairment. Based on our assessment, D-DTODS was associated with a certain validity and reliability for the assessment of ADHD. Additionally, our study demonstrated that using the cut-off score of D-DTODS (20–25) could be better at detecting significant impairment associated with ADHD than detecting the symptoms of ADHD. Thus, combination of D-DTODS and a rating scale for ADHD symptoms could give a more comprehensive ADHD assessment. Because D-DTODS consists of fewer items than the standard rating scales, the simple and friendly format of D-DTODS is easy-to-use and the time to complete D-DTODS could be less than 5 min. Thus, the D-DTODS could be a potential rating scale to support all-day ADHD management by assessing ADHD during specific time periods across the day at home and in school from parent's perspective.

As it is different from SNAP-IV 26 that takes into account the DSM diagnostic criteria of ADHD, and from the WFIRS-P for which the items were specifically designed to address functional impairments commonly associated with ADHD [19,20], the D-DTODS comprises much more general items that give an overall impression of parents’ observation on the daily activities in children with ADHD and how they vary across the day. As a cross-sectional study, our study didn't use test–retest method but follow the reliability assessment method, which calculated the Cronbach's-α for the internal consistence of the 20 question times in the questionnaire - children with difficulties (QCD), another rating scale assessing ADHD throughout the day [21]. Because the calculated Cronbach's-α associated with D-DTODS total score was above generally accepted cut-off for reliability (0.7) [22] even under the potential confounding effects associated with the wearing off of the ADHD treatments, which could lead to more varied functional impairment in the afternoon and evening time of the day. Future studies are warranted to use a test–retest approach to confirm the reliability of D-DTODS in the patients with ADHD.

The moderate strength of the correlations between D-DTODS total scores and the two rating scales for the general assessment of ADHD demonstrated that D-DTODS had a certain level of capacity to assess the symptoms and function impairment associated with ADHD in Chinese children. Because D-DTODS was designed to assess ADHD across the day but not for general assessment, the differences in the designs of these rating scales were expected to reduce the correlations. Another rating scale developed to assess ADHD across the day, QCD, was also associated with moderate correlation (coefficient 0.514) with the rating scale for the general assessment of ADHD likely due to the same reason. Additionally, 65.5% of the included patients with ADHD were receiving treatment when our study was conducted. The high medication treatment rate in our study cohort could affect the correlations between D-DTODS and the two reference rating scales because medications were usually given in the morning time before school and the variance of treatment effects associated with the wearing off of the medications for ADHD across the day could be captured by D-DTODS but not by the two reference rating scales for general ADHD assessment. Thus, the validity of D-DTODS for ADHD assessment throughout the day using the two rating scales for general assessment could be underestimated and warrant future research to verify the validity of D-DTODS using the rating scale, such as QCD, which had a similar design to assess ADHD throughout the day.

The correlation coefficients between D-DTODS and the two standard rating scales were highly comparable. Because SNAP-IV 26 measures ADHD symptoms, and WFIRS-P is a functional impairment rating scale, this finding suggests that the ‘difficulty’ rated by the D-DTODS perhaps taps into both the symptoms and functional impairments associated with ADHD. For example, the difficulty in doing homework can be interpreted as the function reduction associated with inattentive symptom and also the indicator for the severity of the inattentive symptom. Thus, the intrinsic correlation between the SNAP-IV 26 and WFIRS-P (r: 0.648; 95% CI: 0.494–0.802) could be the main reason for the highly comparable validity associated with D-DTODS for the two reference rating scales. Additionally, the observed variance associated with coefficients between the D-DTODS subsets and the two reference rating scales may factor the differences in the designs of the three rating scales. For example, the correlation associated with the D-DTODS subset for the morning activities before school was much weaker than the other D-DTODS subsets likely due to the lack of assessment for morning activities in both SNAP-IV 26 and WFIRS-P and/or the comparable difficulties of morning activities between healthy children and ADHD children [3]. One study assessing the validity and reliability of QCD for the assessment of ADHD across the day, observed similar impact of ADHD assessment by time periods of the day on the correlation variance across QCD subsets [23]. Thus, the observed variance of the D-DTODS subscale scores by time domains throughout the day could be considered as the incremental validity to confirm the variance of ADHD symptoms and functional impairment across the day, which couldn't be detected by SNAP-IV 26 and WFIRS-P.

The two criteria rating scales identified the cut-off D-DTODS scores for significant ADHD symptoms and functional impairment. We recommend using D-DTODS score from 20 to 25 as the discrimination cut-off to indicate significant ADHD. The patients who had D-DTODS score ≥20 may have more severe ADHD symptoms and more impaired functions, and therefore require immediate attention from their caregivers. Because D-DTODS was not designed for the diagnosis but the measurement for the functional impairment during specific time periods of the day, the cut-off D-DTODS for significant ADHD should not be used for diagnosis but for disease monitoring and indicating the needs of ADHD management in specific time periods of the day. Thus, the cut-off ADHD score could be used as an outcome measure to assess the treatment effects as well.

Our study has significant clinical implications given the fact that ADHD management needs a parent rating scale to monitor disease severity and assess optimal treatment response across the day for optimal management of ADHD [24]. Even though the current parent rating scales are able to assess both symptoms and functional impairment of ADHD in both home and school settings, these rating scales were based on the overall observation and can't differentiate the disease severity and treatment response across different times in the day and support timely and accurate adjustment of ADHD management for better outcomes. D-DTODS can meet this clinical need by measuring the difficulty of general activities across the day and help with tailoring treatment to individual needs. Thus, D-DTODS could be used by parents and teachers to identify specific problem times during the day and guide the development of contingency management strategies. The current ADHD medications include both stimulant and nonstimulant medications, which are different in pharmacology and pharmacokinetics and time course of actions across the day [25,26]. Using the D-DTODS can help the clinician use these different medication properties to adjust treatment effects across the day and to meet the clinical needs of patients during the specific time periods of the day.

As the first study assessing the clinical utility of D-DTODS for ADHD in the Chinese setting, our study was associated with several limitations to be aware of and addressed. Of the initially identified 230 qualified patients with ADHD, the parents of 30 patients with ADHD declined our study. Additionally, the parents of the included children with ADHD from the four large cities were likely to have higher social economic status and be more active in seeking medical care for their ADHD children than the parents of ADHD children in the Chinese general setting. For example, the ADHD treatment rate in our study cohort was 63.5%, which is much higher than the previously reported treatment rate in Chinese children with ADHD [27]. Thus, the selection bias associated with our study could confound our assessment and also reduce the generalizability of the assessed reliability and validity of ADHD. Our cross-sectional study design didn't allow us to use a conventional test–retest method [28] to assess the internal consistency of D-DTODS and to assess the validity of D-DTODS in a longitudinal setting where the rating scales are used to assess ADHD treatment effects [29]. Thus, using D-DTODS to assess treatment effects in patients with ADHD should be cautioned before the reliability of D-DTODS is well established by test–retest method. D-DTODS was originally designed to profile difficulty across the day to ensure that treatments are effective across the whole day. Thus, future studies are still needed to assess the validity and reliability of D-DTODS in the patients with ADHD under treatment and support the utilization of D-DTODS to monitor treatment effects across the day. Finally, the D-DTODS contained the question item regarding the functional impairment at school where teacher could make more reliable and valid assessment. The D-DTODS was solely assessed by parents who might reduce the validity and reliability for the ADHD assessment during the time period in school.

In summary, this cross-sectional study was the first study proving that parent-completed D-DTODS was associated with satisfactory validity and reliability to assess ADHD functional impairments across the day in Chinese children and adolescents. The clinical utility of D-DTODS to assess ADHD treatment response should be further evaluated in longitudinal settings for all-day ADHD management.

Summary points

The first Chinese study validating Dundee Difficult Times of the Day Scale (D-DTODS), a parent rating scale consisting of ten question items to assess functional impairment associated with attention-deficit/hyperactivity disorder (ADHD) throughout the day.

The observed internal consistence associated with the ten question items of D-DTODS warrant future research using test–retest method to confirm the reliability of D-DTODS for ADHD assessment throughout the day.

The significant correlation between D-DTODS and the two general rating scales, Swanson, Nolan and Pelham, version IV 26-item teacher and parent rating scale (SNAP-IV-26) and Weiss functional impairment rating scale for parents form (WFIRS-P), suggested that D-DTODS was valid to assess the functional impairment associated with ADHD.

The varied subscale scores of D-DTODS for the five-time domains throughout the day in the ADHD patients indicated the variance of the functional impairment of ADHD throughout the day.

High treatment rate in the study subjects with ADHD could confound the functional impairment of ADHD throughout the day as the wearing off of the treatments might not be able to control the symptoms and functional impairment associated with ADHD throughout the day.

The varied subscale scores of D-DTODS in the study subjects with ADHD supported the needs of monitoring and managing ADHD throughout the day.

D-DTODS could be a suitable rating scale to assess functional impairment associated with ADHD throughout the day in a busy clinical setting.

D-DTODS should be used as a rating scale for the assessment of functional impairment associated with ADHD throughout the day but not a diagnosis tool for ADHD.

Open access

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/

Author contributions

Y Du, X Ke, Y Zheng, L Su, Y Cheng, Y Chen, Z Yuan and Y Zhang formulated the research idea and developed the study protocol. W Chen developed the study operation plan and study package. Y Du, X Ke, Y Zheng and L Su enrolled the eligible patients and collected data. D Coghill provided the permission to use the D-DTODS in this study. Y Chen developed the data analysis plan and performed the data analyses. Y Du, X Ke, Y Zheng, L Su, D Coghill, Y Zhang and W Chen contributed to the manuscript development. The developed manuscript has been critically reviewed and approved by all authors.

Financial & competing interests disclosure

This study was funded by Eli Lilly China. Y Du, X Ke, Y Zhen and L Su have received speaker honoraria from Eli Lilly China. W Chen is the founder of both Changsha Normin Medical Technology Ltd and Normin Health Consulting Ltd, which receive industry funds for health economics and outcomes research. Y Chen, Y Zhang, Y Cheng and Z Yuan are employees of Eli Lilly China. D Coghill has received honoraria or research support from Eli Lilly, Janssen–Cilag, Medice, Novartis and Shire and royalties from Oxford University Press. He developed the D-DTODS as an open-access instrument for which there is no charge, and did not receive any support and has nothing additional to declare for his contribution to this study. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.

Ethical conduct of research

The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.

References

Mittal VA, Walker EF. Diagnostic and statistical manual of mental disorders. Psychiatry Res. 189(1), 158–159 (2011).