An investigation to choose the proper therapy technique in the management of autism spectrum disorder
Publication: Journal of Comparative Effectiveness Research
Abstract
Aim: Autism spectrum disorder is a class of neurological disorders that affect the development of brain functions. This study aims to evaluate, compare and rank the therapy techniques used in the management of autism spectrum disorder using multicriteria decision-making approaches. Materials & methods: Fuzzy PROMETHEE and fuzzy TOPSIS approaches were used. Fuzzy PROMETHEE utilizes a pair-wise comparison of alternatives under the fuzzy environment while fuzzy TOPSIS utilizes geometric distance from the positive ideal solution under the fuzzy environment for the evaluation of the effectiveness of the alternatives.The techniques selected for evaluation are applied behavioral analysis, cognitive behavioral therapy, speech therapy and pharmacological therapy such as Risperidone and Aripiprazole. Criteria used in this study include efficacy, cost and side effects, and their weights are assigned based on specific patient conditions. Results: The results indicate that applied behavioral analysis, cognitive behavioral therapy and speech therapy are the most preferred techniques, followed by Aripiprazole and Risperidone. Conclusion: More criteria could be considered and the weights could be assigned according to the patient profile.
Lay abstract
Autism spectrum disorder is a neurodevelopmental disorder (affecting the development of the brain) that usually presents during childhood. Because autism spectrum disorder has no cure, selecting the best therapy to manage the disorder is important for therapists, parents, health institutions and researchers with an interest in these types of disorders. This study focuses on comparing specific therapy techniques by using multicriteria decision-making methods. The result obtained by a decision-maker is not always the same, as different decision-makers may come up with different solutions depending on things like the cost and how well a therapy works on different aspects of the disorder. The outcome of the study indicates that applied behavioral analysis, cognitive behavioral therapy and speech therapy are preferred treatment alternatives, followed by treatment with Risperidone and Aripiprazole. Further analyses are needed to obtain more accurate patient-specific results that will incorporate specific patient demographics and data, as well as looking at the combination of two or three techniques.
Autism spectrum disorder (ASD) is a heterogenous group of neurodevelopmental disorder. The term ASD is used by the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5) and additional specifiers are used to differentiate patients [1]. Clinical findings in ASD include impairment in social communication and interaction, restrictive interest, repetitive behaviors, sensory abnormalities and intellectual disability. ASD can be co-morbid with other conditions like Attention Deficit and Hyperactivity Disorder, anxiety, depression and epilepsy. The diagnosis of ASD is mainly based on a detailed developmental history and observation of the patient [2].
Although the underlying mechanisms and etiology of ASD remain unclear, there are several factors that are demonstrated to increase the risk of ASD. These include advanced paternal age, birth trauma causing hypoxia [3,4], gestational diabetes, maternal obesity, short duration in between pregnancies and valproate usage during pregnancy [3,4]. Environmental risk factors were hypothesized to increase risk via genetic and epigenetic effects like inflammation, hypoxia causing oxidative damage and ischemic process [4]. With increased accessibility and usage of genetic tests more than 100 genes and genomic regions have been associated with ASD [5,6]. In addition, family studies displayed a large genetic contribution with an estimated heritability ranging from approximately 40–90% [7,8].
The Global Burden of Disease Study estimated that 52 million people were living with autism globally in 2010, denoting one in 132 individuals [9]. The Autism and Developmental Disabilities Monitoring Network demonstrated ASD prevalence as 18.5 per 1,000 (one in 54) children aged 8 years, and ASD was 4.3-times as prevalent among boys as among girls [10]. The prevalence of ASD in people with mental disorders was demonstrated to range from 4 to 9.9% [11].
Disability and functioning across the lifespan highly vary among different individuals with ASD with different clinical involvement. A broad variety of functional differences had been demonstrated in respect to disability and exceptionality by Schipper et al. [12]. Sixty-five percent of people with ASD had been reported a profound or severe disability at least one of the functioning domains (Australian Bureau of Statistics). Outcomes of individuals with ASD highly vary according to cognitive, linguistic, social and behavioral limitations. It is hard to categorize and evaluate clinical studies collectively as the have different sampling methods, diagnostic criteria and follow-up. In a study with quantifiable criteria of outcomes, among 68 adults with autism with an IQ level of ≥50 majority (58%) had poor or very poor outcomes. Intelligence quotient (IQ) seems to be an important prognostic factor for the outcomes [13,14].
As a result of the heterogeneity in reviewing indicators of ASD, lack of biological diagnostic inferences and changing diagnostic criteria, tracking the spread of ASD is one of the unique challenges. Although the initial symptoms appear during the early growth stage, it may be difficult to define these symptoms as being diagnostic indications for ASD until the child has reached the important developmental stages where it is ensured that the child is unable to meet the criteria of social, educational and professional stages. The complex overlap in diagnostic criteria and clinical differences for decision-making in addition to the policy of standards that determine the eligibility for health benefits and educational programs complicate the possibility of identifying ASD as a behavioral or exceptional health diagnosis. There are various dimensions of the disorder like impaired social communication, restrictive interest, repetitive behaviors, sensory abnormalities and intellectual disability. Moreover, co-morbid disorders are demonstrated to be higher when compared with population [15,16]. Linguistic delay, motor problems, epilepsy, sleep disturbances and eating problems are observed in preschool children with autism [17,18]. For school-aged children anxiety, Attention Deficit and Hyperactivity Disorder, obsessive compulsive disorder and disruptive behaviors are observed [19], and for adolescents and adults’ depressive symptoms observed more [20].
Treatment of ASD can be broadly divided into behavioral/psychological interventions such as applied behavioral analysis (ABA) and biological interventions such as medication. In addition, treatment can be divided into those that address key underlying symptoms such as applied behavioral analysis (ABA) and those that treat associated features such as mood or aggression.
Most of the current interventions are based on ABA. There is great diversity among different interventions. For example, mode of delivery (parent mediated/therapist implemented), length of training (12-week to 2-year program), intensity (few hours/week to 15 h/week) highly varies. Individual outcomes within same groups also vary. Use of cognitive behavioral therapy (CBT), for instance, demonstrated to be effective in the treatment of anxiety in individuals with autism with sufficient cognitive and language skills [21]. Cognitive impairment and linguistic problems set limitation for CBT.
The medications that are reported to be beneficial for autism are not directly against the autistic domains. The drugs that are approved by the US FDA for the treatment of irritability and agitation (including aggression, self-injury and tantrums) in autism are the neuroleptics Risperidone and Aripiprazole [22–24]. Both of these drugs have side effects like sedation, weight gain and risk of movement disorders [23].
The cost of treatment for autistic patients in the USA reached US$268 billion in 2015 and is expected to increase to US$461 billion by 2025 [25]. For a professional, it is important to make the right decision for the benefit of the patients. It is crucial and potentially difficult to choose the best treatment choice. Several parameters underlie the approach taken to the correct treatment and these requirements are not always well described and presented. Efficiency and side effects are the key criteria for assessment, though other criteria such as cost, safety, treatment time and patient delivery should be taken into account. Well-known models of treatment help health providers choose and have better strategies for treating conditions. An additional advantage of evaluating modes of care is that it helps to speed up patient remission and improve prognostic outcomes.
In the absence of uncertainties, the fuzzy multicriteria decision-making (MCDM) methods applied have become an important method to help professionals compare and make better alternatives. This is a powerful approach to addressing decision-making issues involving the selection and implementation of the system based on a variety of critical criteria for characterizing alternatives [26,27].
The ideal treatment is acknowledged as including features such as having efficiency, a rapid duration of action, safety, an absence of any adverse side effects and comfort. Therefore, it is the task of the clinician to determine the most appropriate option for each individual for maximizing the patient’s comfort while minimizing the patient’s risks and treatment’s possible side effects. To do so, the clinician carries out a preoperative evaluation and assesses all possible factors, including individuals’ medical history in a cost-effective manner that might have an impact on the selection process of an optimal treatment technique. This study aims to evaluate, compare and rank the therapy techniques used in the management of ASD using the fuzzy preference ranking organization method for enrichment evaluation (PROMETHEE) and technique for order preference by similarity to ideal solution (TOPSIS) – Multi Criteria Optimization and Compromise Solution, MCDM methods. Applying MCDM methods in evaluating treatment options of ASD, based on the selected criteria and their assigned weights, may contribute to a more accurate decision-making process for the clinicians and result in better management of the disorder.
Specific therapy techniques
The investigation done in this study is to apply MCDM methods to the selecting the ‘right’ therapy technique in the management of ASD. In order to evaluate the therapy techniques, first, we defined the widely used techniques and extract the important parameters of interest such as efficacy, cost and side effects of each technique. Here, we list and summarize the most common therapy techniques, then, explain the MCDM techniques, and finally show how to apply them.
The plasticity of the brain is highest during childhood and decreases with aging. Based on this, ASD treatments and interventions delivered during preschool ages are considered to have the highest benefit, even though this has not proven empirically. Clinical trials on ASD are inappropriate for a comparison. The main limitations are limited outcome measurements, costs and placebo effects [28].
CBT, for instance, requires trained personnel and is time consuming. It has been demonstrated in adult studies to be better applied to individuals with sufficient cognitive and language skills [21]. Research has indicated that considerable progress in cognitive and language skills in an infant with ASD can be achieved when behavioral therapy is mandated and applied during their preschool years. There is no single optimal treatment for all children with ASD [29]. The goal of treatment in autistic cases is to improve the child’s functional abilities, quality of life and provide support to the patient. A major limitation is that behavior is affected by many factors and these need to be considered within the context of the developing child.
The National Institute of Mental Health [30] recommends starting treatment immediately after diagnosis. Early detection and treatment for ASD are important as suitable care can reduce individual impairments while helping them to overcome difficulties by learning new skills and improving most of their strengths. The heterogenicity of ASD and the diversity of therapy options makes it difficult to determine a single and optimal treatment for the condition. The treatment system can include different kinds of behavioral therapy techniques to improve communication and behavior, and sometimes medications may help to manage co-morbid conditions related to ASD. The goal of the treatments is the same for all types of therapeutic techniques, namely to reduce ASD symptoms, improve communication skills and function to be able to provide a better quality of life. Behavioral and social therapy, speech therapy and pharmacological therapy were the therapies of choice in this study.
The selective serotonin reuptake inhibitors (SSRIs) are widely used in the treatment of symptoms related to ASD. In a review evaluating the efficacy and tolerability of three commonly used SSRIs (fluoxetine, fluvoxamine, citalopram) conducted no convincing evidence for the use of them in the treatment of children with ASD. In addition, evidence of them for clinical utility in adults is inadequate [31]. Hence, SSRIs are not considered in this study.
Behavioral & social therapy
Behavioral and speech-language therapy forms the basis of ASD treatment [30]. Behavioral therapy is an effective and safe approach for dealing with children with ASD. However, the high cost of this approach and the need for labor intensity is one of the disadvantages that is difficult to solve in addition to the fact that it includes a variety of methods, which make it more difficult to decide on the optimal method. Nevertheless, behavior therapy remains the most reliable way of developing the skills of autistic children, provided that the time and energy focus is located, whether parent mediated or therapist implemented.
Applied behavioral analysis
This therapy is based on focused theories aimed at developing learning and adapting, based on an organized scientific approach that includes specific intervention goals for positive reinforcement and repetition of learning experiences, where specific methods are used to teach how to play and socialize and enhance self-care and academic skills, and thus reduce problematic behaviors [32]. In the ABA approach, the child’s abilities are observed by the therapist as a first step to determine what will benefit the applications that help the patient to move away from repetitive and focused behaviors. ABA is the usual starting point for treating children with severe symptoms to enhance skills and improve social communication. Outcomes depend on several factors. For example, parent-mediated, lower intensity, time-limited interventions may overt no improved outcome [21].
Cognitive behavioral therapy
CBT is used for children and adolescent with moderate symptoms, especially for the treatment of anxiety in individuals with autism with sufficient cognitive and language skills [22]. This treatment enhances the knowledge of the affected children at the moment they embarked on the habitual behavioral or mental frequent activity. The therapist determines the behavioral stimuli needed by the ASD children in the case in question in order to help them recognize these moments themselves. These behavioral stimuli interfere in cases of severe fear and anxiety to reduce the impact of the stress of the condition on the effectiveness of these children’s activities.
Speech therapy
Speech therapy deals with verbal and nonverbal social communication in individuals with ASD. This therapeutic technique aims to help autistic people to communicate in ways that are highly effective and practical; therefore, the help of a speech and language pathologist is critical for taking the treatment to a more efficient level. Nonverbal methods like ‘sign language, visual aid systems and picture exchange communication system (PECS)’ are also used [33]. One of the most prominent predictions of a positive outcome of treatment is the presence of a communication discourse before the age of five. At the time of diagnosis, individuals with ASD are evaluated by speech therapists for the best ways to enhance a person’s quality of life by resolving problems related to communication skills. If the individuals with ASD are nonverbal or have significant speech problems, the speech therapist may introduce alternatives to speech.
Pharmacological therapy
Risperidone
It was the first drug approved by the FDA in 2006 for children with autism at the age of five and above, to treat irritation associated with ASD and is a second-generation antipsychotic treatment [34]. Risperidone relieves problematic behaviors, such as mood changes, tantrums and aggressive behaviors toward oneself and others [35]. The main adverse effects of Risperidone include problems with vision, movement, symptoms of dizziness and drowsiness, as well as intestinal problems such as constipation and increase in weight [36].
Aripiprazole
This drug was approved as a treatment used to treat irritability associated with symptoms of autism for children aged 6–17 years old by the FDA in 2009 [37]. The known adverse effects include bowel problems with vomiting and constipation, as well as symptoms of dizziness, drowsiness and weight gain. Serious adverse effects may include serious problems such as neuroleptic malignant syndrome and delayed dyskinesia. The risk increases in the case of the elderly, as it is not recommended for psychosis due to dementia and may even cause death.
Materials & methods
Fuzzy PROMETHEE
Fuzzy PROMETHEE is a practical approach to decision-making toward a specific problem, its applications are classified as relatively simple compared with other approaches. Many approaches and methods have been proposed and developed to solve multicriteria problems, and fuzzy PROMETHEE is one of the MCDM methods [38]. This method is capable of imitating human reasoning ability via computers for decision-making on uncertainties. It is a set theory that is highly applicable and appropriate in creating a knowledge-based system for neurodevelopmental disorder.
The PROMETHEE technique is based on the mutual comparison of each alternative pair with regard to each selected criteria. This model is one of the efficient methods in conception and application compared with other MCDM methods. It is a user-friendly outranking method and has been successfully implemented to the real-life planning problems. PROMETHEE I and PROMETHEE II give partial and net ranking of the alternatives, respectively, while still satisfying simplicity [39]. It requires only two types of information: weights of the criteria considered and the decision-makers preference function when comparing the contribution of the alternatives in terms of each separate criteria [40]. The preference function (pj) denotes the difference between the evaluations obtained with two alternatives (at and at′)) with regard to particular criteria, within a preference degree ranging from 0 to 1. There are six different types of preference functions that can be used to implement PROMETHEE method: usual function, U-shape function, V-shape function, level function, linear function and Gaussian function. The basic steps of the PROMETHEE method [41] are as follows:
Step 1. For each criteria j, determine a specific preference function pj(d).
Step 2. Define the weights of each criteria wT = (w1, w2, …, wk). At the discretion of the decision-maker, each weight of the criteria can be taken equally only if their importance is equal. And also, normalization can be used for the weights;
Step 3. For all the alternatives, at, at′ ∈ A, define the outranking relation π
where pk is the weighted average function, A is the alternative and AχA denotes the set of all possible alternative pairs. Here, π(at, at′) denotes the preference index that is a measure for the intensity of preference of the decision-maker for an alternative at in comparison with an alternative at′ while considering all criteria simultaneously.
Step 4. Determine the leaving and entering outranking flows as follows:
▸ Leaving (or positive) flow for the alternative at:
▸ Entering (or negative) flow for the alternative at:
where n is the number of alternatives. Here, each alternative is compared with n − 1 number of other alternatives. The leaving flow Φ+(at) expresses the strength of alternative at ∈ A, while the entering flow Φ−(at) denotes the weakness of alternative at ∈ A.
Via these outranking flows, the PROMETHEE I method can provide a partial preorder of the alternatives and PROMETHEE II method can provide the complete order based on the net flow.
Step 5. Determine the partial preorder on the alternatives of A according to following principle:
In PROMETHEE I, alternative at is preferred to alternative at′ (atPat′) if it satisfies one of the following conditions:
When two alternatives at and at′ have the same leaving and entering flows, at is in different to at′ (atIat′)
at is incomparable to at′ (atRat′) if;
Step 6. Determine the net outranking flow for each alternative
Via PROMETHEE II, the complete preorder can be obtained by the net flow and defined by:
Basically, the better alternative is the one having the higher Φnet(at) value.
Fuzzy PROMETHEE applicability spans across many fields such as energy [38], nuclear medicine [42,43], material selection problems [44], cancer treatment options [44–46] and water treatment technique [47]. In this paper, we present a fuzzy PROMETHEE approach to evaluate the therapy techniques in the management of ASD and to select the best fit among all therapy technique types. The criteria are expanded and considered for the ranking, which incorporates a decision-making method dependent on multicriteria and pair correlation of available options. The augmentation of criteria is essentially created by the subject taking the issue effectively through considering the situational weight of the preferences. While addressing the ranking problem utilizing PROMETHEE, two conceivable outcomes are possible: the first gives a partial outcome and the second produces a complete ranking. To execute the technique we need two data, criteria weight and the preference function while figuring the distinction of choices commitment for individual criteria.
Application of fuzzy PROMETHEE to therapy techniques
To evaluate therapy techniques in the management of ASD, we need to choose suitable criteria. Among the accessible criteria, we have chosen the cost of therapy per session, duration of the treatment, efficacy, side effect, safety and lastly, the duration of action of each therapy technique. As autistic children can show different symptoms, finding the best therapy can be difficult. Symptoms can range from mild to severe and can change as the child grows. Weight was assigned based on the importance of each criteria. An autistic patient may undergo one or a combination of therapies depending on the symptoms shown and the severity of the disorder. While undergoing therapy, the patient may be affected by the side effects that could present in addition to the benefits of the therapy. Therapy techniques such as Risperidone may cause moderate and manageable side effects such as movement problems, sleepiness, dizziness, trouble seeing, constipation and increased weight. However, Aripiprazole may cause dizziness, vomiting, sleepiness, weight gain, constipation and movement disorder, neuroleptic malignant syndrome and tardive dyskinesia. Table 1 shows the therapy techniques, their respective parameters and corresponding values.
| Therapy techniques | ABA | CBT | Speech therapy | Risperidone | Aripiprazole |
|---|---|---|---|---|---|
| Criteria | |||||
| Cost per session | H | H | M | M | H |
| Duration of treatment | VH | H | H | M | M |
| Efficacy on social/behavioral functioning | VH | VH | H | L | L |
| Efficacy on irritability/aggression | M | M | L | VH | VH |
| Efficacy on depression | L | L | 0 | L | L |
| Efficacy on speech | M | 0 | H | 0 | 0 |
| Side effects | 0 | 0 | 0 | M | M |
| Safety | VH | VH | VH | M | M |
| Duration of action | L | M | M | H | H |
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy; H: High; L: Low; M: Medium; VH: Very high; VL: Very low.
To successfully evaluate the therapy techniques, the triangular fuzzy scale was utilized. Weight were assigned after analyzing the important features of therapy such as the cost per session, duration of treatment, efficacy on different aspects of ASD, side effects, safety and duration of action. In this study, we conducted two experiments for two different scenarios: Child A and Child B. Child A had very severe aggression, mild depression, fairly normal speech and infinite money, while Child B with autism had no money, no aggression or depression, but severe speech and language problems. The efficacy of the therapy was assigned to the maximum weight and classified as very high for both children. This is because efficacy is the most important criteria in improving the functional abilities, quality of life and survivability of the patient. Cost per session and duration of treatment were also classified as very high for Child B due to the financial issues, while the cost was assigned to the minimum importance for Child A. Other parameters such as side effects, safety and duration of action were given lesser importance because they do not directly affect the survivability and well-being of the patients. However, families and clinicians are responsible for making the final decision on the selection of the weights based on patients’ specific conditions. As a result, careful consideration should be given to each criteria before assigning weight. In this study, weight assignments were decided upon according to the several experts’ opinion. It is also necessary to note that computing the outranking ratio depends on the weight assigned to the criteria. Table 2 shows the linguistic variables and their corresponding fuzzy numbers, while Table 3 shows the preferences for each criteria and child.
| Priority weight of criteria | Fuzzy number | Rating of criteria for Child A | Rating of criteria Child B |
|---|---|---|---|
| VH | (0.75, 1.00, 1.00) | Efficacy on depression, efficacy on aggression | Cost per session, efficacy on speech, duration of treatment |
| H | (0.50, 0.75, 1.00) | Safety, side effect, efficacy on social functioning | Safety, side effect |
| M | (0.25, 0.50, 0.75) | Duration of action | Duration of action |
| L | (0, 0.25, 0.50) | Duration of treatment | Efficacy on social functioning |
| VL | (0, 0, 0.25) | Cost per session, efficacy on speech | Efficacy on depression, efficacy on aggression |
H: High; L: Low; M: Medium; VH: Very high; VL: Very low.
| Criteria | Min/max | Weight (Child A) | Weight (Child B) |
|---|---|---|---|
| Cost per session | Min | 0.08 | 0.92 |
| Duration of treatment | Min | 0.25 | 0.92 |
| Efficacy on social/behavioral functioning | Max | 0.75 | 0.25 |
| Efficacy on irritability/aggression | Max | 0.92 | 0.08 |
| Efficacy on depression | Max | 0.92 | 0.08 |
| Efficacy on speech | Max | 0.08 | 0.92 |
| Side effects | Min | 0.75 | 0.75 |
| Safety | Max | 0.75 | 0.75 |
| Duration of action | Min | 0.50 | 0.50 |
Fuzzy TOPSIS application
TOPSIS is one of the other effectively used MCDM technique constructed by Hwang and Yoon in 1981 [48]. It aims to rank the alternatives even with conflicting criteria based on the relative closeness to the positive ideal solution (PIS) by assuming that the decision-maker wants a solution, which is closest to the PIS and furthest to the negative ideal solution (NIS). In this technique, PIS contains the combination of the best points for each criteria where the NIS contains the combination of the worst points for each criteria. It can be applied only for the numerical data. Therefore, the non-numerical data need a preprocess before applying to TOPSIS technique. Even though the fuzzy sets have been presented by Zadeh in 1965, fuzzy-based MCDM techniques gained the popularity since 2000s, and these approaches have been applied in many fields after. Fuzzy sets enable to decision-maker convert the vague or linguistic data to numerical data to be used in analytical techniques [49].
The process of TOPSIS technique can be summarized as following steps:
Step 1. Construction of the decision matrix
In this step, the data of the alternatives and their features (criteria) should be determined in matrix form and the importance weights of each criteria (wj) should be specified by the decision-maker/s where . The decision-making problem with m – alternatives with n – criteria should be expressed as mχn matrix (X) in size as shown below.
(Eq. 1)
Step 2. Normalization of the decision matrix
The commonly used normalization technique is given in Equation (1). And normalization of the decision matrix can be calculated by Equation (2).
(Eq. 2)
Step 3. Determination of the normalized weighted matrix
The weighted normalized matrix values (vij) can be obtained by multiplying the normalized values of alternatives with their corresponding criteria importance weights as shown in Equation (3).
(Eq. 3)
where i = 1, …., m; j = 1, …., n.
Step 4. Calculation of the PIS & NIS
PIS (A+) and the NIS (A−) can be obtained based on the Equations (4) and (5), respectively.
(Eq. 4)
(Eq. 5)
where i = 1, …., m; j = 1, …., n.
i denotes the criteria aims to maximization and j denotes the criteria aims to minimization.
Step 5. Calculation of the distance to PIS & NIS
The calculation of the distances of the alternatives to the PIS ( ) and NIS ( ) can be determined with the help of the following Equations (6) and (7), which are defined based on the n-dimensional Euclidean metric.
(Eq. 6)
(Eq. 7)
Step 6. Calculation of the relative closeness to PIS (Ri)
The relative distances of the alternatives to the PIS can be calculated with the help of the following Equation (8), and the highest R-value represents the most ideal alternative.
(Eq. 8)
The ranking of the alternatives can be obtained based on their R-values. In fuzzy TOPSIS analysis, we used the same data and the importance weights as used in fuzzy PROMETHEE analysis. The distance between the weighted normalized values of the alternatives and PIS (d+) and the distance between the weighted normalized values of the alternatives and NIS (d-) has been calculated and shown in Table 4.
| Criteria | Min/max | Child A A+/A- | Child B A+/A- |
|---|---|---|---|
| Cost per session | Min | 0.0054/0.0081 | 0.0602/0.0902 |
| Duration of treatment | Min | 0.0159/0.0293 | 0.0566/0.1041 |
| Efficacy on social/behavioral functioning | Max | 0.0894/0.0243 | 0.0288/0.0078 |
| Efficacy on irritability/aggression | Max | 0.1127/0.0306 | 0.0095/0.0026 |
| Efficacy on depression | Max | 0.0920/0.0000 | 0.0077/0.0000 |
| Efficacy on speech | Max | 0.0013/0.0000 | 0.1481/0.0000 |
| Side effects | Min | 0.0000/0.1061 | 0.0000/0.1026 |
| Safety | Max | 0.0792/0.0430 | 0.0766/0.0416 |
| Duration of action | Min | 0.0192/0.0577 | 0.0186/0.0558 |
Results & discussion
The net flow shows the complete ranking results for the therapy techniques. Integration of therapy techniques has been facilitated by the embedded decision lab scenario ranking table. The resulting ranking for the therapy techniques indicates that ABA is the most preferred technique, followed by CBT, speech therapy, Aripiprazole and Risperidone for Child A, as presented in Table 5.
| Ranking | Therapy techniques | Positive outranking flow | Negative outranking flow | Net flow |
|---|---|---|---|---|
| 1 | ABA | 0.0554 | 0.0216 | 0.0338 |
| 2 | CBT | 0.0458 | 0.0201 | 0.0257 |
| 3 | Speech therapy | 0.0337 | 0.0416 | -0.0079 |
| 4 | Risperidone | 0.0352 | 0.0605 | -0.0253 |
| 5 | Aripiprazole | 0.0346 | 0.0609 | -0.0263 |
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy; PROMETHEE: Preference ranking organization method for enrichment evaluation.
Figure 1 shows the positive and negative aspects of each technique based on the assigned weights. As can be observed from Figure 1, ABA has a wide positive standing for efficacy on social functioning, safety, side effects and duration of action and a narrow negative standing for cost per session, duration of treatment and efficacy on irritability. This explanation applies to the remaining therapy techniques.

Figure 1. Positive and negative aspects of each treatment options for autism spectrum disorder for Child A.
The higher the criteria stands in the graph on the positive side, the higher it contributes to the positive side of the technique. Similarly, the lower the criteria stands in the graph on the negative side, the higher it contributes to the negative side of the technique.
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy.
The outcome for Child B is different from that of Child A. In this scenario, speech therapy came first in the ranking, followed by CBT and ABA, while Risperidone and Aripiprazole were the least preferred techniques (Table 6). Similarly, Figure 2 shows the positive and negative aspects of each therapy techniques. It should be noted that the criteria on positive and negative sides of the graph in Figures 1 and 2 for speech therapy do not change, their order in the graph changes due to the given different weights to different criteria.
| Ranking | Therapy techniques | Positive outranking flow | Negative outranking flow | Net flow |
|---|---|---|---|---|
| 1 | Speech therapy | 0.0368 | 0.0109 | 0.0258 |
| 2 | CBT | 0.0309 | 0.0144 | 0.0164 |
| 3 | ABA | 0.0399 | 0.0290 | 0.0109 |
| 4 | Risperidone | 0.0240 | 0.0445 | -0.0206 |
| 5 | Aripiprazole | 0.0168 | 0.0493 | -0.0326 |
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy; PROMETHEE: Preference ranking organization method for enrichment evaluation.

Figure 2. Positive and negative aspects of each treatment option for autism spectrum disorder for Child B.
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy.
With the fuzzy PROMETHEE method, clinicians are assured of making a better decision regarding the most effective therapy technique based on collective weight and preference selection either with the family of the patient or with the health team personnel. Tables 7 and 8 below show the results obtained with fuzzy TOPSIS. As it is seen from Tables 7 and 8 for Child A and Child B, respectively, almost the same results as fuzzy PROMETHEE were obtained with fuzzy TOPSIS. The complete ranking of the alternatives has been obtained based on the relative closeness to the PIS (Ri).
| Ranking | Therapy techniques | d+ | d- | Ri |
|---|---|---|---|---|
| 1 | ABA | 0.0534 | 0.1551 | 0.7439 |
| 2 | CBT | 0.0572 | 0.1502 | 0.7244 |
| 3 | Speech therapy | 0.1261 | 0.1948 | 0.6070 |
| 4 | Risperidone | 0.1359 | 0.1291 | 0.4873 |
| 5 | Aripiprazole | 0.1359 | 0.1290 | 0.4870 |
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy; TOPSIS: Technique for order preference by similarity to ideal solution.
| Ranking | Therapy techniques | d+ | d- | Ri |
|---|---|---|---|---|
| 1 | Speech therapy | 0.0358 | 0.1885 | 0.8403 |
| 2 | ABA | 0.0750 | 0.1529 | 0.6710 |
| 3 | CBT | 0.1549 | 0.1139 | 0.4237 |
| 4 | Risperidone | 0.1884 | 0.0572 | 0.2329 |
| 5 | Aripiprazole | 0.1908 | 0.0478 | 0.2032 |
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy; TOPSIS: Technique for order preference by similarity to ideal solution.
It should be noted that although the ABA tops the list of other therapy techniques for Child A, based on the need, condition and alternatives presented, preferred therapy technique for the management of ASD may differ from one decision-maker to another, as seen for the example in Child B. In addition, more than one therapeutic technique may be combined for better treatment results. The result obtained from this study does not denote a standard but shows the applicability of the method. Therefore, the result obtained by a decision-maker is not a standard as different decision-makers may come up with different solutions depending on some factors such as cost, duration of treatment, side effect, safety and duration of action. As mentioned earlier, the cost of treatment for autistic patients in the USA is expected to increase to US$461 billion by 2025. A family having a child with ASD might not always afford the high treatment cost. Therefore, cost-effective analysis for Child B was performed by setting the weight of cost to a very high importance. A clinician/pediatrician’s opinion is essential for the analysis, in terms of selecting the criteria and assigning the weight of each criteria. More research and clinical trials are needed before translating this method to the clinical settings.
Sensitivity analysis
Here, we determined how different values of an independent variable affect a particular dependent variable under a given set of assumptions. In other words, it measures how various sources of uncertainty in a mathematical model contribute to the model’s overall uncertainty [50]. To apply sensitivity analysis, the weight of a criteria is altered, leaving the weights of other criteria unaltered. Hence, sensitivity analysis was done by marginally changing the values of a criteria weight from the original value as shown in Table 3 into a new value (see Table 9) and then examining that the impact that the singular decision will have on the overall result.
| Criteria | Min/max | Weight (Child A) |
|---|---|---|
| Cost per session | Min | 0.08 |
| Duration of treatment | Min | 0.25 |
| Efficacy on social/behavioral functioning | Max | 0.75 |
| Efficacy on irritability/aggression | Max | 0.92 |
| Efficacy on depression | Max | 0.92 |
| Efficacy on speech | Max | 0.08 |
| Side effects | Min | 0.08 |
| Safety | Max | 0.75 |
| Duration of action | Min | 0.50 |
The boldfaced value indicates the altered value for the sensitivity analysis, the side effect.
For this analysis, the weight of side effect was altered and changed from 0.75 to 0.08, whereas the weight for other criteria such as cost per session, duration of treatment, side effect, safety and duration of action remained unchanged. This indicates a slight change in the sensitivity of one criteria to another. The result derived from this alteration as shown in Table 10 showed that even with that degree of alteration, the results remained the same with just a slight change in the values of positive, negative and net flow ranking. CBT remained the most preferred treatment technique.
| Ranking | Therapy techniques | Positive outranking flow | Negative outranking flow | Net flow |
|---|---|---|---|---|
| 1 | ABA | 0.0617 | 0.0244 | 0.0373 |
| 2 | CBT | 0.0509 | 0.0227 | 0.0281 |
| 3 | Speech therapy | 0.0372 | 0.0470 | -0.0098 |
| 4 | Risperidone | 0.0397 | 0.0670 | -0.0273 |
| 5 | Aripiprazole | 0.0391 | 0.0674 | -0.0283 |
ABA: Applied behavioral analysis; CBT: Cognitive behavioral therapy; PROMETHEE: Preference ranking organization method for enrichment evaluation.
Limitations
Assigning weights to criteria and grading alternatives are the major challenges in any study pertaining to MCDM. This is mainly due to the differences in the priorities of the decision-makers, healthcare providers, consultants etc. Sometimes, a decision-maker’s priority may not be same and this will result in giving a weight they deem to be appropriate to a criteria regardless of the opinions of other decision-makers. Placing a standard would have been the solution to this disparity, but it is frequently observed that patients may exhibit different and unfamiliar responses to treatment. This warrants flexibility from the side of the decision-maker to ensure the adverse effects do not outweigh the benefits. In addressing this, sensitivity testing is always required.
In addition, individualized interventions are important in clinical setting. For example, although CBT had been demonstrated to top the list; it should be kept in mind that its applicability is very limited in intellectually disabled cases. Likewise, Aripiprazole cannot be used in a case having severe side effects of the drug like dyskinesia. In addition, improvement of domains other than irritability cannot be expected.
Conclusion
Because ASD has no cure, selecting the best therapy technique for the management of the disorder has become a necessity for therapists, parents, health institutions and researchers with an interest in neurodevelopmental disorders. This study utilized MCDM methods, fuzzy PROMETHEE and fuzzy TOPSIS. The outcome of the study indicates that ABA, CBT and speech therapy are preferred treatment alternatives, while Risperidone and Aripiprazole are to be the least effective alternatives. However, more alternatives can be included depending on the decision-maker’s preference. The proposed method is unique and can consolidate both qualitative and quantitative data. It has also been demonstrated to be efficient, feasible, user-friendly and practical in decision-making for the caregiver. Other MCDM approaches could be used for comparison in future studies. Further analyses are needed to obtain more accurate patient-specific results that will incorporate specific patient demographics and data, as well as the combination of two or three techniques.
•
Autism spectrum disorder (ASD) has no known cause or cure; however, the effective management of the neurological disorder is essential for the survival of the patient.
•
Based on the assigned weight to the criteria of therapy techniques and specific patient profile in this study, applied behavioral analysis, cognitive behavioral therapy and speech therapy prove to be the most preferred therapy technique for the management of ASD.
•
Mathematical computation and at large multicriteria decision-making algorithm plays a crucial role in therapeutic neuroscience.
•
Based on the need, condition and alternatives presented, preferred therapy technique for the management of ASD may differ from one decision-maker to another.
•
More than one therapeutic technique may be combined for better treatment results.
•
Pharmacological therapy does not only pose an adverse effect but also cost more when compared with the other therapeutic techniques.
•
The method applied in this study can accommodate and use both quantitative and qualitative data. This makes possible a holistic evaluation of the therapy techniques.
•
The result obtained from the study does not denote a standard but shows the applicability of the method.
•
A clinician/pediatrician’s opinion is essential for the analysis, in terms of defining and assigning the weight of each criteria.
•
More research and clinical trials are needed before translating this method to the clinical setting.
Financial & competing interests disclosure
The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
No writing assistance was utilized in the production of this manuscript.
References
1.
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. American Psychiatric Association, VA, USA (2013).
2.
Risi S, Lord C, Gotham K et al. Combining information from multiple sources in the diagnosis of autism spectrum disorders. J. Am. Acad. Child Adol. Psych. 45(9), 1094–1103 (2006).
3.
Wu S, Wu F, Ding Y, Hou J, Bi J, Zhang Z. Advanced parental age and autism risk in children: a systematic review and meta-analysis. Acta Psychiatr. Scand. 135(1), 29–41 (2016).
4.
Modabbernia A, Velthorst E, Reichenberg A. Environmental risk factors for autism: an evidence-based review of systematic reviews and meta-analyses. Mol. Autism 8, 13 (2017).
5.
Sanders S, He X, Willsey A et al. Insights into autism spectrum disorder genomic architecture and biology from 71 risk loci. Neuron 87(6), 1215–1233 (2015).
6.
Satterstrom F, Kosmicki J, Wang J et al. Large-scale exome sequencing study implicates both developmental and functional changes in the neurobiology of autism. Cell 180(3), 568–584.e23 (2020).
7.
Cross-Disorder Group of the Psychiatric Genomics Consortium. Genetic relationship between five psychiatric disorders estimated from genome-wide SNPs. Nature Genet. 45(9), 984–994 (2013).
8.
Gaugler T, Klei L, Sanders S et al. Most genetic risk for autism resides with common variation. Nature Genet. 46(8), 881–885 (2014).
9.
Baxter A, Brugha T, Erskine H, Scheurer R, Vos T, Scott J. The epidemiology and global burden of autism spectrum disorders. Psychol. Med. 45(3), 601–613 (2014).
10.
Christensen D, Maenner M, Bilder D et al. Prevalence and characteristics of autism spectrum disorder among children aged 4 years – Early Autism and Developmental Disabilities Monitoring Network, Seven Sites, United States, 2010, 2012, and 2014. MMWR Surveill. Summ. 68(2), 1–19 (2019).
11.
Tromans S, Chester V, Kiani R, Alexander R, Brugha T. The prevalence of autism spectrum disorders in adult psychiatric inpatients: a systematic review. Clin. Pract. Epidemiol. Mental Health 14(1), 177–187 (2018).
12.
Schipper E, Lundequist A, Coghill D et al. Ability and disability in autism spectrum disorder: a systematic literature review employing the International Classification of Functioning, Disability and Health-Children and Youth Version. Autism Res. 8(6), 782–794 (2015).
13.
Halfon N, Forrest C, Lerner R, Faustman E, Tullis E, Son J. Introduction to the Handbook of Life Course Health Development. Springer, New York, NY, 1–16 (2020).
14.
Howlin P, Goode S, Hutton J, Rutter M. Adult outcome for children with autism. J. Child Psychol. Psych. 45(2), 212–229 (2004).
15.
Lai M, Kassee C, Besney R et al. Prevalence of co-occurring mental health diagnoses in the autism population: a systematic review and meta-analysis. Lancet Psych. 6(10), 819–829 (2019).
16.
Havdahl A, Bishop S. Heterogeneity in prevalence of co-occurring psychiatric conditions in autism. Lancet Psych. 6(10), 794–795 (2019).
17.
Mannion A, Leader G, Healy O. An investigation of comorbid psychological disorders, sleep problems, gastrointestinal symptoms and epilepsy in children and adolescents with autism spectrum disorder. Res. Autism Spectrum Disord. 7(1), 35–42 (2013).
18.
Soke G, Maenner M, Christensen D, Kurzius-Spencer M, Schieve L. Prevalence of co-occurring medical and behavioral conditions/symptoms among 4- and 8-year-old children with autism spectrum disorder in selected areas of the United States in 2010. J. Autism Develop. Disord. 48(8), 2663–2676 (2018).
19.
Chandler S, Howlin P, Simonoff E et al. Emotional and behavioural problems in young children with autism spectrum disorder. Develop. Med. Child Neurol. 58(2), 202–208 (2015).
20.
Pezzimenti F, Han G, Vasa R, Gotham K. Depression in youth with autism spectrum disorder. Child Adol. Psych. Clin. N. Am. 28(3), 397–409 (2019).
21.
Lord C, Brugha T, Charman T et al. Autism spectrum disorder. Nature Rev. Dis. Primers 6(1), 5 (2020).
22.
McCracken J, McGough J, Shah B et al. Risperidone in children with autism and serious behavioral problems. N. Engl. J. Med. 347(5), 314–321 (2002).
23.
Owen R, Sikich L, Marcus R et al. Aripiprazole in the treatment of irritability in children and adolescents with autistic disorder. Pediatrics 124(6), 1533–1540 (2009).
24.
McPheeters M, Warren Z, Sathe N et al. A systematic review of medical treatments for children with autism spectrum disorders. Pediatrics 127(5), e1312–e1321 (2011).
25.
Leigh JP, Du J. Brief Report: forecasting the economic burden of autism in 2015 and 2025 in the United States. J. Autism Dev. Disord. 45, 4135–4139 (2015).
26.
Ozsahin I, Abebe ST, Mok GS. A multi-criteria decision-making approach for schizophrenia treatment techniques. Arch. Psych. Psychother. 22(2), 52–61 (2020).
27.
Ozsahin I. Identifying a personalized anesthetic with fuzzy PROMETHEE. Healthcare Informatics Res. 26(3), 201–211 (2020).
28.
Hill A, Zuckerman K, Hagen A et al. Aggressive behavior problems in children with autism spectrum disorders: prevalence and correlates in a large clinical sample. Res. Autism Spectrum Disord. 8(9), 1121–1133 (2014).
29.
Stahmer AC, Schreibman L, Cunningham AB. Toward a technology of treatment individualization for young children with autism spectrum disorders. Brain Res. 1380, 229–239 (2011).
30.
NIMH. Autism spectrum disorder. (2020). https://www.nimh.nih.gov/health/topics/autism-spectrum-disorders-asd/index.shtml
31.
Williams K, Brignell A, Randall M, Silove N, Hazell P. Selective serotonin reuptake inhibitors (SSRIs) for autism spectrum disorders (ASD). Cochrane Database Syst. Rev. (8), https://doi.org/10.1002/14651858.CD004677.pub3 (2013).
32.
Green J, Charman T, McConachie H et al. Parent-mediated communication-focused treatment in children with autism (PACT): a randomised controlled trial. Lancet 375(9732), 2152–2160 (2010).
33.
Lequia J, Machalicek W, Rispoli M. Effects of activity schedules on challenging behavior exhibited in children with autism spectrum disorders: a systematic review. Res. Autism Spectrum Disord. 6(1), 480–492 (2012).
34.
Rogers S, Vismara L. Evidence-based comprehensive treatments for early autism. J. Clin. Child Adol. Psychol. 37(1), 8–38 (2008).
35.
Kirino E. Efficacy and tolerability of pharmacotherapy options for the treatment of irritability in autistic children. Clin. Med. Insights Pediatr. 8, 17–30 (2014).
36.
Hasnain M, Vieweg W, Hollett B. Weight gain and glucose dysregulation with second-generation antipsychotics and antidepressants: a review for primary care physicians. Postgrad. Med. 124(4), 154–167 (2012).
37.
Yin X. A review for existing complementary and alternative medical therapies for autism spectrum disorder. J. Psychol. Res. 2(4), (2020).
38.
Goumas M, Lygerou V. An extension of the PROMETHEE method for decision making in fuzzy environment: ranking of alternative energy exploitation projects. Eur. J. Operational Res. 123(3), 606–613 (2000).
39.
Ülengin F, İlker Topcu Y, Şahin Ş. An integrated decision aid system for Bosphorus water-crossing problem. Eur. J. Operational Res. 134(1), 179–192 (2001).
40.
Macharis C, Springael J, De Brucker K, Verbeke A. PROMETHEE and AHP: the design of operational synergies in multicriteria analysis. Eur. J. Operational Res. 153(2), 307–317 (2004).
41.
Geldermann J, Spengler T, Rentz O. Fuzzy outranking for environmental assessment. Case study: iron and steel making industry. Fuzzy Sets Systems. 115(1), 45–65 (2000).
42.
Ozsahin D, Isa N, Uzun B, Ozsahin I. Effective analysis of image reconstruction algorithms in nuclear medicine using fuzzy PROMETHEE. 2018 Advances in Science and Engineering Technology International Conferences (ASET). United Arab Emirates (2018).
43.
Ozsahin D, Uzun B, Musa M, Şentürk N, Nurçin F, Ozsahin I. Evaluating nuclear medicine imaging devices using fuzzy PROMETHEE method. Proc. Comp. Sci. 120, 699–705 (2017).
44.
Ozsahin I, Uzun Ozsahin D, Maisaini M, Mok G. Fuzzy PROMETHEE analysis of leukemia treatment techniques. World Cancer Res. J. 6, e1315 (2020).
45.
Uzun D, Uzun B, Sani M et al. Evaluating cancer treatment alternatives using fuzzy PROMETHEE method. Int. J. Adv. Comp. Sci. App. 8(10), 177–182 (2017).
46.
Ozsahin D, Ozsahin I. A fuzzy PROMETHEE approach for breast cancer treatment techniques. IJMRHS 7(5), 29–32 (2018).
47.
Gokcekus H, Ozsahin D, Mustapha M. Simulation and evaluation of water sterilization devices. Desalination Water Treat. 177, 431–436 (2020).
48.
Hwang CL, Yoon K. Multiple Attribute Decision Making: Methods and Applications. Springer-Verlag, NY, USA (1981).
49.
Palczewski K, Sałabun W. The fuzzy TOPSIS applications in the last decade. Proc. Comp. Sci. 159, 2294–2303 (2019).
50.
Schey C, Connolly M. Multi-criteria decision analysis (MCDA): testing a proposed MCDA model for orphan drugs. Value Health 17(7), A556 (2014).
Information & Authors
Information
Published In
Pages: 423 - 437
PubMed: 33709772
Copyright
© 2021 Future Medicine Ltd.
History
Received: 3 August 2020
Accepted: 27 January 2021
Published online: 12 March 2021
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
An investigation to choose the proper therapy technique in the management of autism spectrum disorder. (2021) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2020-0162
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Dr. N. Hinduja, Dr. Gunda Sridhar, Dr. Sneha Bevinamarad, Effectiveness of Individualized Homoeopathic Remedies in Managing Autism Spectrum Disorder: A Pilot Study, International Journal of Research and Review, 10.52403/ijrr.20260457, (552), (2026).
- Xing Wang, Rui-Xuan Zhu, Xiao-Gang Du, Hu-jie Song, Applying Target Trial Emulation to Evaluate Acupuncture Combined with Rehabilitation for Autism Spectrum Disorder in Children: A Retrospective Single-Center Real-World Protocol, Neuropsychiatric Disease and Treatment, 10.2147/NDT.S563199, Volume 21, (2513-2523), (2025).
- Zining DING, A Holistic Educational Therapy Approach for Autism and Comorbid Emotional-Behavioral Disorders, The Asian Educational Therapist, 10.64663/aet.7, 3, 3, (3-16), (2025).
- Nabilah Abughazalah, Majid Khan, An intelligent fuzzy-neural framework for autism sensory assessment using hierarchical linguistic modeling and risk-based temporal decision-making, Scientific Reports, 10.1038/s41598-025-15730-1, 15, 1, (2025).
- Md. Al Imran, Md. Saiful Islam, Md. Sharif Hossain, Shahina Pardhan, Nakiba Bari, Zebunnesa Zeba, Knowledge and practice among caregivers having children with autism in Bangladesh: findings from a cross-sectional study, BMC Research Notes, 10.1186/s13104-024-07074-2, 18, 1, (2025).
- Efe Precious Onakpojeruo, Berna Uzun, Leena R. David, Ilker Ozsahin, Dilber Uzun Ozsahin, Selection Techniques in Genetic Algorithm, 2024 17th International Conference on Development in eSystem Engineering (DeSE), 10.1109/DeSE63988.2024.10912015, (411-416), (2024).
- Mary Jane Samonte, Kennichi Nitta, Caitlin Datu, Rosanne Erpelo, Critical Analysis on the Design and Development of an E-Learning System for Autism Spectrum Disorder Intervention through Gamified Augmented Reality and Emotion-Recognition, Proceedings of the 2024 7th International Conference on Information Management and Management Science, 10.1145/3695652.3695658, (400-409), (2024).
- Ayesha Kanwal, Kashif Javed, Sara Ali, Muhammad Attique Khan, Shrooq Alsenan, Areej Alasiry, Mehrez Marzougui, Saddaf Rubab, ALATT-network: automated LSTM-based framework for classification and monitoring of autism spectrum disorder therapy tasks, Signal, Image and Video Processing, 10.1007/s11760-024-03540-3, 18, 12, (9205-9221), (2024).
- Dilber Uzun Ozsahin, Declan Ikechukwu Emegano, Riad Alsabbagh, Basil Bartholomew Duwa, Ilker Ozsahin, Temperature reducer machine, Practical Design and Applications of Medical Devices, 10.1016/B978-0-443-14133-1.00005-7, (243-254), (2024).
- Ilker Ozsahin, Efe Precious Onakpojeruo, Berna Uzun, Dilber Uzun Ozsahin, Tracy A. Butler, A Multi-Criteria Decision Aid Tool for Radiopharmaceutical Selection in Tau PET Imaging, Pharmaceutics, 10.3390/pharmaceutics15041304, 15, 4, (1304), (2023).
